Upcoming updates to the AWS Lambda and AWS Lambda@Edge execution environment

NOTE: On May 23 we announced an update to the timeframe seen below.

AWS Lambda was first announced at AWS re:Invent 2014. Amazon CTO Werner Vogels highlighted the aspect of needing to run no servers, no instances, nothing, you just write your code. In 2016, we announced the launch of Lambda@Edge, which lets you run Lambda functions to customize content that CloudFront delivers, executing the functions in AWS locations closer to the viewer.

At AWS, we talk often about “shared responsibility” models. Essentially, those are the places where there is a handoff between what we as a technology provider offer you and what you as the customer are responsible for. In the case of Lambda and Lambda@Edge, one of the key things that we manage is the “execution environment.” The execution environment is what your code runs inside of. It is composed of the underlying operating system, system packages, the runtime for your language (if a managed one), and common capabilities like environment variables. From the customer standpoint, your primary responsibility is for your application code and configuration.

In this post, we outline an upcoming change to the execution environment for Lambda and Lambda@Edge functions for all runtimes with the exception of Node.js v10. As with any update, some functionality could be affected. We encourage you to read through this post to understand the changes and any actions that you might need to take.

Update overview

AWS Lambda and AWS Lambda@Edge run on top of the Amazon Linux operating system distribution and maintain updates to both the core OS and managed language runtimes. We are updating the Lambda execution environment AMI to version 2018.03 of Amazon Linux. This newer AMI brings updates that offer improvements in capabilities, performance, security, and updated packages that your application code might interface with.

This does not apply to the recently announced Node.js v10 runtime which today runs on Amazon Linux 2.

The majority of functions will benefit seamlessly from the enhancements in this update without any action from you. However, in rare cases, package updates may introduce compatibility issues. Potential impacted functions may be those that contain libraries or application code compiled against specific underlying OS packages or other system libraries. If you are primarily using an AWS SDK or the AWS X-Ray SDK with no other dependencies, you will see no impact.

You have the following options in terms of next steps:

Take no action before the automatic update of the execution environment starting June 11 for all newly created functions, June 25th for all updates to previously existing functions and July 16 for all existing functions.
Proactively test your functions against the new environment starting today.
Configure your functions to delay the execution environment update until July 23 to allow for a longer testing window.

In addition to the overall timeline for this change, this post also provides instructions on the following:

How to test your functions for this new execution environment locally and on Lambda/Lambda@Edge.
How to proactively update your functions.
How to delay the migration until July 23.

Update timeline

The following is the timeline for the update, which is broken up over five phases:

May 14, 2019 – Begin Testing: You can begin testing your functions for the new execution environment locally with AWS SAM CLI or using an Amazon EC2 instance running on Amazon Linux 2018.03. You can also proactively enable the new environment in AWS Lambda using the opt-in mechanism described in the original announcement post.
June 11, 2019 – New Function Create: All newly created functions will result in those functions running on the new execution environment, unless they have a delayed-update layer configured.
June 25, 2019 – Existing Function Update: All newly created functions or existing functions that you update will result in those functions running on the new execution environment, unless they have a delayed-update layer configured.
July 16, 2019 – General Update: Existing functions begin using the new execution environment on invoke, unless they have a delayed-update layer configured.
July 23, 2019 – Delayed Update End: All functions with a delayed-update layer configured start being migrated automatically.
July 29, 2019 – Migration End: All functions have been migrated over to the new execution environment.

Recommended Approach

decision tree

You only have to act if your application uses dependencies that are compiled to work on the previous execution environment. Otherwise, you can continue to deploy new and updated Lambda functions without needing to perform any other testing steps. For those who aren’t sure if their functions use such dependencies, we encourage you to do a new deployment of your functions and to test their functionality.

There are two options for when you can start testing your functions on the new execution environment:

You can begin testing today using the opt-in mechanism described later.
Starting June 11, all newly created functions use the new execution environment.

If you confirm that your functions would be affected by the new execution environment, you can begin re-compiling or building your dependencies using the new reference AMI for the execution environment today and then repeating the testing. The final step for existing applications is to redeploy them after June 25 to use the new execution environment or to deploy them with the opt-in configured starting today.

Building your dependencies and application for the new execution environment

Because we are basing the environment off of an existing Amazon Linux AMI, you can start with building and testing your code against that AMI on EC2. With an updated EC2 instance running this AMI, you can compile and build your packages using your normal processes. For the list of AMI IDs in all public Regions, check the release notes. To start an EC2 instance running this AMI, follow the steps in the Launching an Instance Using the Launch Instance Wizard topic in the Amazon EC2 User Guide.

Opt-in/Delayed-Update with Lambda layers

Some of you may want to begin testing as soon as you’ve read this announcement. Some know that they should postpone until later in the timeline.

To give you some control over testing, we’re releasing two special Lambda layers. Lambda layers can be used to provide shared resources, code, or data between Lambda functions and can simplify the deployment and update process. These layers don’t actually contain any data or code. Instead, they act as a special flag to Lambda to run your function executions either specifically on the new or old execution environment.

The Opt-In layer allows you to start testing today. You can use the Delayed-Update layer when aren’t ready to deploy to the new execution environment, but you know that you need to create new functions or make updates to your existing functions or their configuration after June 11. The Delayed-Update layer extends the initial period available to you to deploy your functions by one week until the end of July 22, without changing the execution environment.

Neither layer brings any performance or runtime changes beyond this. After July 22, the layers will have no functionality. In a future deployment, you should remove them from any function configurations.

The ARNs for the two scenarios:

To OPT-IN to the update to the new execution environment, add the following layer:

arn:aws:lambda:::awslayer:AmazonLinux1803

To DELAY THE UPDATE to the new execution environment until July 22, add the following layer:

arn:aws:lambda:::awslayer:AmazonLinux1703

The action for adding a layer to your existing functions requires an update to the Lambda function’s configuration. You can do this with the AWS CLI, AWS CloudFormation or AWS SAM, popular third-party frameworks, the AWS Management Console, or an AWS SDK.

Validating your functions

There are several ways for you to test your function code and assure that it will work after the execution environment has been updated.

Local testing

We’re providing an update to the AWS SAM CLI to enable you to test your functions locally against this new execution environment. The AWS SAM CLI uses a Docker image that mirrors the live Lambda environment locally wherever you do development. To test against this new update, make sure that you have the most recent update to AWS SAM CLI version 0.16.0. You also should have an AWS SAM template configured for your function.

Install or update the AWS SAM CLI:

$ pip install --upgrade aws-sam-cli

-Or-

$ pip install aws-sam-cli

Confirm that you have a valid AWS SAM template:
```
$ sam validate -t <template file name>
```
If you don’t have a valid AWS SAM template, you can begin with a basic template to test your functions. The following example represents the basic needs for running your function against a variety. The Runtime value must be listed in the AWS Lambda Runtimes topic.
```
AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::Serverless-2016-10-31'

Resources:
  myFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      CodeUri: ./ 
      Handler: YOUR_HANDLER
      Runtime: YOUR_RUNTIME
```

With a valid template, you can begin testing your function with mock event payloads. To generate a mock event payload, you can use the AWS SAM CLI local generate-event command. Here is an example of that command being run to generate an Amazon S3 notification type of event:

sam local generate-event s3 put --bucket munns-test --key somephoto.jpeg

{
  "Records": [
    {
      "eventVersion": "2.0", 
      "eventTime": "1970-01-01T00:00:00.000Z", 
      "requestParameters": {
        "sourceIPAddress": "127.0.0.1"
      }, 
      "s3": {
        "configurationId": "testConfigRule", 
        "object": {
          "eTag": "0123456789abcdef0123456789abcdef", 
          "sequencer": "0A1B2C3D4E5F678901", 
          "key": "somephoto.jpeg", 
          "size": 1024
        }, 
        "bucket": {
          "arn": "arn:aws:s3:::munns-test", 
          "name": "munns-test", 
          "ownerIdentity": {
            "principalId": "EXAMPLE"
          }
        }, 
        "s3SchemaVersion": "1.0"
      }, 
      "responseElements": {
        "x-amz-id-2": "EXAMPLE123/5678abcdefghijklambdaisawesome/mnopqrstuvwxyzABCDEFGH", 
        "x-amz-request-id": "EXAMPLE123456789"
      }, 
      "awsRegion": "us-east-1", 
      "eventName": "ObjectCreated:Put", 
      "userIdentity": {
        "principalId": "EXAMPLE"
      }, 
      "eventSource": "aws:s3"
    }
  ]
}

You can then use the AWS SAM CLI local invoke command and pipe in the output from the previous command. Or, you can save the output from the previous command to a file and then pass in a reference to the file’s name and location with the -e flag. Here is an example of the pipe event method:

sam local generate-event s3 put --bucket munns-test --key somephoto.jpeg | sam local invoke myFunction
2019-02-19 18:45:53 Reading invoke payload from stdin (you can also pass it from file with --event)
2019-02-19 18:45:53 Found credentials in shared credentials file: ~/.aws/credentials
2019-02-19 18:45:53 Invoking index.handler (python2.7)

Fetching lambci/lambda:python2.7 Docker container image......
2019-02-19 18:45:53 Mounting /home/ec2-user/environment/forblog as /var/task:ro inside runtime container
START RequestId: 7c14eea1-96e9-4b7d-ab54-ed1f50bd1a34 Version: $LATEST
{"Records": [{"eventVersion": "2.0", "eventTime": "1970-01-01T00:00:00.000Z", "requestParameters": {"sourceIPAddress": "127.0.0.1"}, "s3": {"configurationId": "testConfigRule", "object": {"eTag": "0123456789abcdef0123456789abcdef", "key": "somephoto.jpeg", "sequencer": "0A1B2C3D4E5F678901", "size": 1024}, "bucket": {"ownerIdentity": {"principalId": "EXAMPLE"}, "name": "munns-test", "arn": "arn:aws:s3:::munns-test"}, "s3SchemaVersion": "1.0"}, "responseElements": {"x-amz-id-2": "EXAMPLE123/5678abcdefghijklambdaisawesome/mnopqrstuvwxyzABCDEFGH", "x-amz-request-id": "EXAMPLE123456789"}, "awsRegion": "us-east-1", "eventName": "ObjectCreated:Put", "userIdentity": {"principalId": "EXAMPLE"}, "eventSource": "aws:s3"}]}
END RequestId: 7c14eea1-96e9-4b7d-ab54-ed1f50bd1a34
REPORT RequestId: 7c14eea1-96e9-4b7d-ab54-ed1f50bd1a34 Duration: 1 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 14 MB

"Success! Parsed Events"

You can see the full output of your function in the logs that follow the invoke command. In this example, the Python function prints out the event payload and then exits.

With the AWS SAM CLI, you can pass in valid test payloads that interface with data in other AWS services. You can also have your Lambda function talk to other AWS resources that exist in your account, for example Amazon DynamoDB tables, Amazon S3 buckets, and so on. You could also test an API interface using the local start-api command, provided that you have configured your AWS SAM template with events of the API type. Follow the full instructions for setting up and configuring the AWS SAM CLI in Installing the AWS SAM CLI. Find the full syntax guide for AWS SAM templates in the AWS Serverless Application Model documentation.

Testing in the Lambda console

After you have deployed your functions after the start of the Update/Create phase or with the Opt-In layer added, test your functions in the Lambda console.

In the Lambda console, select the function to test.
Select a test event and choose Test.
If no test event exists, choose Configure test events.
1. Choose Event template and select the relevant invocation service from which to test.
2. Name the test event.
3. Modify the event payload for your specific function.
4. Choose Create and then return to step 2.

The results from the test are displayed.

Conclusion

With Lambda and Lambda@Edge, AWS has allowed developers to focus on just application code without the need to think about the work involved managing the actual servers that run the code. We believe that the mechanisms provided and processes described in this post allow you to easily test and update your functions for this new execution environment.

Some of you may have questions about this process, and we are ready to help you. Please contact us through AWS Support, the AWS forums, or AWS account teams.

AWS Compute Blog