How do I create a Lambda layer using a simulated Lambda environment with Docker?

Last updated: 2021-03-12

I want to create an AWS Lambda layer that's compatible with target environment runtimes. How can I do that?

Short description

When you develop Lambda functions, you choose between supported Lambda runtime versions. To create Lambda functions and layers that are compatible with specific runtimes, you can use an AWS Serverless Application Model (AWS SAM) to automatically create the new resources. Or, you can use the AWS SAM's underlying amazon/aws-sam-cli-build-image Docker images to create them.


Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.

The following instructions use Python as an example. For other coding languages, adapt the steps accordingly.

1.    If you haven't already, install Docker for Linux, Windows, or macOS on your computer.

2.    Create the appropriate directory structure and specify your dependencies in the pip requirements file (requirements.txt). For more information, see Requirements files in the pip user guide.

For example, if you want your Lambda layer to be compatible with Python versions 3.6 and 3.8, the structure looks like the following:

├── requirements.txt
└── python/
    └── lib/
        ├── python3.6/
        │   └── site-packages/
        └── python3.8/
            └── site-packages/

Note: The total unzipped size of the function and all layers can't exceed the unzipped deployment package size limit of 250 MB.

For more information about Python version support in Lambda, see Building Lambda functions with Python.

3.    Run the following command once for each runtime that you specified in the directory structure. Doing this installs the libraries compatible with that runtime in the appropriate subfolder.

docker run -v "$PWD":/var/task "" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.6/site-packages/; exit"

Important: Replace 3.6 with 3.7 or 3.8, depending on the compatible libraries that you want to install.

4.    Run the following command to zip the python folder and create the layer package. The .zip file is deflated in the /opt folder of the Lambda environment, and this folder is accessible from your Lambda function code.

zip -r python > /dev/null

Note: Replace mypythonlibs with any name that you prefer.

5.    Run the following command to create or update the layer. Include the compatible runtimes that you specified earlier.

aws lambda publish-layer-version --layer-name mypythonlibs --description "My python libs" --zip-file fileb:// --compatible-runtimes "python3.6" "python3.8"

Important: Replace mypythonlibs with the package name that you used in the previous step. Replace My python libs with any description that you prefer.

In the output, note the layer's Amazon Resource Name (ARN). You need the ARN for step 6.

6.    Run the following command to update your Lambda function configuration to use the layer:

aws lambda update-function-configuration --layers arn:aws:lambda:us-east-2:123456789012:layer:mypythonlibs:1 --function-name my-function

Important: Replace arn:aws:lambda:us-east-2:123456789012:layer:mypythonlibs:1 with the layer ARN from step 5.

(Optional) Use the Docker images for your Lambda package

You can also use the amazon/aws-sam-cli-build-image Docker images directly for your Lambda package, without creating a layer.

1.    Run the following command to get the required versions of your dependencies:

docker run -v "$PWD":/var/task "" /bin/sh -c "pip install -r requirements.txt -t libs; exit"

Important: Replace 3.6 with 3.7 or 3.8, depending on the compatible libraries that you want to install.

2.    Create a Lambda deployment package that includes your function code and the correct libraries.