How do I create a Lambda layer using a simulated Lambda environment with Docker?

Last updated: 2019-12-19

I want to create an AWS Lambda layer that's compatible with target environment runtimes. How do I do that?

Short Description

When you develop Lambda functions, you choose between supported Lambda runtime versions. Using the lambci/lambda Docker images, you can simulate the live Lambda environment and create a layer that's compatible with the runtimes that you specify. For more information, see lambci/lambda on the Docker website.

Keep in mind that lambci/lambda images are not an exact copy of the Lambda environment and some files may be missing.

Note: The AWS Serverless Application Model (AWS SAM) also uses the lambci/lambda Docker images when you run sam local start-api to test a serverless application locally.

Resolution

Note: These instructions use Python as an example. For other languages, adapt the steps accordingly.

1.    If you haven't already, install Docker for Linux, Windows, or MacOS on your computer.

2.    Create the appropriate directory structure and specify your dependencies in the pip Requirements file (requirements.txt). For more information, see Requirements Files in the pip User Guide.

For example, if you want your Lambda layer to be compatible with Python versions 3.6 and 3.8, the structure looks like the following:

├── requirements.txt
└── python/
    └── lib/
        ├── python3.6/
        │   └── site-packages/
        └── python3.8/
            └── site-packages/

Note: The total unzipped size of the function and all layers can't exceed the unzipped deployment package size limit of 250 MB.

For more information about Python version support in Lambda, see Building Lambda Functions with Python.

3.    Run the following command once for each runtime that you specified in the directory structure. This installs the libraries compatible with that runtime in the appropriate subfolder.

Note: Replace 3.6 with 3.7 or 3.8 depending on the compatible libraries that you want to install.

docker run -v "$PWD":/var/task "lambci/lambda:build-python3.6" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.6/site-packages/; exit"

4.    Create the layer package by zipping the python folder. The .zip file will be deflated in the /opt folder of the Lambda environment, directly accessible from your Lambda function code.

Note: Replace mypythonlibs with any name that you prefer.

zip -r mypythonlibs.zip python > /dev/null

5.    Run the following command to create or update the layer. Be sure to include the compatible runtimes that you specified earlier.

Note: Replace mypythonlibs with the package name that you used in the previous step. Replace My python libs with any description that you prefer.

aws lambda publish-layer-version --layer-name mypythonlibs --description "My python libs" --zip-file fileb://mypythonlibs.zip --compatible-runtimes "python2.7" "python3.6" "python3.7"

In the output, note the layer's Amazon Resource Name (ARN). You'll need it for the next step.

6.    Run the following command. This updates your Lambda function configuration to use the layer.

Note: Replace arn:aws:lambda:us-east-2:123456789012:layer:mypythonlibs:1 with the layer ARN from the previous step.

aws lambda update-function-configuration --layers arn:aws:lambda:us-east-2:123456789012:layer:mypythonlibs:1 --function-name my-function

(Optional) Use the Docker images for your Lambda package

You can also use the lambci/lambda Docker images directly for your Lambda package, without creating a layer. Run the following command to get the required versions of your dependencies:

Note: Replace 3.6 with 3.7 or 3.8 depending on the compatible libraries that you want to install.

docker run -v "$PWD":/var/task "lambci/lambda:build-python3.6" /bin/sh -c "pip install -r requirements.txt -t libs; exit"

Then create a Lambda deployment package including your function code and the correct libraries.