How do I create a Lambda layer using a simulated Lambda environment with Docker?

Last updated: 2021-01-07

I want to create an AWS Lambda layer that's compatible with target environment runtimes. How can I do that?

Short description

When you develop Lambda functions, you choose between supported Lambda runtime versions. Using the lambci/lambda Docker images, you can simulate the live Lambda environment and create a layer that's compatible with the runtimes that you specify. For more information, see lambci/lambda on the Docker website.

Keep in mind that lambci/lambda images are not an exact copy of the Lambda environment, and some files might be missing.

Note: The AWS Serverless Application Model (AWS SAM) also uses the lambci/lambda Docker images when you run sam local start-api to test a serverless application locally.

Resolution

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.

The following instructions use Python as an example. For other coding languages, adapt the steps accordingly.

1.    If you haven't already, install Docker for Linux, Windows, or macOS on your computer.

2.    Create the appropriate directory structure and specify your dependencies in the pip requirements file (requirements.txt). For more information, see Requirements files in the pip User guide.

For example, if you want your Lambda layer to be compatible with Python versions 3.6 and 3.8, the structure looks like the following:

├── requirements.txt
└── python/
    └── lib/
        ├── python3.6/
        │   └── site-packages/
        └── python3.8/
            └── site-packages/

Note: The total unzipped size of the function and all layers can't exceed the unzipped deployment package size limit of 250 MB.

For more information about Python version support in Lambda, see Building Lambda Functions with Python.

3.    Run the following command once for each runtime that you specified in the directory structure. Doing this installs the libraries compatible with that runtime in the appropriate subfolder.

docker run -v "$PWD":/var/task "lambci/lambda:build-python3.6" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.6/site-packages/; exit"

Important: Replace 3.6 with 3.7 or 3.8, depending on the compatible libraries that you want to install.

4.    Run the following command to zip the python folder and create the layer package. The .zip file will be deflated in the /opt folder of the Lambda environment, which is accessible from your Lambda function code.

zip -r mypythonlibs.zip python > /dev/null

Note: Replace mypythonlibs with any name that you prefer.

5.    Run the following command to create or update the layer. Make sure to include the compatible runtimes that you specified earlier.

aws lambda publish-layer-version --layer-name mypythonlibs --description "My python libs" --zip-file fileb://mypythonlibs.zip --compatible-runtimes "python2.7" "python3.6" "python3.7"

Important: Replace mypythonlibs with the package name that you used in the previous step. Replace My python libs with any description that you prefer.

In the output, note the layer's Amazon Resource Name (ARN). You'll need it for the next step.

6.    Run the following command to update your Lambda function configuration to use the layer.

aws lambda update-function-configuration --layers arn:aws:lambda:us-east-2:123456789012:layer:mypythonlibs:1 --function-name my-function

Important: Replace arn:aws:lambda:us-east-2:123456789012:layer:mypythonlibs:1 with the layer ARN from the previous step.

(Optional) Use the Docker images for your Lambda package

You can also use the lambci/lambda Docker images directly for your Lambda package, without creating a layer.

1.    Run the following command to get the required versions of your dependencies:

docker run -v "$PWD":/var/task "lambci/lambda:build-python3.6" /bin/sh -c "pip install -r requirements.txt -t libs; exit"

Important: Replace 3.6 with 3.7 or 3.8, depending on the compatible libraries that you want to install.

2.    Create a Lambda deployment package that includes your function code and the correct libraries.