AWS Chalice is a framework for writing serverless applications in Python that provides familiar, declarative APIs to help you write your application. From the first version of this framework, we’ve provided a deployment packager that handles the details of how to package your application for AWS Lambda. This lets you specify third-party package dependencies using the standard requirements.txt file, and Chalice will automatically handle the rest of the details.

Lambda layers are a way to pull in additional dependencies in code that are separate from the deployment package for a Lambda function. These Lambda layers can be reused by multiple Lambda functions, which allows you to keep your deployment packages small as well as reduce the total storage space needed for all your Lambda functions.

In version 1.8.0 of Chalice, we added support for layers. This allowed you to take existing layers and add them to your Chalice application. You could use tools such as AWS SAM CLI to help you manage these external layers.

Today, in version 1.18.0 of Chalice, we’ve added support for Chalice to automatically manage a shared Lambda layer that contains all your third-party and vendored code. This layer can be used for all your Lambda functions in your application and can also be reused in other Chalice applications and Lambda functions created without Chalice. Enabling this functionality is as simple as enabling a config option in your .chalice/config.json file. This provides several benefits:

• This shared layer is created once and used for all Lambda functions in your application. By default, when creating or updating a Lambda function, Chalice will send the entire contents of the zip file that contains your app and all of its package dependencies. This is repeated for each Lambda function in your application. With automatic layers enabled this zip file is sent to Lambda once and then reused. This results in improved deployment times as well as reduced network usage.
• Using automatic layers will reduce the total storage space needed in Lambda, which has a default quota of 75 GB for function and layer storage.
• The shared layer that Chalice creates is packaged such that any other Chalice app or Python Lambda function can reuse this layer and import Python packages from this layer without any additional code changes. This can be used to manage common Python dependencies across multiple applications.

Using automatic layers

To demonstrate these improvements, we’ll create two Chalice applications that show the different options available for using Lambda layers. In one application, we’ll configure Chalice to automatically manage a shared layer for us. In our second application, we’ll look at how we can configure Chalice to use an existing layer.

First, we’ll create a virtual environment, install Chalice, and create a new project. In our example, we’re using Python 3.7.

$ python3 -m venv /tmp/venv37
$ . /tmp/venv37/bin/activate
$ pip install chalice
$ chalice new-project layerapp
$ cd layerapp

Next we’ll update our application to use automatic layers. To do this we’ll update our .chalice/config.json file and set "automatic_layer": true.

{
  "automatic_layer": true,
  "stages": {
    "dev": {
      "api_gateway_stage": "api"
    }
  },
  "version": "2.0",
  "app_name": "layerapp"
}

Next, we’ll add several Python package dependencies to our application. We’ll add the AWS SDK for Python (boto3) as well as Pandas, a package that provides data structures for data analysis, time series, and statistics. We do this by updating our requirements.txt file with the following contents.

boto3==1.14.43
pandas==1.1.0

Pandas also demonstrates one of the unique capabilities of Chalice to create a deployment package that contains native C dependencies. Normally, pip will install packages that match the platform you’re running on, but Chalice will correctly use the python package that matches the platform for the Lambda runtime (manylinux1 wheels). It does this transitively for all your third-party dependencies. With Pandas, Chalice will also correctly use the appropriate Linux python package for Numpy, a dependency of Pandas.

We’ll have each Lambda function import a Python package and return its path.

from chalice import Chalice

app = Chalice(app_name='layerapp')


@app.lambda_function()
def boto3_import(event, context):
    import boto3
    return {'boto3': boto3.__file__}


@app.lambda_function()
def pandas_import(event, context):
    import pandas
    return {'pandas': pandas.__file__}

When we run chalice deploy, we’ll see that Chalice creates a shared layer package.

$ chalice deploy
Creating shared layer deployment package.
Creating app deployment package.
Creating lambda layer: layerapp-dev-managed-layer
Creating IAM role: layerapp-dev
Creating lambda function: layerapp-dev-boto3_import
Creating lambda function: layerapp-dev-pandas_import
Resources deployed:
  - Lambda Layer ARN: arn:aws:lambda:us-west-2:12345:layer:layerapp-dev-managed-layer:1
  - Lambda ARN: arn:aws:lambda:us-west-2:12345:function:layerapp-dev-boto3_import
  - Lambda ARN: arn:aws:lambda:us-west-2:12345:function:layerapp-dev-pandas_import

Chalice also places the third-party package files in the correct location so we can import these packages in our application without adding any custom code.

We can test our Lambda functions using the chalice invoke command.

$ chalice invoke -n boto3_import
{"boto3": "/opt/python/lib/python3.7/site-packages/boto3/__init__.py"}

$ chalice invoke -n pandas_import
{"pandas": "/opt/python/lib/python3.7/site-packages/pandas/__init__.py"}

As part of this change, Chalice will also cache the deployment package locally and reuse the generated zip files if your requirements.txt and vendor/ files have not changed. We can see this behavior if we rerun the chalice deploy command:

$ chalice deploy
Creating shared layer deployment package.
  Reusing existing shared layer deployment package.
Creating app deployment package.
  Reusing existing app deployment package.
Updating lambda layer: layerapp-dev-managed-layer
Updating policy for IAM role: layerapp-dev
Updating lambda function: layerapp-dev-boto3_import
Updating lambda function: layerapp-dev-pandas_import
Resources deployed:
  - Lambda Layer ARN: arn:aws:lambda:us-west-2:12345:layer:layerapp-dev-managed-layer:2
  - Lambda ARN: arn:aws:lambda:us-west-2:12345:function:layerapp-dev-boto3_import
  - Lambda ARN: arn:aws:lambda:us-west-2:12345:function:layerapp-dev-pandas_import

Chalice will also delete the previous layer version when updating its shared layer.

Reusing existing layers

We can reuse this shared layer we’ve created in other Chalice apps and Lambda functions. In this example, we’ll create a Chalice app that reuses this layer we’ve created. This is done by specifying the layers option in your .chalice/config.json file. This is useful if our new application has the same shared dependencies as our previous application. This is shown in the diagram below.

To do this, we’ll create a new Chalice application in a new directory.

$ cd /tmp
$ chalice new-project reuse-layer
$ cd reuse-layer

Next, we’ll add a layers configuration option in our .chalice/config.json file. We need to specify the ARN of the Lambda layer that was created previously. This was shown in the output of the chalice deploy command.

{
  "version": "2.0",
  "app_name": "reuse-layer",
  "layers": ["arn:aws:lambda:us-west-2:12345:layer:layerapp-dev-managed-layer:2"],
  "stages": {
    "dev": {
      "api_gateway_stage": "api"
    }
  }
}

Next to test our application, we’ll update our app.py file to try and import the Pandas library and return its file location.

from chalice import Chalice

app = Chalice(app_name='reuse-layer')


@app.lambda_function()
def pandas_import_reuse(event, context):
    import pandas
    return {'pandas': pandas.__file__}

Now we can run chalice deploy.

$ chalice deploy
Creating deployment package.
Updating policy for IAM role: reuse-layer-dev
Creating lambda function: reuse-layer-dev-pandas_import
Resources deployed:
  - Lambda ARN: arn:aws:lambda:us-west-2:12345:function:reuse-layer-dev-pandas_import_reuse

Finally, we can test our new application by invoking the pandas_import_reuse function.

$ chalice invoke -n pandas_import
{"pandas": "/opt/python/lib/python3.7/site-packages/pandas/__init__.py"}

We can see that our Lambda function ran successfully and was able to pull in the shared layer code which enabled us to import the Pandas package. We can also check the size of our deployment package to confirm that we didn’t add the Pandas package.

$ du -hs .chalice/deployments/*
 16K	.chalice/deployments/f7b6e19947b08408954593fbc1a4ec7e-python3.7.zip
$ unzip -l .chalice/deployments/*
Archive:  .chalice/deployments/f7b6e19947b08408954593fbc1a4ec7e-python3.7.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
    57584  01-01-1980 00:00   chalice/app.py
      500  01-01-1980 00:00   chalice/__init__.py
      298  01-01-1980 00:00   app.py
---------                     -------
    58382                     3 files

Next Steps

For more information on application packaging, see our chalice documentation. This feature also works when generating AWS SAM templates as well as Terraform templates. Try out automatic layers today and let us know what you think! You can give us feedback on our GitHub repository.