AWS Compute Blog

Upcoming changes to the Python SDK in AWS Lambda

Update (April 26, 2022):

In response to customer feedback, we have decided to cancel the change described in this blog post. The version of the AWS SDK included in the AWS Lambda runtimes for Python 2.7, Python 3.6 and Python 3.7 will continue to include the ‘requests’ module in Botocore. No action is required for customers using these runtimes. The Lambda runtimes for Python 3.8 and later do not include the ‘requests’ module. Customers using ‘requests’ with these runtimes should package the ‘requests’ module with their function code or as a Lambda layer.

Update (November 11, 2021):

The deprecation date for the Lambda service to bundle the requests module in the AWS SDK is now March 31, 2022.

Update (March 29, 2021):

The deprecation date for the Lambda service to bundle the requests module in the AWS SDK is now December 1, 2021.

Update (January 19, 2021):

The deprecation date for the Lambda service to bundle the requests module in the AWS SDK is now March 31, 2021.

Update (November 23, 2020):

For customers using inline code in AWS CloudFormation templates that include the cfn-response module, we have recently removed this module’s dependency on botocore.requests. Customers will need to update their Lambda function to attach the updated cfn-response module. Change the Python version or trigger an update to the function code to attach the updated modules.

Update (March 10, 2020):

The deprecation date for the Lambda service to bundle the requests module in the AWS SDK is now January 30, 2021.

This blog post describes an upcoming change to the AWS SDK that affects Python developers using the requests module in Botocore. This post explains why the changes are happening, and describes what Python developers must do to continue using the requests library.

Please note that this change affects the Python 2.7, Python 3.6 and Python 3.7 runtimes. The Python 3.8 runtime is not affected.

The upcoming changes to AWS SDK

Botocore is a low-level interface to many services in the AWS Cloud. The package is the foundation for the AWS CLI and also Boto3, which is the AWS SDK for Python. In August 2018, Botocore was refactored to allow pluggable HTTP clients.

One of the main changes is that the requests library was replaced with urllib3. Additionally, the requests dependency was also unvendored, meaning Botocore can now support a range of versions of urllib3, instead of relying on a specific version. From version 1.13.0, the requests module is no longer part of the AWS SDK for Python. These changes create additional flexibility for Python developers, and can result in performance improvements for applications using Botocore.

Although the SDK has removed the requests module, the Lambda service continues to bundle the requests module in the AWS SDK until January 30, 2021. This is so builders have additional time to decide on the best course of action for their Python Lambda functions that rely on the requests module.

Best practices for using the AWS SDK

For convenience, the Lambda service includes the AWS SDK in its execution environment. This allows Lambda console users to easily use the SDK functionality within the integrated code window. It also allows Lambda to interoperate with the growing number of AWS services and features released to customers.

The best practice for Lambda development is to bundle all dependencies used by your Lambda function, including the AWS SDK. By doing this, your code uses the bundled version and is not affected when the version in the execution environment is upgraded. This is preferable to using the included version of the SDK, since this version can change, and in rare cases might affect compatibility with your code.

If you are currently bundling the SDK version in your Lambda version, you do not need to take any further action. Your code continues to use the bundled version and the upcoming changes do not affect you.

Importing the requests module directly into your code

If you are using the AWS SDK in the Lambda execution environment and do not want to bundle a version into your zipped deployment package, you have a couple of additional options.

First, you can install the requests module into your Python environment and import the module directly. Currently, you may be importing the library from Botocore in your Lambda function using this code:

from botocore.vendored import requests

First, to install the requests module, enter the following in a terminal window:

pip install requests -t ./

After the installation, update the import statement in your code as shown below:

import requests

This updates your dependency from the Botocore vendored version to a locally packaged version of the module. As a result, your code is unaffected after the requests module is no longer available via the AWS SDK in the execution environment.

For more information on bundling libraries, read how to build an AWS Lambda deployment package for Python.

Using AWS Lambda layers

AWS has also published Lambda layers containing version 1.12.221 of the AWS SDK, which includes the requests module in Botocore. To use this, first identify the layer ARN you need, using the Python runtime version and AWS Region.

Using this method, you do not need to change the code in the Lambda function. With the layer attached to the function, you can continue to use the requests module with:

from botocore.vendored import requests

To add this layer to your function in the Lambda console:

  1. Select your function from the Functions list.
  2. Locate the function’s ARN near the top of the display. In the ARN, note the AWS Region code after arn:aws:lambda:. In the Function code panel, note the Python version in the Runtime dropdown.Identifying region and runtime from Lambda function
  3. Using both the AWS Region code and Python version, find the ARN from the lists below. For Python versions 2.7, 3.6 and 3.7, locate the layer ARN for your Region:
    Region ARN
    ap-northeast-1 arn:aws:lambda:ap-northeast-1:249908578461:layer:AWSLambda-Python-AWS-SDK:4
    us-east-1 arn:aws:lambda:us-east-1:668099181075:layer:AWSLambda-Python-AWS-SDK:4
    ap-southeast-1 arn:aws:lambda:ap-southeast-1:468957933125:layer:AWSLambda-Python-AWS-SDK:4
    eu-west-1 arn:aws:lambda:eu-west-1:399891621064:layer:AWSLambda-Python-AWS-SDK:4
    us-west-1 arn:aws:lambda:us-west-1:325793726646:layer:AWSLambda-Python-AWS-SDK:4
    ap-east-1 arn:aws:lambda:ap-east-1:118857876118:layer:AWSLambda-Python-AWS-SDK:4
    ap-northeast-2 arn:aws:lambda:ap-northeast-2:296580773974:layer:AWSLambda-Python-AWS-SDK:4
    ap-northeast-3 arn:aws:lambda:ap-northeast-3:961244031340:layer:AWSLambda-Python-AWS-SDK:4
    ap-south-1 arn:aws:lambda:ap-south-1:631267018583:layer:AWSLambda-Python-AWS-SDK:4
    ap-southeast-2 arn:aws:lambda:ap-southeast-2:817496625479:layer:AWSLambda-Python-AWS-SDK:4
    ca-central-1 arn:aws:lambda:ca-central-1:778625758767:layer:AWSLambda-Python-AWS-SDK:4
    eu-central-1 arn:aws:lambda:eu-central-1:292169987271:layer:AWSLambda-Python-AWS-SDK:4
    eu-north-1 arn:aws:lambda:eu-north-1:642425348156:layer:AWSLambda-Python-AWS-SDK:4
    eu-west-2 arn:aws:lambda:eu-west-2:142628438157:layer:AWSLambda-Python-AWS-SDK:4
    eu-west-3 arn:aws:lambda:eu-west-3:959311844005:layer:AWSLambda-Python-AWS-SDK:4
    sa-east-1 arn:aws:lambda:sa-east-1:640010853179:layer:AWSLambda-Python-AWS-SDK:4
    us-east-2 arn:aws:lambda:us-east-2:259788987135:layer:AWSLambda-Python-AWS-SDK:4
    us-west-2 arn:aws:lambda:us-west-2:420165488524:layer:AWSLambda-Python-AWS-SDK:5
    cn-north-1 arn:aws-cn:lambda:cn-north-1:683298794825:layer:AWSLambda-Python-AWS-SDK:4
    cn-northwest-1 arn:aws-cn:lambda:cn-northwest-1:382066503313:layer:AWSLambda-Python-AWS-SDK:4
    us-gov-west arn:aws-us-gov:lambda:us-gov-west-1:556739011827:layer:AWSLambda-Python-AWS-SDK:4
    us-gov-east arn:aws-us-gov:lambda:us-gov-east-1:138526772879:layer:AWSLambda-Python-AWS-SDK:4

    In this example, using Python 3.6 in us-west-2, the layer ARN is arn:aws:lambda:us-west-2:420165488524:layer:AWSLambda-Python-AWS-SDK:5.

  4. Select the Layers view in the Designer panel, then choose Add a layer.Choose Add a layer from Lambda.
  5. Select the Provide a layer version ARN option, and then paste the relevant ARN from the list above into the Layer version ARN field. Choose Add.Add layer to function

The Lambda function is now using the layer.

Alternatively, you can use the CLI to add a Lambda Layer to your existing Lambda function. To do this, first identify the correct layer ARN using the procedure above. Next, execute the following command from a terminal window, replacing the layer parameter with the ARN:

aws lambda update-function-configuration --function-name yourFunctionName --layers arn:aws:lambda:xxxxxxxxxxxxxxxxxxxx

If you are using the AWS Serverless Application Model, you can add the layer to an existing function using the following YAML in the SAM template, replacing arn with the ARN:

Layers:
  - arn

To learn more, see our documentation describing how to add layers to Lambda functions.

Conclusion

The change to Botocore helps improve flexibility and performance for the AWS SDK. If you are bundling a fixed AWS SDK version with your Python version, you do not need to take any action. If you are using the AWS SDK included in the execution environment and want to continue using the requests module, you can include the Lambda Layer with the appropriate AWS SDK version, or include the requests module directly in your application package.