How do I troubleshoot Lambda function failures?
Last updated: 2021-05-19
When I try to invoke my AWS Lambda function, it fails and returns an error. How do I troubleshoot Lambda function failures?
To troubleshoot Lambda function failures, first determine what's causing the error by using one or more of the AWS services and features listed in this article. Then, follow the links provided to review the troubleshooting best practices for each issue.
Identify and troubleshoot any permissions errors
If the security permissions for your Lambda deployment package are incorrect, you see one of the following errors:
- EACCES: permission denied, open '/var/task/index.js'
- cannot load such file -- function
- [Errno 13] Permission denied: '/var/task/function.py'
If your AWS Identity and Access Management (IAM) user (or the role that you assume), doesn't have permission to invoke a function, then you receive the following error:
User: arn:aws:iam::123456789012:user/developer is not authorized to perform: lambda:InvokeFunction on resource: my-function
To troubleshoot Lambda permissions errors
Review your Lambda log file entries in AWS CloudTrail. The requester making calls to Lambda must have the AWS Identity and Access Management (IAM) permissions required to invoke your function. To grant the required permissions, update your Lambda function permissions.
Identify and troubleshoot any code errors
If there are issues with your Lambda code, you see many types of errors. The following are some of the more common Lambda code-related errors:
- Unable to marshal response: Object of type AttributeError is not JSON serializable
- Issue: The AWS SDK included on the runtime is not the latest version
- (Node.js) Function returns before code finishes executing
To troubleshoot Lambda code errors
1. Review your Amazon CloudWatch Logs for Lambda.
You can use CloudWatch to view all logs generated by your function's code and identify potential issues. For more information, see Accessing Amazon CloudWatch Logs for AWS Lambda. For details on function logging, see the following Lambda function logging instructions for the programming language that you're using:
- Python Lambda function logging instructions
- Node.js Lambda function logging instructions
- Java Lambda function logging instructions
- Go Lambda function logging instructions
- C# Lambda function logging instructions
- PowerShell Lambda function logging instructions
- Ruby Lambda function logging instructions
Note: If your function is returning a stack trace, then the error message in the stack trace specifies what's causing the error.
2. Use AWS X-Ray to identify any code performance bottlenecks. If your Lambda function uses downstream AWS resources, microservices, databases, or HTTP web APIs, then you can use AWS X-Ray to help troubleshoot code performance issues. For more information, see Using AWS Lambda with AWS X-Ray.
3. Confirm that your function's deployment package can import any required dependencies. Follow the deployment package instructions for the programming language that you're using:
- Python Lambda deployment package instructions
- Node.js Lambda deployment package instructions
- Java Lambda deployment package instructions
- Go Lambda deployment package instructions
- C# Lambda deployment package instructions
- PowerShell Lambda deployment package instructions
- Ruby Lambda deployment package instructions
Note: You can also use Lambda layers to add dependencies that are outside of your deployment package.
4. (For code deployed as a container image) Confirm that you're installing the runtime interface client and deploying the image correctly. Follow the container image instructions for the programming language that you're using:
Identify and troubleshoot any networking errors
If there are issues with your Lambda networking configuration, you see many types of errors. The following are some of the most common Lambda networking-related errors:
If your function is in a virtual private cloud (VPC) and then loses internet access or times out, then you see the following error:
connect ETIMEDOUT 220.127.116.11:443
Task timed out after 10.00 seconds
If the VPC that your function is in reaches its elastic network interface limit, you see the following error:
ENILimitReachedException: The elastic network interface limit was reached for the function's VPC.
If the Transmission Control Protocol (TCP) connection is dropped, you see the following error:
Connection reset by peer
To troubleshoot Lambda networking errors
1. Confirm that there's a valid network path to the endpoint that your function is trying to reach. For more information, see Configuring a Lambda Function to Access Resources in a VPC.
2. (For functions connected to an Amazon VPC) Confirm that your function has access to the internet. For more information, see How do I give internet access to a function that's connected to an Amazon VPC? Also, see How do I troubleshoot timeout issues with a Lambda function in a VPC?
Note: If you can't determine why your function code isn't reaching a public endpoint after reviewing your VPC configuration, turn on VPC Flow Logs. VPC Flow Logs allow you to see all the network traffic flowing to and from a VPC, allowing you to determine why a specific request was denied or didn't route. For more information, see Troubleshoot networking issues in Lambda.
Identify and troubleshoot any throttling errors
If your function gets throttled, you see the following error:
To troubleshoot Lambda throttling errors
Review your CloudWatch metrics for Lambda. For more information, see AWS Lambda CloudWatch metrics.
Key metrics to monitor:
Note: If requests to invoke your function arrive faster than the function can scale or exceed your concurrency limit, then the requests fail with a 429 throttling error. For more information, see AWS Lambda function scaling. Also, How do I troubleshoot Lambda function throttling with "Rate exceeded" and 429 "TooManyRequestsException" errors?
Identify and troubleshoot any Invoke API 500 and 502 errors
If your invoke request fails, then you see any of the following 502 or 500 server-side errors:
- You have exceeded the maximum limit for Hyperplane ENIs for your account
To troubleshoot Lambda Invoke API 500 and 502 errors
Follow the instructions in How do I troubleshoot 502 and 500 errors when I invoke my Lambda function? For a list of possible errors with descriptions, see Errors in the Lambda invoke API reference.
Identify and troubleshoot any container image errors
If you're using container images and there's an issue with a container image, you see any of the following errors:
- "errorType": "Runtime.InvalidEntrypoint"
- Error: You are using an AWS CloudFormation template, and your container ENTRYPOINT is being overridden with a null or empty value.
To troubleshoot Lambda container image errors
Follow the instructions in Troubleshoot container image issues in Lambda.
Note: If you're not sure what type of issue is causing the error, follow the instructions in the Identify and troubleshoot Lambda code errors section of this article.