How do I troubleshoot Lambda function failures?

Last updated: 2021-05-19

To troubleshoot Lambda function failures, first determine what's causing the error by using one or more of the AWS services and features listed in this article. Then, follow the links provided to review the troubleshooting best practices for each issue.

If the security permissions for your Lambda deployment package are incorrect, you see one of the following errors:

• EACCES: permission denied, open '/var/task/index.js'
• cannot load such file -- function
• [Errno 13] Permission denied: '/var/task/function.py'

If your AWS Identity and Access Management (IAM) user (or the role that you assume), doesn't have permission to invoke a function, then you receive the following error:

User: arn:aws:iam::123456789012:user/developer is not authorized to perform: lambda:InvokeFunction on resource: my-function

To troubleshoot Lambda permissions errors

Review your Lambda log file entries in AWS CloudTrail. The requester making calls to Lambda must have the AWS Identity and Access Management (IAM) permissions required to invoke your function. To grant the required permissions, update your Lambda function permissions.

For more information, see Understanding AWS Lambda log file entries, Troubleshooting AWS Lambda identity and access and IAM: lambda:InvokeFunction not authorized.

If there are issues with your Lambda code, you see many types of errors. The following are some of the more common Lambda code-related errors:

• Unable to marshal response: Object of type AttributeError is not JSON serializable
• Issue: The AWS SDK included on the runtime is not the latest version
• (Node.js) Function returns before code finishes executing
• KeyError

To troubleshoot Lambda code errors

1.    Review your Amazon CloudWatch Logs for Lambda.

You can use CloudWatch to view all logs generated by your function's code and identify potential issues. For more information, see Accessing Amazon CloudWatch Logs for AWS Lambda. For details on function logging, see the following Lambda function logging instructions for the programming language that you're using:

• Python Lambda function logging instructions
• Node.js Lambda function logging instructions
• Java Lambda function logging instructions
• Go Lambda function logging instructions
• C# Lambda function logging instructions
• PowerShell Lambda function logging instructions
• Ruby Lambda function logging instructions

Note: If your function is returning a stack trace, then the error message in the stack trace specifies what's causing the error.

2.    Use AWS X-Ray to identify any code performance bottlenecks. If your Lambda function uses downstream AWS resources, microservices, databases, or HTTP web APIs, then you can use AWS X-Ray to help troubleshoot code performance issues. For more information, see Using AWS Lambda with AWS X-Ray.

3.    Confirm that your function's deployment package can import any required dependencies. Follow the deployment package instructions for the programming language that you're using:

• Python Lambda deployment package instructions
• Node.js Lambda deployment package instructions
• Java Lambda deployment package instructions
• Go Lambda deployment package instructions
• C# Lambda deployment package instructions
• PowerShell Lambda deployment package instructions
• Ruby Lambda deployment package instructions

Note: You can also use Lambda layers to add dependencies that are outside of your deployment package.

4.    (For code deployed as a container image) Confirm that you're installing the runtime interface client and deploying the image correctly. Follow the container image instructions for the programming language that you're using:

• Python Lambda container image instructions
• Node.js Lambda container image instructions
• Java Lambda container image instructions
• Go Lambda container image instructions
• C# Lambda container image instructions
• Ruby Lambda container image instructions

If there are issues with your Lambda networking configuration, you see many types of errors. The following are some of the most common Lambda networking-related errors:

If your function is in a virtual private cloud (VPC) and then loses internet access or times out, then you see the following error:

connect ETIMEDOUT 176.32.98.189:443
Task timed out after 10.00 seconds

If the VPC that your function is in reaches its elastic network interface limit, you see the following error:

ENILimitReachedException: The elastic network interface limit was reached for the function's VPC.

If the Transmission Control Protocol (TCP) connection is dropped, you see the following error:

Connection reset by peer

To troubleshoot Lambda networking errors

1.    Confirm that there's a valid network path to the endpoint that your function is trying to reach. For more information, see Configuring a Lambda Function to Access Resources in a VPC.

2.    (For functions connected to an Amazon VPC) Confirm that your function has access to the internet. For more information, see How do I give internet access to a function that's connected to an Amazon VPC? Also, see How do I troubleshoot timeout issues with a Lambda function in a VPC?

Note: If you can't determine why your function code isn't reaching a public endpoint after reviewing your VPC configuration, turn on VPC Flow Logs. VPC Flow Logs allow you to see all the network traffic flowing to and from a VPC, allowing you to determine why a specific request was denied or didn't route. For more information, see Troubleshoot networking issues in Lambda.

If your function gets throttled, you see the following error:

Rate exceeded
429 TooManyRequestsException

To troubleshoot Lambda throttling errors

Review your CloudWatch metrics for Lambda. For more information, see AWS Lambda CloudWatch metrics.

Key metrics to monitor:

• ConcurrentExecutions
• UnreservedConcurrentExecutions
• Throttles

Note: If requests to invoke your function arrive faster than the function can scale or exceed your concurrency limit, then the requests fail with a 429 throttling error. For more information, see AWS Lambda function scaling. Also, How do I troubleshoot Lambda function throttling with "Rate exceeded" and 429 "TooManyRequestsException" errors?

If your invoke request fails, then you see any of the following 502 or 500 server-side errors:

• InvalidRuntimeException
• InvalidSecurityGroupIDException
• InvalidZipFileException
• KMSAccessDeniedException
• KMSNotFoundException
• You have exceeded the maximum limit for Hyperplane ENIs for your account
• SubnetIPAddressLimitReachedException

To troubleshoot Lambda Invoke API 500 and 502 errors

Follow the instructions in How do I troubleshoot 502 and 500 errors when I invoke my Lambda function? For a list of possible errors with descriptions, see Errors in the Lambda invoke API reference.

If you're using container images and there's an issue with a container image, you see any of the following errors:

• "errorType": "Runtime.InvalidEntrypoint"
• Error: You are using an AWS CloudFormation template, and your container ENTRYPOINT is being overridden with a null or empty value.

To troubleshoot Lambda container image errors

Follow the instructions in Troubleshoot container image issues in Lambda.

Note: If you're not sure what type of issue is causing the error, follow the instructions in the Identify and troubleshoot Lambda code errors section of this article.