How do I troubleshoot Lambda function throttling with "Rate exceeded" and 429 "TooManyRequestsException" errors?

Resolution

Lambda functions are sometimes throttled to protect your resources and downstream applications. Even though Lambda automatically scales to accommodate incoming traffic, your function can still be throttled for various reasons.

To troubleshoot Lambda throttling issues, including Rate exceeded and TooManyRequestsException errors, review the following steps.

Verify what resource is throttled

Throttle errors might not be from your Lambda function. Throttles can also occur on API calls during your function's invocation.

To confirm what resource is throttled, check the following resources.

Check your Amazon CloudWatch Logs to see if there are throttling errors, but no corresponding data points in the Lambda Throttles metrics

If there are no Lambda Throttles metrics, then the throttling is happening on API calls in your Lambda function code.

Check your function code for any throttled API calls

If certain API calls are throttled, make sure that you use exponential backoff in your code to retry the API calls.

Note: If you need a higher transactions per second (TPS) quota for an API call, then request a service quota increase. Not all quotas are adjustable.

Check your function's concurrency metrics

Review your Lambda metrics in Amazon CloudWatch

Check the ConcurrentExecutions metric for your function in the AWS Region where you see throttling.

Compare the ConcurrentExecutions metric with the Throttles metric for the same timestamp

View the Maximum statistic for ConcurrentExecutions and the Sum statistic for Throttles. See if the maximum ConcurrentExecutions are close to your account-level concurrency quota in the Region, along with corresponding data points in the Throttles graph.

Check if you're exceeding the initial burst concurrency quota for a particular AWS Region

On the Metrics page for Lambda in the CloudWatch console, reduce the graph's time range to 1 minute. If you're limited by burst scaling, then you see a spike of Throttles that corresponds to a stair-step pattern of ConcurrentExecutions on the graph.

Note: To work around burst concurrency limits, you can configure provisioned concurrency.

Check for spikes in Duration metrics for your function

Concurrency depends on function duration. If your function code is taking too long to complete, there might not be enough compute resources.

Try increasing the function's memory setting. Then, use AWS X-Ray and CloudWatch Logs to isolate the cause of duration increases.

Note: Changing the memory setting can affect the charges that you incur for execution time.

You can add the function to an Amazon Virtual Private Cloud (Amazon VPC). When the function is in an Amazon VPC, see How do I give internet access to a Lambda function that's connected to an Amazon VPC?

Check for an increase in Errors metrics for your function

Increased errors can lead to retries and cause an overall increase in invocations. Increased invocations can lead to an increase in concurrency. Use CloudWatch Logs to identify and eliminate errors, and have your function code handle exceptions.

Note: Your function can also be throttled based on invocation requests per Region (requests per second), which is 10 times your concurrent executions quota.

Configure reserved concurrency

Confirm that you configured reserved concurrency on your function

Use the Lambda console to check the setting for reserved concurrency. If reserved concurrency isn't configured, then the function uses unreserved concurrency. When invocations with functions exceed the unreserved concurrency, throttling occurs.

Note: If you configured a function with zero reserved concurrency, then the function is throttled because it can't process any events. Make sure that you increase the value to a number greater than zero.

Review the Maximum statistic in CloudWatch for your function

See if the function metric hits the maximum value for the ConcurrentExecutions metric at any point.

Increase the reserved concurrency for your function to a concurrency value that keeps the function from being throttled

Use the Lambda console to change the setting, or call the PutFunctionConcurrency API.

Use exponential backoff in your app

To retry throttled requests, it's a best practice to use exponential backoff in your application that's calling your Lambda function.

Use a dead-letter queue

For asynchronous event sources, such as Amazon Simple Storage Service (Amazon S3) and Amazon EventBridge, configure your function with a dead-letter queue (DLQ). Then, verify the error handling for asynchronous invocation configuration. The DLQ catches any events that are discarded due to constant throttles and can protect your data if you're seeing significant throttling.

Note: For Amazon Simple Queue Service (Amazon SQS) event sources, you must configure the DLQ on the Amazon SQS queue.

Request a service quota increase

If your workload requires a higher service quota for concurrent executions, then use the Service quota console to request a service quota increase.

---

Related information

Best practices for working with AWS Lambda functions

Troubleshoot invocation issues in AWS Lambda

Error handling and automatic retries in AWS Lambda