How can I resolve API throttling or "Rate Exceeded" errors in Elastic Beanstalk?
Last updated: 2019-10-17
How can I resolve API throttling or "Rate exceeded" errors when I use AWS Elastic Beanstalk?
API calls to any AWS service can't exceed the maximum allowed API request rate per account and per AWS Region. It doesn't matter if the calls come from an application, the AWS Command Line Interface (AWS CLI), or the AWS Management Console. If API requests exceed the maximum rate, you receive a “Rate Exceeded” error, and API calls are then throttled.
You can receive errors from API calls that are made directly to Elastic Beanstalk, as well as other AWS services (such as AWS CloudFormation) that are managed by Elastic Beanstalk.
Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent AWS CLI version.
To prevent or resolve “Rate Exceeded” errors and throttling, try these solutions:
Validate which API calls are being throttled
1. In your Elastic Beanstalk event stream, identify the throttling error. Note the timeframe when you received the error.
Note: If API calls are coming from an application or script outside of Elastic Beanstalk, then look for the timeframe in your application logs.
2. For requests that you found in the timeframe with RequestLimitExceeded errors, use AWS CloudTrail to view events and validate the eventName, eventSource (the service), and userAgent.
Validate "describe" calls in your Elastic Beanstalk environment
Too many "describe" calls contribute to the total API requests measured against the allowed request rate.
For example, you can validate the "describe" calls for the discovery of an Amazon Elastic Compute Cloud (Amazon EC2) instance in your Elastic Beanstalk environment with either of the following approaches.
Use EnvironmentResourceDescription. For example:
$ aws ec2 describe-instances --filters Name="tag:elasticbeanstalk:environment-id",Values="Your_Environment_ID"
Use the DescribeInstances API call with filters for your Elastic Beanstalk tags. For example:
$ aws ec2 describe-instances --filters Name="tag:elasticbeanstalk:environment-name",Values="Your_Environment_Name"
Note: The DescribeInstances API call rate tends to have a higher rate limit than the EnvironmentResourceDescription API call.
Check calls from third-party applications
Third-party applications can make continuous calls to Elastic Beanstalk or other AWS services managed by Elastic Beanstalk.
To verify that your third-party applications aren't making unnecessary calls to the Elastic Beanstalk service or any Elastic Beanstalk integrated services, use CloudTrail to check the user agent making the API calls.
Note: Third-party applications, such as Datadog or New Relic, can make API calls to Elastic Beanstalk. For more information, see rate-limit alerts for Datadog (from the Datadog website) or New Relic (from the New Relic website). Third-party applications aren't supported by AWS.
Use error retries and exponential backoffs
Use error retries and exponential backoffs to help limit the rate of API calls.
Note: Each AWS SDK implements automatic retry logic and exponential backoff algorithms.
Request a service limit increase in the AWS Support Center
Before you request a service limit increase, complete the following:
- Identify the API call and call rate.
- Try using error retries or exponential backoffs.
- Include your AWS Region, the timeframe related to the throttling issues, and the use-case for the limit increase in your request for a service limit increase.
Request a service quota increase.