When I make requests to Amazon Simple Storage Service (Amazon S3), Amazon S3 returns a response similar to one of the following messages:
AmazonS3Exception: Internal Error (Service: Amazon S3; Status Code: 500; Error Code: 500 Internal Error; Request ID: A4DBBEXAMPLE2C4D) AmazonS3Exception: Slow Down (Service: Amazon S3; Status Code: 503; Error Code: 503 Slow Down; Request ID: A4DBBEXAMPLE2C4D)
How can I troubleshoot these errors?
The error code 500 Internal Error indicates that Amazon S3 is unable to handle the request at that time. The error code 503 Slow Down typically indicates that the requests to the S3 bucket are very high, exceeding the request rates described in Request Rate and Performance Guidelines.
Because Amazon S3 is a distributed service, a very small percentage of 5xx errors are expected during normal use of the service. All requests that return 5xx errors from Amazon S3 can and should be retried, so we recommend that applications making requests to Amazon S3 have a fault-tolerance mechanism to recover from these errors.
To resolve or avoid 5xx errors, try these solutions:
- Enable a retry mechanism in the application making requests
- Configure your application to increase request rates gradually
- Distribute objects across multiple prefixes
Enable a retry mechanism in the application making requests
Because of the distributed nature of Amazon S3, requests that return 500 or 503 errors can be retried. It's a best practice to build retry logic into applications that make requests to Amazon S3. We recommend that you set the retry count for your applications to 10.
All AWS SDKs have a built-in retry mechanism with an algorithm that uses exponential backoff. This algorithm implements increasingly longer wait times between retries for consecutive error responses. Most exponential backoff algorithms use jitter (randomized delay) to prevent successive collisions. For more information, see Error Retries and Exponential Backoff in AWS.
Configure your application to gradually increase request rates
To avoid the 503 Slow Down error, try configuring your application to start with a lower request rate (transactions per second). Then, increase the application's request rate exponentially. Amazon S3 automatically scales to handle a higher request rate.
Distribute objects across multiple prefixes
The request rates described in Request Rate and Performance Guidelines apply per prefix in an S3 bucket. To set up your bucket to handle overall higher request rates and to avoid 503 Slow Down errors, you can distribute objects across multiple prefixes. For example, if you're using your S3 bucket to store images and videos, you can distribute the files into two prefixes similar to the following:
If the request rate on the prefixes increases gradually, Amazon S3 scales up to handle requests for each of the two prefixes (3,500 PUT/POST/DELETE or 5,500 GET requests per second), so the overall request rate handled by the bucket doubles.