How can I prevent HTTP 504 gateway timeout errors in Amazon OpenSearch Service?

Last updated: 2021-07-30

I'm trying to resolve a 504 gateway timeout error in Amazon OpenSearch Service (successor to Amazon Elasticsearch Service). How can I do this?

Short description

A load balancer sits in front of each OpenSearch Service domain. The load balancer distributes incoming traffic to the data nodes. If OpenSearch Service requests don't complete and return a confirmation within the idle timeout period, the load balancer closes the TCP connection. As a result, you'll receive an HTTP 504 gateway timeout error. An HTTP 504 gateway timeout error doesnot necessarily indicate a problem with your cluster—it just means that the request couldn't be completed within the idle timeout period.

Gateway timeout errors usually occur when you send too many requests at the same time, or when you send complex requests. In both cases, the result is the same: OpenSearch Service can't complete the request within the idle timeout period.

Resolution

Use one or more of the following methods to resolve HTTP 504 gateway timeout errors:

  • Enable slow logs for your OpenSearch Service index, and then specify logging thresholds. Slow logs can help you determine if a particular query is taking a long time to complete. If so, tune the query to resolve the 504 error. For more information, see Viewing Amazon OpenSearch Service slow logs.
  • Reduce the amount of data that OpenSearch Service must query for the requests. This reduces the amount of time required for the requests to complete.
  • Switch to a larger instance type. For more information, see Choosing instance types and testing.
  • Configure exponential backoff and retry mechanisms in your application so that requests that time out are sent again.
  • Use bulk requests instead of individual requests. This reduces the per-request overhead for the cluster.