Gokul shows you how
to troubleshoot 504 errors
with Classic Load Balancers

504-error-classic-gokul

I see HTTP 504 errors in Classic Load Balancer access logs, CloudWatch metrics, or when connecting to my service through a Classic Load Balancer. How do I fix this?

An HTTP 504 error is a HTTP status code that indicates a gateway or proxy has timed out. When troubleshooting, investigate the following:

Check your load balancer’s idle timeout, and modify if necessary

The most common reason for a load balancer to return HTTP 504 errors is that a corresponding back-end instance did not respond to the request within the currently configured idle timeout (by default, a Classic Load Balancer’s idle timeout is 60 seconds).

If CloudWatch metrics are enabled, check CloudWatch metrics for your load balancer—if latency data points are equal to your currently configured load balancer timeout value, and there are data points in the HTTPCode_ELB_5XX metric, then at least one request has timed out.

To resolve this, either modify the idle timeout for your load balancer so that the HTTP request will be completed within the idle timeout period, or tune your application to respond more quickly.

Make sure that your back-end instances keep connections open

If a back-end instance closes a TCP connection to the load balancer before the load balancer has reached its idle timeout value, the load balancer may not be able to fulfill the request, generating an HTTP 504 error.

To resolve this, enable keep-alive settings on your back-end instances, and set the keep-alive timeout to a value greater than the load balancer’s idle timeout.

(Apache only) Disable TCP_DEFER_ACCEPT

When TCP_DEFER_ACCEPT is enabled for Apache back-end instances, the load balancer initiates a connection by sending a SYN to the back-end instance; the back-end instance responds with a SYN-ACK, and the load balancer sends an empty ACK to the back-end instance.

Because the last ACK is empty, the back end does not accept the ACK, and instead resends a SYN-ACK back to the load balancer. This can result in a subsequent SYN retry timeout. When the back-end instance closes the connection without sending a FIN or RST to the load balancer, the load balancer considers the connection to be established when it is not. Then when the load balancer sends requests through this TCP connection, the back end responds with an RST, generating a 504 error.

To resolve this, set AcceptFilter http and AcceptFilter https to none.

(Apache only) Disable the event MPM, and optimally configure the prefork and worker MPMs

The event MPM should not be used on back-end instances that are registered to a load balancer, because it dynamically closes connections, which results in HTTP 504 errors being sent to the clients.

For optimal performance when using the prefork and worker MPMs, presuming the load balancer is configured with a 60-second idle timeout, use these values:

 

mod_prefork MPM

mod_worker MPM

 

Timeout

65

65

 

KeepAliveTimeout

65

65

 

KeepAlive

On

On

 

MaxKeepAliveRequests

10000

0

 

AcceptFilter http

none

none

 

AcceptFilter https

none

none

 

Did this page help you? Yes | No

Back to the AWS Support Knowledge Center

Need help? Visit the AWS Support Center

Published: 2016-09-15