How can I troubleshoot Application Load Balancer session stickiness issues?

5 minute read
0

I have an Application Load Balancer that uses duration-based stickiness or application-based stickiness sessions. The load balancer is configured to route all user session requests to the same target. However, user session requests are getting routed to different targets.

Short description

Sticky sessions use cookies to help the client maintain a connection to the same instance over a cookie's lifetime. Using sticky sessions configures a load balancer to bind user sessions to a specific instance. This means that all requests from a user during a session are sent to the same instance. However, this assumption about the connection can cause imbalances over time.

A sticky session can fail if:

  • The registered target didn't generate a cookie
  • The client didn't return the cookie in the request header
  • The cookies aren't formatted correctly
  • The duration-based session has passed
  • The session request is passing through multiple load balancers
  • The target health status changed to unhealthy
  • An AWS service disabled stickiness

Note: Before you begin, make sure that you review the requirements and considerations section in Sticky sessions for your Application Load Balancer.

Resolution

Application-based session stickiness

1.    Check for HTTP errors with your load balancer. For instructions, see Troubleshoot your Application Load Balancers.

2.    To check for cookies placed on the backend instance or load balancer, run a curl command similar to the following:

Note: Replace the DNS name with your load balancer name.

[ec2-user@ip-172-31-22-85 ~]$ curl -vko /dev/null internal-EXAMPLE-ELB-1430759361.eu-central-1.elb.amazonaws.com

Note: The Linux curl utility can also be installed on Windows OS.

Example output:

...

< Set-Cookie: PHPSESSID=k0qu6t4e35i4lgmsk78mj9k4a4; path=/

< Set-Cookie:

AWSALBAPP-0=438DC7A50C516D797550CF7DE2A7DBA19D6816D5E6FB20329CD6AEF2B40030B12FF2839757A60E2330136A2182D27D049FB9D887FBFE9E80FB0724130FB3A86A4B0BAC296FDEB9E943EC9272FF52F5A8AEF373DF33;PATH=/

...

3.    To verify that the registered target generated an application cookie, send an HTTP request to the instance IP address similar to the following:

[ec2-user@ip-172-31-22-85 ~]$ curl -vko /dev/null 172.31.30.168

Example output:

...

< Set-Cookie: PHPSESSID=5pq74110nuir60kpapj04mglg4; path=/

...

4.    Verify that the cookie name generated by the registered target is the same name as the cookie on the load balancer from steps 2 and 3.

5.    If the target generated an application cookie and the load balancer generated an AWSALBAPP cookie, make sure that the client sends both cookies in subsequent requests. If the client fails to include either the application or the AWSELB cookie, then stickiness fails. To verify that the client sends both the application and AWSALBAPP cookies, take a packet capture on the client. Use the tcpdump or Wireshark utility for analysis, or use the browsers web-debugging tool to retrieve the cookie information in the request header.

Note: If you are working with AWS Support, you can gather this information by generating a HAR file. HAR files can capture sensitive information, such as user names, passwords, and keys, so be sure to remove any sensitive information from a HAR file.

6.    Check which backend instance that the load balancer routed the request to. You can display the instance ID using Instance metadata by running a script similar to the following:

<?php $instance_id =file_get_contents('http://169.254.169.254/latest/meta-data/instance-id');
echo "instance id = " . $instance_id . "\\xA";
?>

7.    Review your Elastic Load Balancing access logs to check whether requests from the same user are routed to different registered targets. For instructions, see Access logs for your Application Load Balancer.

8.    Verify that the health status of all targets under the target group where stickiness is enabled are healthy. If the target health status changes to “unhealthy,” then stickiness breaks and the load balancer stops routing requests to that target. A new healthy target is then automatically selected by the load balancer and establishes a sticky session. The load balancer continues to route requests to the new target even if the health status is "unhealthy." For more information on health checks, see Health checks for your target groups.

9.    Check for any AWS services, for example, Amazon Elastic Kubernetes Service (Amazon EKS), that might have disabled the stickiness in your load balancer. Check the View CloudTrail events with the API name “ModifyTargetGroupAttributes” and attribute “stickiness.enabled”.

Duration-based session stickiness

1.    Run a command similar to the following with the load balancer DNS name to check for an AWSALB cookie in the response:

[ec2-user@ip-172-31-22-85 ~]$ curl -vko /dev/null internal-TESTELB-1430759361.eu-central-1.elb.amazonaws.com

Example output:

...< Set-Cookie: 
AWSALB=438DC7A50C516D797550CF7DE2A7DBA19D6816D5E6FB20329CD6AEF2B40030B12FF2839757A60E2330136A2182D27D049FB9D887FBFE9E80FB0724130FB3A86A4B0BAC296FDEB9E943EC9272FF52F5A8AEF373DF33;PATH=/
...

Note: If there isn't an AWSALB cookie in the response, then there is no stickiness between the client and the backend instance.

2.    If the load balancer generated an AWSALB cookie, then be sure that the client sends this cookie in subsequent requests. If the client fails to include the AWSALB cookie, then stickiness doesn't work. To verify that the client sends the AWSALB cookie, take a packet capture on the client. Use the tcpdump or Wireshark utility for analysis, or use the browsers web-debugging tool to retrieve the cookie information in the request header.

Note: If you are working with AWS Support, you can gather this information by generating a HAR file. HAR files can capture sensitive information, such as user names, passwords, and keys, so be sure to remove any sensitive information from a HAR file.

3.    Check the duration configured on the load balancer. If the cookie expiration passed, then client sessions no longer stick to the registered target until a new cookie is issued by the load balancer.

4.    If your request is passing through multiple load balancers, verify that stickiness is enabled on only one load balancer. If more than one load balancer generates a cookie, then the load balancer replaces the original cookie and stickiness fails.

For Classic Load Balancers, see How can I troubleshoot the Classic Load Balancer session stickiness feature?


Related information

Why is Elastic Load Balancing unequally routing my load balancer traffic?

Configure sticky sessions for your Classic Load Balancer

How do I set up weighted target groups for my Application Load Balancer?

AWS OFFICIAL
AWS OFFICIALUpdated 2 years ago