How do I troubleshoot scaling issues with my Amazon EC2 Auto Scaling group?
Last updated: 2020-12-21
My Amazon Elastic Compute Cloud (Amazon EC2) Auto Scaling group isn't scaling correctly. How do I troubleshoot this?
Based on your scaling issues, perform the following checks on your Amazon EC2 Auto Scaling configurations:
My Amazon EC2 Auto Scaling group isn’t following the scaling policy I configured
Check your scaling policies to see whether an event triggers more than one policy. If two policies are evaluated at the same time, Amazon EC2 Auto Scaling follows the policy with the greater impact. For example, assume you have one policy to add two instances and another policy to add four instances. In this case, Amazon EC2 Auto Scaling adds four instances when both policies are triggered at the same time.
My Amazon EC2 Auto Scaling group is scaling out instead of scaling in
Verify if a scale-out policy and a scale-in policy are triggered at the same time. When both scale-out and scale-in policies are triggers at the same time, Amazon EC2 Auto Scaling follows the scale-out policy to confirm availability.
To verify if scale-out and scale-in policies are triggered at the same time:
My Amazon EC2 Auto Scaling group isn’t responding to a CloudWatch alarm or health check replacement
- Check if your Auto Scaling group already reached its minimum or maximum number of instances. To find the specified minimum or maximum capacity, review your Amazon EC2 Auto Scaling group's details using the Amazon EC2 console. Amazon EC2 Auto Scaling doesn't show an error message in the group's activity history when scaling is blocked due to minimum or maximum capacity.
- Check suspended processes for your Auto Scaling group. Amazon EC2 Auto Scaling can't evaluate a CloudWatch scaling policy for suspended Terminate, Launch, or Alarm Notification processes. Amazon EC2 Auto Scaling can't evaluate a health check replacement for suspended ReplaceUnhealthy or HealthCheck processes.
- If you're using a simple scaling policy, check if your instances are in a cooldown period. Simple scaling policies remain suspended until after the cooldown period.
- If you're using a step scaling policy or target tracking policy, check for an in progress instance warmup. If you specified a warm-up time, the Auto Scaling group doesn't count instances toward the group metrics until after the warmup.
- Check your CloudWatch alarm to be sure that it's triggering scaling activity correctly. For more information, see How do I ensure that Amazon CloudWatch alarms trigger scaling of my Auto Scaling group?
- Check if there is a lifecycle hook configured for your Amazon EC2 Auto Scaling group. If there is a lifecycle hook, you might need to either complete the lifecycle hook or wait for the timeout period to end. For more information, see the following section "My instance is in stuck in the Pending:Wait or Terminating:Wait state during scaling activity".
My instance is stuck in the Pending:Wait or Terminating:Wait state during scaling activity
If you configured a lifecycle hook for your Amazon EC2 Auto Scaling group, an instance might be paused in the Pending:Wait or Terminating:Wait state. Except for step scaling policies, other scaling activities are suspended until the instance moves out of the Pending:Wait or Terminating:Wait state.
To determine if you have a lifecycle hook configured, run the following AWS Command Line Interface (AWS CLI) command:
aws autoscaling describe-lifecycle-hooks --auto-scaling-group-name my-asg-name
Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.
To continue the scaling process, wait for the timeout period to end (one hour by default), or complete the lifecycle hook.
For information on managing lifecycle hooks from the Amazon EC2 console, see Amazon EC2 Auto Scaling lifecycle hooks.
My Amazon EC2 Auto Scaling group isn’t responding to a scheduled action
- Check the time zone you configured for the scheduled action. You must schedule actions in Coordinated Universal Time (UTC).
- Review your Amazon EC2 Auto Scaling group’s activity history from the Amazon EC2 console. Verify if there is any scaling activity that conflicts with your scheduled action.
- If you're scheduling scale-out and scale-in actions, check that you scheduled one action for scaling out and another action for scaling in. You can't use the same scheduled action to both scale in and scale out.
I reached my Amazon EC2 instance quota
If your Amazon EC2 Auto Scaling group isn't scaling due to your EC2 instance quota, you receive a message similar to the following:
Launching a new EC2 instance. Status Reason: Your quota allows for 0 more running instance(s). You requested at least 1. Launching EC2 instance failed.