Why doesn’t my Amazon EC2 Auto Scaling policy trigger when my CloudWatch alarm changes states?

Last updated: 2020-12-17

I've configured an Amazon CloudWatch alarm to trigger my Amazon Elastic Compute Cloud (Amazon EC2) Auto Scaling policy. Why doesn't my Amazon EC2 Auto Scaling policy trigger when my CloudWatch alarm changes state?

Short description

When a CloudWatch alarm transitions to a new alarm state (OK, ALARM, or INSUFFICIENT_DATA), the alarm invokes any configured actions for that state. Amazon EC2 Auto Scaling only uses the period configured on the alarm to determine if the state should change. So, for Auto Scaling actions, the alarm continues to invoke the configured action for every minute that the alarm remains in the new state, regardless of the configured period.

Common reasons why a CloudWatch alarm state change doesn't trigger an Amazon EC2 Auto Scaling policy include:

  • The Auto Scaling action isn't enabled for the CloudWatch alarm, which prevents the scaling policy from being invoked.
  • The scaling policy in the Auto Scaling group is disabled. A disabled policy prevents the group from being evaluated.
  • The Auto Scaling group has conflicting simple scaling policies or step scaling policies, which prevent some of the policies from being triggered.
  • The Auto Scaling group has an incomplete lifecycle hook, which prevents all simple scaling policies from being applied. A pending instance also causes delays in step and target tracking scaling policies. This is because Auto Scaling doesn't count the instance towards the group's capacity until after the lifecycle hook finishes and the warmup time completes (for scale out). The instance is still counted towards the group's capacity (for scale in) to prevent overly scaling. lifecycle hook completes when it times out or when a CompleteLifecycleAction API or AWS Command Line Interface (AWS CLI) call is made.

Resolution

Before you start, be sure that your CloudWatch alarm is transitioning to the ALARM state. If an alarm's configuration doesn't match the threshold of the metric that it's monitoring, the alarm might not transition to the ALARM state. If an alarm doesn't change states, it doesn't trigger Amazon EC2 Auto Scaling policies. For more information on how CloudWatch alarms are evaluated, see Evaluating an Alarm.

Verify that your CloudWatch alarm enters the ALARM state when expected by checking the alarm's Threshold value. Increase or decrease the Threshold to match your expected value. Also, review the Period and Evaluation Period of the alarm. You might need to edit your alarm Period and Evaluation Period to trigger your Amazon EC2 Auto Scaling policy as expected. For more information on how to be sure your alarm triggers actions, see How can I be sure that CloudWatch alarms trigger actions?.

Important: When creating or editing alarms, keep the following points in mind:

  • Be sure that you haven't suspended scaling processes (AlarmNotification, Launch, or Terminate) for your Amazon EC2 Auto Scaling group. Be sure to resume these scaling processes if they're suspended.
  • Never directly edit the alarms associated with target tracking policies. Editing these alarms might cause unintended effects. The threshold of these alarms is automatically determined based on the target value set in the scaling policy.

Check if Amazon EC2 Auto Scaling actions are enabled for the CloudWatch alarm

For a CloudWatch alarm to invoke an Amazon EC2 Auto Scaling policy, the ActionsEnabled parameter must be enabled in the alarm's configuration. Be sure the ActionsEnabled parameter is true in your alarm's configuration.

Note: If you create or update your alarm using the CloudWatch console, the ActionsEnabled parameter is set to true by default.

To check and enable alarm actions using the AWS CLI:

Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.

1.    Check your current configuration using describe-alarms as follows. Be sure to replace myalarm with your alarm's ID.

aws cloudwatch describe-alarms --alarm-names "myalarm" --query 'MetricAlarms[].ActionsEnabled'

2.    Review the output. If the ActionsEnabled parameter is not set to true, enable alarm actions using enable-alarm-actions as follows. Be sure to replace myalarm with your alarm's ID.

aws cloudwatch enable-alarm-actions --alarm-names myalarm

To check and enable alarm actions using the CloudWatch API:

1.    Check your current configuration using DescribeAlarms.

2.    If actions are not enabled for your alarm, enable actions using EnableAlarmActions.

Check the simple scaling and step scaling policies for your Amazon EC2 Auto Scaling group

To check your group's scaling policies using the Amazon EC2 console:

1.    Sign in to the Amazon EC2 console.

2.    In the navigation pane, under Auto Scaling, choose Auto Scaling Groups.

3.    In the content pane, select your Auto Scaling group.

4.    Choose the Automatic Scaling tab.

5.    Note the Policy type (Step scaling, Simple scaling, or Target tracking).

To check scaling policies using the AWS CLI, use the command describe-policies for the ID of your Auto Scaling group with the --policy-types parameter. The output lists the policies of each type (SimpleScaling, StepScaling, or Target tracking).

To check scaling policies using an API, use the call DescribePolicies with the parameter PolicyTypes. The output lists the policies of each type (SimpleScaling, StepScaling, or Target tracking).

If you have one simple scaling policy in effect, then any other simple scaling policies aren't invoked until the following conditions are met:

  • The simple scaling policy currently in effect has completed.
  • The cooldown period for the Amazon EC2 Auto Scaling policy has elapsed. A simple scaling policy honors the Amazon EC2 Auto Scaling policy's default or specified cooldown period.

Note: The execution of a simple scaling policy doesn't completely block the execution of step scaling or target tracking policies. Be sure that contradictory policies aren't applied at the same time.

Check for Auto Scaling lifecycle hooks in your Amazon EC2 Auto Scaling policy

When an Auto Scaling lifecycle hook is in effect, simple scaling policies aren't executed. If you use a simple scaling policy in your Auto Scaling group, be sure to stop any lifecycle hooks.

Note: Step scaling policies still trigger if there is a lifecycle hook in progress. However, the policies scale slowly because the instances don't start their warmup timer until after the lifecycle hook ends.

Check that all lifecycle hooks are completed with a result of CONTINUE or ABANDON after their global timeout periods or heartbeat timeout periods expire.

To check for lifecycle hook actions using the Amazon EC2 console:

1.    Sign in to the Amazon EC2 console.

2.    In the navigation pane, under Auto Scaling, choose Auto Scaling Groups.

3.    In the content pane, select your Auto Scaling group.

4.    Choose the Activity tab and then scroll to the Activity history section.

5.    Review the activity for any ongoing lifecycle hook actions.

6.    For steps to end a lifecycle hook, see Complete the Lifecycle Hook. To complete lifecycle hook actions using the AWS CLI, use the command complete-lifecycle-action. To complete lifecycle hook actions using an API, make a CompleteLifecycleAction call.