How do I verify that EC2 Auto Scaling lifecycle actions complete when using AWS CodeDeploy?
Last updated: 2022-08-31
I’m using AWS CodeDeploy with Amazon Elastic Compute Cloud (Amazon EC2) Auto Scaling lifecycle hooks. However, lifecycle actions don’t complete and EC2 instances don’t launch.
CodeDeploy uses Amazon EC2 Auto Scaling lifecycle hooks to perform deployments. Lifecycle hooks might influence the sequence of EC2 instance lifecycle states due to:
- Failed deployments which cause related lifecycle actions to fail.
- Multiple deployment groups associated with a single EC2 Auto Scaling group which exceed CodeDeploy limits and cause timeouts.
- Leftover lifecycle hooks associated with deleted applications which cause timeouts.
Review the scaling activities history of your Auto Scaling group:
- In the EC2 console navigation pane, choose Auto Scaling Groups under Auto Scaling. In the content pane, select your Auto Scaling group. Then choose the Activity History view.
- In the AWS Command Line Interface (AWS CLI), use the command describe-scaling-activities.
Then, check for the following issues in the history.
The following error typically indicates a failed deployment with CodeDeploy:
Description: Launching a new EC2 instance: i-f362a82e. Status Reason: Instance failed to complete user's Lifecycle Action: Lifecycle Action with token 0fc62a4b-153b-4dba-9bf4-9f233e72a697 was abandoned: Lifecycle Action Completed with ABANDON Result Cause: At 2016-07-22T16:16:56Z an instance was started in response to a difference between desired and actual capacity, increasing the capacity from 2 to 3.
Multiple deployment groups associated with a single EC2 Auto Scaling group
The following error typically occurs when multiple deployment groups are associated with a single EC2 Auto Scaling group:
Description: Launching a new EC2 instance: i-649f40a2. Status Reason: Instance failed to complete user's Lifecycle Action: Lifecycle Action with token 95c60857-3c0a-4a67-8518-6dc18797e421 was abandoned: Heartbeat Timeout Cause: At 2016-07-23T05:08:35Z an instance was started in response to a difference between desired and actual capacity, increasing the capacity from 1 to 2.
This error indicates a timeout error due to the five-minute limit in CodeDeploy. This timeout can occur when multiple deployments send commands to the CodeDeploy agent at the same time.
Consolidate your deployments in a single deployment group. Be sure to delete unnecessary deployment groups to prevent unnecessary lifecycle hooks. For more information, see Working with Deployment Groups in CodeDeploy.
Leftover lifecycle hooks from deleted applications
Check for active lifecycle hooks associated with deployment groups that have deleted applications. This issue can occur when an application is deleted before its deployment groups are deleted.
Conclude the lifecycle hooks using EC2 instances in an Auto Scaling group fail to launch and receive the error "Heartbeat Timeout.” To understand the result when you conclude a lifecycle hook, check for '--lifecycle-action-result" parameter in CompleteLifecycleAction API call in Cloudtrail.