How do I verify that EC2 Auto Scaling lifecycle actions complete when using AWS CodeDeploy?

3 minute read
0

I’m using AWS CodeDeploy with Amazon Elastic Compute Cloud (Amazon EC2) Auto Scaling lifecycle hooks. However, lifecycle actions don’t complete and EC2 instances don’t launch.

Short description

CodeDeploy uses Amazon EC2 Auto Scaling lifecycle hooks to perform deployments. Lifecycle hooks might influence the sequence of EC2 instance lifecycle states due to:

  • Failed deployments which cause related lifecycle actions to fail.
  • Multiple deployment groups associated with a single EC2 Auto Scaling group which exceed CodeDeploy limits and cause timeouts.
  • Leftover lifecycle hooks associated with deleted applications which cause timeouts.

Resolution

Review the scaling activities history of your Auto Scaling group:

  • In the EC2 console navigation pane, choose Auto Scaling Groups under Auto Scaling. In the content pane, select your Auto Scaling group. Then choose the Activity History view.
  • In the AWS Command Line Interface (AWS CLI), use the command describe-scaling-activities.

Then, check for the following issues in the history.

Failed deployments

The following error typically indicates a failed deployment with CodeDeploy:

Description: Launching a new EC2 instance: i-f362a82e. Status Reason: 
Instance failed to complete user's Lifecycle Action: Lifecycle Action 
with token 0fc62a4b-153b-4dba-9bf4-9f233e72a697 was abandoned: Lifecycle
 Action Completed with ABANDON Result Cause: At 2016-07-22T16:16:56Z 
an instance was started in response to a difference between desired and 
actual capacity, increasing the capacity from 2 to 3.

Check your deployment details to find out why your deployment failed. For more information, see Troubleshooting CodeDeploy.

Multiple deployment groups associated with a single EC2 Auto Scaling group

The following error typically occurs when multiple deployment groups are associated with a single EC2 Auto Scaling group:

Description: Launching a new EC2 instance: i-649f40a2. Status Reason: 
Instance failed to complete user's Lifecycle Action: Lifecycle Action 
with token 95c60857-3c0a-4a67-8518-6dc18797e421 was abandoned: Heartbeat
 Timeout Cause: At 2016-07-23T05:08:35Z an instance was started 
in response to a difference between desired and actual capacity, 
increasing the capacity from 1 to 2.

This error indicates a timeout error due to the five-minute limit in CodeDeploy. This timeout can occur when multiple deployments send commands to the CodeDeploy agent at the same time.

Consolidate your deployments in a single deployment group. Be sure to delete unnecessary deployment groups to prevent unnecessary lifecycle hooks. For more information, see Working with Deployment Groups in CodeDeploy.

Leftover lifecycle hooks from deleted applications

Check for active lifecycle hooks associated with deployment groups that have deleted applications. This issue can occur when an application is deleted before its deployment groups are deleted.

Conclude the lifecycle hooks using EC2 instances in an Auto Scaling group fail to launch and receive the error "Heartbeat Timeout.” To understand the result when you conclude a lifecycle hook, check for '--lifecycle-action-result" parameter in CompleteLifecycleAction API call in Cloudtrail.


Related information

Integrating CodeDeploy with Auto Scaling

AWS OFFICIAL
AWS OFFICIALUpdated 2 years ago
No comments