What should I do when my Amazon ECS cluster fails to delete as part of an AWS CloudFormation stack?

4 minute read
0

My Amazon Elastic Container Service (Amazon ECS) cluster fails to delete.

Short description

An Amazon ECS cluster might fail to delete due to an issue with underlying resource dependencies. When an Amazon ECS cluster is created, AWS CloudFormation creates resources, including Auto Scaling groups, virtual private clouds (VPC), or load balancers. These resources are associated with the cluster, and their presence can prevent the deletion of the cluster. Other issues with AWS CloudFormation can also prevent the deletion of an Amazon ECS cluster.

Note: Clusters created through the console first-run experience (after November 24, 2015) or the cluster creation wizard have an underlying AWS CloudFormation stack. During the cluster deletion process, your stack EC2ContainerService-yourClusterName might run into the following errors:

  • "The vpc 'vpc-1234567' has dependencies and cannot be deleted"
  • "The security group sg-123456 failed to delete due to the error "resource sg-123456 has a dependent object"
  • "User: arn:aws:sts::1111222233334444:assumed-role/example-role/example-user is not authorized to perform: ecs:DeleteCluster on resource: arn:aws:ecs:Region:1111222233334444:cluster/example-cluster"

Cluster deletion fails, and the AWS CloudFormation stack moves to the DELETE_FAILED state.

If you can't delete your ECS cluster, complete the following steps.

Resolution

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.

Verify the AWS Identity and Access Management (IAM) permissions

1.    Open the IAM console.

2.    In the navigation pane, choose Roles.

3.    In the Roles page, enter the IAM role that's specified in the error message. You can check the error message from CloudTrail Events by filtering to the DeleteCluster API calls.
Note: This is the IAM role or user that's trying to delete the Amazon ECS cluster.

4.    Choose the IAM role or user.

5.    Choose the Permissions tab.

Check if the permissions policy contains the ecs:Delete* permission. This permission is required by the IAM role or user to delete the ECS cluster.
If the permission is missing, grant the required access to the IAM user or role. For more information, see AmazonECS_FullAccess.

Delete the cluster by skipping the resources with dependencies

1.    Open the AWS CloudFormation console.

2.    To find the stack that failed, for Filter, choose Active, and then choose Failed.

3.    Select the failed stack that won't delete.

4.    Choose Actions, and then choose Delete Stack.

5.    Select the check boxes next to the resources that failed to delete.

6.    Choose Yes, Delete.

Important: If you can't delete a resource, but you still want to delete the stack, then choose to retain that resource. You can also retain resources by using the AWS CLI delete-stack command. Use the resource.

7.    Delete the Amazon ECS cluster.

Delete the retained resources

The following examples show you how to delete resources that are commonly associated with the underlying AWS CloudFormation stack.

Security group dependency example

In this example, the inbound or outbound rules of one security group refer to another security group or groups. This dependency can prevent the deletion of your cluster.

1.    To find security groups associated with the security group that you want to delete, run the following AWS CLI command:

aws ec2 describe-security-groups --filters Name=ip-permission.group-id,Values=[sg-xxxxxxxxx] --region us-east-1 | jq '.SecurityGroups[] .GroupId'

Note: jq is a command line JSON processor.

2.    Clear the dependencies for the security group that you want to delete.

3.    Delete the security group resource.

VPC dependency example

Note: Common VPC dependencies include describe-internet-gateways, describe-subnets, or describe-instances.

1.    To identify the dependent resources of your VPC, run the following AWS CLI command. Replace vpc-xxxxxxxx with your VPC value, and replace RegionId with your AWS Region code:

aws ec2 describe-subnets --filters "Name=vpc-id,Values=vpc-xxxxxxxx" --region RegionId | grep SubnetId

Note: You can run similar describe_* commands for other dependencies.

2.    Clear the dependencies for the VPC that you want to delete.

3.    Delete the VPC.

For more information, see I tried to delete my Amazon VPC, but I received a dependency error. How can I delete my Amazon VPC?


Related information

Delete stack fails

AWS OFFICIAL
AWS OFFICIALUpdated a year ago