How do I delete an AWS Lambda-backed custom resource that's stuck in DELETE_FAILED status or DELETE_IN_PROGRESS status in CloudFormation?

Last updated: 2021-03-19

My custom resource is stuck in DELETE_FAILED status or DELETE_IN_PROGRESS status in AWS CloudFormation. I want to delete my custom resource.

Short description

Choose one of the following solutions:

  • Delete a custom resource that's stuck in DELETE_FAILED status
  • Delete a custom resource that's stuck in DELETE_IN_PROGRESS status

Resolution

Delete a custom resource that's stuck in DELETE_FAILED status

You get an error message if you try to delete a stack when your AWS Lambda-backed custom resource is missing the logic to handle delete requests or has bugs in its delete logic. The error message is: "Custom Resource failed to stabilize in expected time." The status of the stack changes to DELETE_FAILED.

To delete your stack, complete the following steps:

1.    Open the CloudFormation console.

2.    Choose the stack that contains your custom resource that's stuck in DELETE_FAILED status.

3.    Choose Actions, and then choose Delete Stack.

4.    In the pop-up window that provides a list of resources to retain, choose the custom resource that's stuck in DELETE_FAILED status. Then, choose Delete.

5.    Choose Actions, and then choose Delete Stack.

The status of your stack changes to DELETE_COMPLETE.

Note: Your custom resource isn't a physical resource, so you don't have to clean up your custom resource after stack deletion.

Delete a custom resource that's stuck in DELETE_IN_PROGRESS status

If your stack gets stuck in DELETE_IN_PROGRESS status while deleting a custom resource, then your stack might be missing the logic to handle delete requests. Or, your stack might include incorrect delete logic.

To force the stack to delete, you must manually send a SUCCESS signal. The signal requires the ResponseURL and RequestId values, which are both included in the event that's sent from CloudFormation to Lambda.

1.    To print the event to your logs and retrieve the values of ResponseURL and RequestId from the logs, include the following logic at the beginning of your Lambda function.

Note: If you already have the following logic in your Lambda function, skip to step 2.

Example logic:

exports.handler = function(event, context) {
    console.log("REQUEST RECEIVED:\n" + JSON.stringify(event));
...
}

Important: If your Lambda function doesn't include the logic to print the event to logs, then you can't get the ResponseURL and RequestId values that are required for sending a manual signal. In this case, you must wait for approximately one hour for the CloudFormation stack to timeout and go to the DELETE_FAILED state. Then, complete the steps in the Delete a custom resource that's stuck in DELETE_FAILED status section to delete your stack.

2.    In your CloudFormation template, identify the name of the Lambda function where your custom resource is sending requests. You can find the function name from the ServiceToken property of the AWS::CloudFormation::CustomResource or Custom::String resource. For example:

MyCustomResource: 
  Type: "Custom::PingTester"
  Properties: 
    ServiceToken:
      !Sub |
        arn:aws:lambda:us-east-1:111122223333:function:awsexamplelambdafunction

3.    Open the Lambda console.

4.    In the navigation pane, choose Functions, and then choose the function that you identified in step 1. For example, awsexamplelambdafunction is the function name in the preceding code example.

5.    Choose the Monitoring tab, and then choose View logs in CloudWatch.

6.    In the Amazon CloudWatch console, choose the latest log.

Note: You can see the CloudWatch logs only if your function has access to Amazon CloudWatch Logs for log streaming.

7.    In the latest log, identify the event where RequestType is set to Delete, and then copy the values for RequestId, ResponseURL, StackId, LogicalResourceId, and PhysicalResourceId. For example:

Received event: {
  "RequestType": "Delete",
  "ServiceToken": "arn:aws:lambda:us-east-1:111122223333:function:awsexamplelambdafunction",
  "ResponseURL": "https://cloudformation-custom-resource-response-useast1.s3.us-east-1.amazonaws.com/arn%3Aaws%3Acloudformation%3Aus-east-1%3A111122223333%3Astack/awsexamplecloudformation/33ad60e0-5f25-11e9-a734-0aa6b80efab2%7CMyCustomResource%7Ce2fc8f5c-0391-4a65-a645-7c695646739?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20170313T0212304Z&X-Amz-SignedHeaders=host&X-Amz-Expires=7200&X-Amz-Credential=QWERTYUIOLASDFGBHNZCV%2F20190415%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=dgvg36bh23mk44nj454bjb54689bg43r8v011uerehiubrjrug5689ghg94hb",
  "StackId": "arn:aws:cloudformation:us-east-1:111122223333:stack/awsexamplecloudformation/33ad60e0-5f25-11e9-a734-0aa6b80efab2",
  "RequestId": "e2fc8f5c-0391-4a65-a645-7c695646739",
  "LogicalResourceId": "MyCustomResource",
  "PhysicalResourceId": "test-MyCustomResource-1URTEVUHSKSKDFF",
  "ResourceType": "Custom::PingTester"

8.    To send a SUCCESS response signal in the response object to the delete request, run the following command. Be sure to include the values that you copied from step 7.

$ curl -H 'Content-Type: ''' -X PUT -d '{
    "Status": "SUCCESS",
    "PhysicalResourceId": "test-CloudWatchtrigger-1URTEVUHSKSKDFF",
    "StackId": "arn:aws:cloudformation:us-east-1:111122223333:stack/awsexamplecloudformation/33ad60e0-5f25-11e9-a734-0aa6b80efab2
  ",
    "RequestId": "e2fc8f5c-0391-4a65-a645-7c695646739",
    "LogicalResourceId": "CloudWatchtrigger"
  }' 'https://cloudformation-custom-resource-response-useast1.s3.us-east-1.amazonaws.com/arn%3Aaws%3Acloudformation%3Aus-east-1%3A111122223333%3Astack/awsexamplecloudformation/33ad60e0-5f25-11e9-a734-0aa6b80efab2%7CMyCustomResource%7Ce2fc8f5c-0391-4a65-a645-7c695646739?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20170313T0212304Z&X-Amz-SignedHeaders=host&X-Amz-Expires=7200&X-Amz-Credential=QWERTYUIOLASDFGBHNZCV%2F20190415%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=dgvg36bh23mk44nj454bjb54689bg43r8v011uerehiubrjrug5689ghg94hb
  '

The status of your custom resources changes to DELETE_COMPLETE in the CloudFormation stack events.