How can I troubleshoot failed remediation actions in AWS Config?

Last updated: 2021-02-17

I followed the instructions for Remediating Noncompliant AWS Resources by AWS Config Rules. However, the remediation action failed, and the AWS Config console displays the Action status error "Action execution failed (details)". I opened the Details page, but it doesn't contain enough information to troubleshoot the issue.

Resolution

Follow these instructions to troubleshoot the remediation actions failure using the AWS Command Line Interface (AWS CLI) or AWS CloudTrail event history.

Note: If you receive errors when running the AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.

AWS CLI

1.    Run the AWS CLI command for describe-remediation-execution-status for a more detailed error message, state, and timestamps for remediation action steps similar to the following:

aws configservice describe-remediation-execution-status \
     --config-rule-name example-rule \
     --region example-region \
     --resource-keys resourceType=example-resource-type,resourceId=example-resource-ID

Note: Replace example-rule, example-region, example-resource-type, and example-resource-ID with your AWS Config rule name, Region, resource type, and resource ID.

2.    You receive an output similar to the following:

{
    "RemediationExecutionStatuses": [
        {
            "ResourceKey": {
                "resourceType": "AWS::EC2::Volume",
                "resourceId": "vol-0b399a24561582586"
            },
            "State": "FAILED",
            "StepDetails": [
                {
                    "Name": "createDocumentStack",
                    "State": "FAILED",
                    "ErrorMessage": "Automation Step Execution fails when it is creating a CloudFormation stack. 
Get Exception from CreateStack API of cloudformation Service. Exception Message from CreateStack API:
[User: arn:aws:sts::xxxxx:assumed-role/config-remediation-sshpublic-role-zkga0ot3/config-remediation-sshpublic is not authorized to perform: cloudformation:CreateStack 
on resource: arn:aws:cloudformation:eu-west-2:xxxxx:stack/DetachEBSVolumeStack2f6d3590-ea2c-424a-97ea-045749f07164/* 
(Service: AmazonCloudFormation; Status Code: 403; Error Code: AccessDenied; Request ID: b8f41dd6-9020-11e9-897d-f9719db1a9e6)]. 
Please refer to Automation Service Troubleshooting Guide for more diagnosis details.",
                    "StartTime": 1560680582.675,
                    "StopTime": 1560680582.884
                },
                {
                    "Name": "detachVolume",
                    "State": "PENDING"
                },
                {
                    "Name": "deleteCloudFormationTemplate",
                    "State": "PENDING"
                }
            ],
            "InvocationTime": 1560680582.419,
            "LastUpdatedTime": 1560680583.67
        }
    ]
}

3.    In the StepDetails list, note the error message and cause of the failure.

CloudTrail event history

1.    Open the AWS CloudTrail console.

2.    Follow the instructions for Viewing CloudTrail events in the CloudTrail console.

3.    The StartAutomationExecution API action is invoked when AWS Config initiates a remediation action. Filter Event name with the StartAutomationExecution API, and in the event details page, copy the requestID.

4.    Open the AWS Systems Manager console, and then choose Automation in the navigation pane.

5.    Paste the requestID in the Automation document search field.

6.    Then, find the action that the remediation performed. Verify that the error relates to AWS Identity and Access Management (IAM) permissions, syntax issues, or incorrect parameters configured in the remediation action.