Automate AWS Config reporting for noncompliant resources that have been non-compliant for a period of time

AWS Config evaluates the configuration settings of your AWS resources. You do this by creating AWS Config rules, which represent your ideal configuration settings. AWS Config provides customizable, predefined rules called AWS Managed Rules to help you get started. While AWS Config continuously tracks the configuration changes that occur among your resources, it checks whether these changes violate any of the conditions in your rules. If a resource violates a rule, AWS Config flags the resource and the rule as noncompliant.

In this post, we provide a solution that can be used to automate AWS Config reporting for resources that have been noncompliant for a period of time. You can choose the day and time for the report generation and get a list of resources that have been noncompliant for more than 30 days. Using this solution, you can avoid notifications from transient changes to your resources.

How AWS Config Works

AWS Config is a service that enables you to assess, audit and evaluate the configurations of your AWS resources. AWS Config tracks configuration changes and maintains a history of up to 7 years. Config also provides aggregated view of resource configuration and compliance status across accounts and regions. If you are using AWS Config rules, AWS Config continuously evaluates your AWS resource configurations for desired settings. Depending on the rule, AWS Config will evaluate your resources either in response to configuration changes or periodically. If a resource violates the conditions of a rule, AWS Config flags the resource and the rule as noncompliant. Refer getting started with AWS Config for more details. Figure 1 depicts the workflow of AWS Config

Figure 1. Workflow of AWS Config

Solution overview:

The solution in this blog post includes a serverless architecture for running a query against AWS Config aggregator. An Amazon EventBridge Rule triggers an AWS Lambda function at the start of every week to find the list of noncompliant resources from noncompliant rules that  are not remediated over a period of 30 days.
The following figure provides the architecture for our solution’s  end-to-end flow:

Figure 2. Solution workflow diagram

1. The Lambda function will run a query against AWS Config, thereby getting the list of noncompliant resources over a period of 30 days.
2. The Lambda function will generate a CSV report and send it via email using Amazon Simple Email Service (Amazon SES)

Prerequisites:

Before getting started, make sure that you have a basic understanding of the following:

• Amazon EventBridge rule that runs on a schedule
• AWS Config
• AWS Config Multi-Account Multi-Region Data Aggregation
• AWS Lambda Function
• Python and Boto3.
• CDK environments.

You will also need to configure the environment

1. AWS Config and AWS Config recorder

• Enable AWS Config in all the accounts in your AWS organization. You can use Quick Setup a capability of AWS Systems Manager, to help with the setup of the AWS Config recorder. Config recorder will detect changes in your resource configurations.

2. To Setup an Aggregator using the AWS Console

• Once the AWS Config Aggregator is configured, note your AWS Config aggregator name.

3. To Setup an Amazon Simple Email Service

• After the setup of Amazon SES, note the Amazon SES arn and email address of sender
• Note: You will need to enable SES production access if you want to send emails to an unverified address in SES.

4. Make sure you have the latest CDK CLI installed
5. Make sure below two libraries exists

python -m pip install aws-cdk-lib
pip install async-timeout

Solution Walkthrough

The Python code in this post was written using the AWS Cloud Development Kit (AWS CDK). To view the code, see the associated GitHub repository. If you’re unfamiliar with AWS CDK, then see Getting started with AWS CDK.

1. You will need to provide following parameter values for the stack:

• • Aggregator – Name of AWS Config Aggregator.
  • SESarn – The Amazon SES arn.
  • Recipient – Email recipient that will get the CSV report.
  • Sender – Email sender as configured on Amazon SES.
  • Weekday- Day-of-week 1-7 or SUN-SAT that the Lambda will run.
  • Hour – The hour (UTC) that the Lambda will run.
  • Minute – The minute (UTC) that the Lambda will run.

2. To get the noncompliant rules for aggregator.

• In this blog post,  we first  use the describe_aggregate_compliance_by_config_rules() API to get a  list of all noncompliant rules captured by the aggregator.

3. To get list of noncompliant resources.

• For each noncompliant rule captured, we run a query to get a  list of noncompliant resources and the recorded time for their noncompliant status using get_aggregate_compliance_details_by_config_rule API.

4. From the noncompliant list of resources, we further filter them  using a ‘Recorded Time’ > 30 days parameter
5. We create a CSV file with the list of non-complaint resources over a period of 30 days with details and send an email to recipient with CSV file as an attachment.

Solution Deployment:

1. Make sure you are logged in to the AWS management console, and have configured your AWS CLI credentials.
2. Clone the solution repository

git clone https://github.com/aws-samples/Config-report

3. Navigate to the cdk directory of the cloned repository

cd Config-Report/config_report

4. Run cdk bootstrap

cdk bootstrap

5. Deploy the Config Report solution

cdk deploy ConfigReport --parameters Aggregator=<name of the aggregator> 
--parameters Recipient=<Recipient email address > --parameters Sender=<Sender
email address > --parameters Weekday=<Day of Week> --parameters Hour=<Hour of
the day> --parameters Minute=<Minute of the day> --parameters SESarn=<SES ARN> 

The “cdk deploy” command adds a new AWS CloudFormation template, which creates a Lambda Function with a weekly trigger. At the end of the deployment, the Lambda function will be triggered automatically based on the weekday of your choice. As a result, you should get the CSV report in the recipient mail box.

Cleanup

To avoid recurring charges, and to clean up all the resources after trying the solution outlined in this post, perform the following:

1. Delete the AWS Config Aggregator.
2. Delete the Amazon SES.
3. Use cdk cli with the cdk destroy command to delete the CloudFormation Stack.

Are you sure you want to delete: cdk-stack (y/n)? y
cdk-stack: destroying.......
DELETION OF RESOURCES....

 ✅  cdk-stack: destroyed

It will not delete the stack that the bootstrap command created. If you want to delete that as well, you can do it from the AWS Console.

Conclusion

In this blog post we demonstrated a sample solution that can help you setup reports for resources that have been noncompliant for more than 30 days. You can also modify the code to run this solution at a different frequency, or to run different queries.

About the author: