How to get a daily report for your resources configuration changes
AWS allows customers to build, experience, and innovate in their AWS accounts, resulting in dynamic environments.
You can manage your resources changes using different controls, such as:
- Preventive controls with AWS Identity and Access Management (IAM) policies
- Detective controls with AWS Config Rules
- Preventive and Detective controls with continuous integration and continuous delivery (CI/CD) pipelines for Infrastructure as Code (IaC)
In addition to the existing controls, Security, Governance, and Operations teams might want additional visibility over the resource configuration changes. AWS Config lets you continuously monitor and record your AWS configurations for supported services. Using advanced queries, we can run queries against our resources inventory.
In this post, you’ll learn how to generate a scheduled resource configuration report. The report will include newly-added resources and resource configuration changes across your multi-account and multi-region AWS environment. Governance teams can review the report, identify risks, track the latest changes, and gain visibility over the environment.
We’ll also share code that will provision a daily AWS Config query, with a CSV report generator and email sender. As a result, the recipients can receive a detailed scheduled report.
All of the samples from the post can be found in this Github repository.
The following diagram describes the AWS Config Daily Reporter flow:
- The Lambda function will run a query against AWS Config, thereby getting the changed configuration during the current day.
- The Lambda function will generate a CSV report and send it via email using Amazon Simple Email Service (Amazon SES).
Before getting started, make sure that you have a basic understanding of the following:
- Amazon EventBridge rule that runs on a schedule
- Multi-Account Multi-Region Data Aggregatione
- AWS Lambda Function
- Python and Boto3.
- CDK environments.
The Python code in this post was written using the AWS Cloud Development Kit (AWS CDK). To view the code, see the associated GitHub repository. If you’re unfamiliar with AWS CDK, then see Getting started with AWS CDK.
For the deployment, use the AWS CDK
stack cdk/cdk/cdk_stack.py. This stack requires the following parameters:
- aggregator – Name of AWS Config Aggregator.
- sesarn – The Amazon SES arn.
- RECIPIENT – Email recipient that will get the CSV report.
- SENDER – Email sender as configured on Amazon SES.
- HOUR – The hour (UTC) that the Lambda will run.
- MINUTE – The minute (UTC) that the Lambda will run.
AWS Config Aggregator
An aggregator is an AWS Config resource type that collects AWS Config configuration data from multiple accounts and regions.
Our solution will use an aggregator to run queries against the environment.
Set Up an Aggregator Using the Console and note your AWS Config aggregator name.
Amazon SES is a cost-effective, flexible, and scalable email service that enables developers to send mail from within any application.
In our solution, we will use Amazon SES to send the configuration changes report CSV via Email.
Setup Amazon Simple Email Service and make a note of your email sender and the Amazon SES arn.
- Make sure you are logged in to the AWS management console, and have configured your AWS CLI credentials. More information can be found here.
- Clone the solution repository
- Navigate to the cdk directory of the cloned repository:
- Run cdk bootstrap:
- Deploy the solution:
The “cdk deploy” command adds a new AWS CloudFormation template, which creates a Lambda Function with a daily trigger. At the end of the deployment, the Lambda function will be triggered automatically, and you should get an email a few minutes later.
Review the code
The config_reporter Lambda function is written in Python 3.8. In this section, we’ll go over the code which can be found here.
The create_report function will run a query against the configured AWS Config Aggregator for any changes made during the last day:
For example, for June 09 2022, it will run the following query:
Next, we’ll transform the JSON results into a CSV file.
The CSV will be saved into the /tmp folder.
For each resource configuration change, we’ll add a column with a link to the resource in the AWS Config console. The get_link function will be used to generate the link:
Eventually, we’ll use the send_email function to send an email using Amazon SES, with the CSV report as an attachment:
As a result, we should get the CSV report with the last day configuration changes to the recipient mail box.
The following table represents an example of the report with one changed Amazon Elastic Compute Cloud (Amazon EC2) instance:
Selecting the link of the resource will take you to the resource timeline in the AWS Config console:
To avoid incurring future charges, delete all the resources used in this solution. Use the following steps to clean up the resources:
In order to delete the solution, please follow this steps:
In this post, we demonstrated how to run a scheduled advanced query against AWS Config, and generate daily configuration report changes.
Use this solution in order to get visibility over your daily changes in your multi-account and multi-region AWS environment.
You can also modify code to run the solution in a different frequency, or to run different queries. For example: query for specific resources.
About the author: