How to use AWS Config and CloudTrail to find who made changes to a resource

AWS Config is a service that continuously tracks and evaluates the configuration changes of your AWS resources. It provides you with a Resource Timeline which helps you to investigate and audit how the resources and their relationships change over time. AWS Config records the configuration of supported resources in a JSON structure known as a Configuration Item (CI).

While using the AWS Config Console, if you navigate to the Resource Timeline of certain resources, you can see the correlated Amazon CloudTrail event link if the event occurred in the last 90 days. AWS Config achieves this by calling the CloudTrail LookupEvents API. By selecting the link, AWS Config will redirect you to the CloudTrail event history. From the Amazon CloudTrail event you can get additional details including but not limited to who made changes to the resource and when.

In this post, we’ll discuss a solution to automate this process and get Amazon Simple Notification Service (Amazon SNS) notifications regarding which user principal made changes to an AWS resource and when.

Solution overview

In this solution, you’ll use a combination of AWS services:

1. The solution is self-contained and deployed using an AWS CloudFormation stack. By default, the stack will create the following resources:

• AWS EventBridge Rules
• Amazon SQS Queue and Queue Policy
• AWS Lambda Function and Event Source Mapping
• AWS IAM Role (AWS Lambda Function Execution Role)
• Amazon SNS Topic
• AWS Key Management Service (AWS KMS) Customer-managed Key
• Amazon CloudWatch Log Group

2. After deploying all resources, you can navigate to the Amazon SNS console and create a subscription, such as an Email notification for the SNS topic.
3. When ready, you can enable the AWS EventBridge Rule. As soon as the Rule is enabled, it will begin to capture the ConfigurationChangeNotification.
4. You can further modify this template and AWS Lambda Function to perform other automation. The template in this post demonstrates how to use Amazon SNS to publish notifications, for example.

How the solution works

The following diagram illustrates the solution architecture:

Figure 1. Solution overview

1. Whenever a create, update or delete event occurs for a resource that’s supported by AWS Config, the AWS Config Recorder will record the configuration change as a CI for that resource to track the changes. Note that AWS Config is a regional service. Therefore, the solution must be deployed on a regional basis.
2. The EventBridge Rule can capture the Config CI Change event notifications for the resource types that are being recorded. The CloudFormation template has a parameter to set the event pattern filter for certain resources types. By default, the event pattern will capture all of the resources type(s) your Configuration Recorder is currently recording. This doesn’t affect the Configuration Recorder.
3. Once captured, the EventBridge Rule then passes the message to an SQS queue.
4. The SQS queue has a 10 minute delivery delay, and after that it triggers the Lambda function to poll the message from the SQS queue. The 10 minute delay allows sufficient time for the resource configuration change event to be propagated in CloudTrail.
5. Then the Lambda function extracts the resource details from the polled message, and calls the CloudTrail LookupEvents API based on the resource name or resource Id for 15 minutes before the CI creation. The LookupEvents API returns the correlated API events. Because it takes a short moment for AWS Config Recorder to record the resources, the API event time will have an earlier timestamp than the CI capture time.
6. Finally, the Lambda Function extracts the event details, such as the user principal, eventName, eventTime, etc., in the retrieved API event for further automations. In this demo, the Lambda Function will parse the event details in plaintext, add Config and CloudTrail console URLs that link to the events, and publish them to an Amazon SNS topic.

Note: In some situations, the LookupEvents API won’t be able to return an event related to the CI recorded; thus, the Lambda function will publish only a direct console URL to the resource CI without event details. This is because some resource types currently don’t support LookupEvents API by resource name/ID, while some resource CIs could have been recorded due to changes in ephemeral workloads or during the Config Recorder baselining period; these operations might not have any correlated API event.

Prerequisites

1. Make sure that AWS Config is enabled in the Region where you’re deploying this solution.
2. Make sure that the user creating the stack has cloudformation:CreateStack permission, while the CloudFormation service role or the AWS Identity and Access Management (IAM) entity used for the stack deployment has the following permissions:

Steps to deploy the solution

1. Save the YAML template available at this GitHub link. You can customize the template according to your use case if needed.
2. On the CloudFormation console, choose “Create stack” in the Region where your AWS Config setup exists.
3. Choose “With new resources” (standard).
4. Choose “Upload a template file, ” and choose the file that you saved.

Figure 2. Figure 2. Upload Template in the CloudFormation console

5. Choose “Next.”
6. Give a name to the stack and provide the following parameters (you can revise them depending on your use case):
   

   Figure 3. Input Parameters for the CloudFormation template

7. Choose “Next.”
8. In the next section, provide a CloudFormation service role with the permissions to create the resources requested in the stack. If no CloudFormation service role is provided, the stack will use the same identity that creates the stack to create the resources. The role should meet the permissions mentioned in the prerequisites section.
9. Choose “Next.”
10. On the last page, tick the option: “I acknowledge that AWS CloudFormation might create IAM resources with custom names.

Figure 4. Acknowledgement option to select

11. Choose “Create stack.”
12. After stack creation, you should see the following resources created, with the stack status as CREATE_COMPLETE:

• LookupCIEventsCloudWatchEventRule
• LookupCIEventsCloudWatchEventRuleOversized
• LookupCIEventsKMSCMK
• LookupCIEventsLambdaFunction
• LookupCIEventsLambdaFunctionEventSourceMapping
• LookupCIEventsLambdaFunctionEventSourceMappingDLQ
• LookupCIEventsLambdaFunctionExecutionRole
• LookupCIEventsLambdaFunctionLogGroup
• LookupCIEventsSNSTopic
• LookupCIEventsSQSPolicy
• LookupCIEventsSQSQueue
• LookupCIEventsSQSQueueDLQ

Figure 5. Resources Created by the CloudFormation Template

13. By default, the EventBridge event rules created by the CloudFormation template are disabled. This is to prevent premature notification capturing. You must enable them to send notifications to SQS. To enable the EventBridge event rules, navigate to the CloudFormation stack’s Resources tab, and select the link to the EventBridge console. Select “Enable” to enable the event rule. You can disable the event rules when you no longer want to capture and receive notifications.

Creating an SNS topic subscription

After the successful deployment of the CloudFormation template, you’ll have an SNS topic created for you and the notifications will be delivered to it. However, you must create and associate a subscription for this SNS topic as follows:

1. Open the Amazon SNS console in the same region, for example “us-east-1” where you created the CloudFormation stack.
2. On the Topics page select the LookupCIEventsSNSTopic topic.
3. Under the subscriptions tab, select “Create Subscription.”
4. From the drop down list under Protocol, select any endpoint type. For this demo, select “Email” so that you can receive an email notification about the CI change event.
5. Under Endpoint, enter the email address.
6. Finally, select “Create Subscription.“

After this you’ll receive an email with a link to confirm this subscription. Select the link to accept. You’ll start receiving SNS notifications once a CI is captured and processed by the solution. The following is a sample message:

Figure 6. Amazon SNS notification received on email

Setting up AWS billing alarms

In addition to the above solution, you can monitor your estimated AWS charges using Amazon CloudWatch and create a billing alarm. The billing alarm triggers if your account billing exceeds the threshold that you specify. For instructions, see Creating a Billing Alarm to monitor your estimated AWS charges. You can also find out the number of CIs recorded for a resource or resource type. This can help you to identify the resource that has contributed most of the charges. For instructions, see Identifying resources most Configuration Changes AWS Config.

Resource clean up

The resources created for the above demonstration are self-contained within the CloudFormation stack. Deleting the stack will delete all of the resources created by the stack. To delete the stack, follow these steps:

1. In the Region where this stack is deployed, navigate to the CloudFormation console.
2. Select the “Stacks” panel on the left, and then select the stack name.
3. On the top right hand side of the stack view, select “Delete” to initiate stack deletion.
4. You can track the deletion progress on the “Events” tab. Once the stack is deleted, the stack status will become “DELETE_COMPLETE”.

Conclusion

In this blog post, we proposed a sample solution to strengthen your detection and response to the root cause of the AWS Config resource Configuration Item (CI) change. Through the sample solution, you learned how to programmatically lookup events related to CI changes in AWS CloudTrail, extract information such as the user principal who made the change from the event, and send the information to an AWS SNS topic as a remedation.The code is available on GitHub. You can customize this solution and add additional remedations, such as performing additional lookup on the recent API calls made by the user principal, or automatically attaching a restrictive policy to the user principal if the resource change was unauthorized.Comments are highly appreciated, and please leave them in the Comments section below. If you have any question about this solution, you can also contact AWS Support.

About the authors:

Mohamed Attalla

Attalla is a Sr. Cloud Support Engineer. He specializes in AWS Config, Logging and Monitoring AWS services. Attalla is passionate about building secure and effective solutions for AWS customers. He enjoys deep dive into compliance related challenges, writing articles and building tutorials. Outside of work, Attalla’s favorite activities include snorkeling and reading. He loves spending time with his daughters.

Shashank Sharma

Shashank is a Cloud Support Engineer at AWS. He specialises in multiple services, being a subject matter expert in AWS Config and AWS Identity and Access Management (IAM) .Shashank possesses a knack for diving deep into customer issues and enjoys devising end to end solutions and also automating them. Outside of work, he is fond of scrumptious food, watching football and playing games on his xbox.

Man Man Chau

Man Man is an AWS Cloud Support Engineer. He solves problems involving complex systems, investigates application and security anomaly, and builds scalable automated solutions. Outside of work, he likes underbiking, spending time in the woods, and spreading memes. He is very proud of his two rescued dogs: Choco and Mochi.

Eduardo Ortiz Pineda

Eduardo is a Security Specialist at AWS Enterprise Support. He is interested in different security topics, automation and helping customers to improve their security posture. Outside of work he spends time with family and friends, enjoying sports, reading and traveling.

Dan Urbano

Dan is a Senior Software Engineer at AWS. He enjoys software architecture design and building services to meet customer needs. Outside of work, Dan loves to play with his dogs, watch baseball, and travel.

IAD, us-east-1, US East (N. Virginia)