Event-driven framework to integrate AWS Backup service with CSPM tools

Many organizations use third-party Cloud Security Posture Management (CSPM) tools (for example Wiz.io) to continuously detect and remediate misconfiguration from build time to runtime across hybrid clouds such as AWS. CSPM tools often use AWS resource tags to enhance their security and compliance monitoring capabilities. Tags are key-value pairs that you can assign to AWS resources to help you manage, organize, and secure them. CSPM tools can use these tags to:

• Identify and group resources based on specific criteria (for example environment, application, and owner)
• Apply security and compliance rules based on tag values
• Generate reports and visualizations that are tailored to your organization’s tagging conventions

Organizations typically use AWS Backup for centralized backup management, compliance, and governance. AWS Backup provides Data protection of application resources on AWS and hybrid services, multi-account and multi-Region ransomware recovery and data protection compliance with real-time analytics and insights. This is the full list of features supported by AWS Backup. AWS Backup efficiently manages and automates backup processes, generating events associated with backup activities. These events can be used to apply tags to the original resources, enabling CSPM (Cloud Security Posture Management) tools to effectively check for compliance by examining tags on the backed-up resources, thus strengthening the overall compliance posture of the organization.

This post demonstrates using an event-driven framework that uses Amazon EventBridge to respond to the events generated by AWS Backup, trigger an AWS Lambda function to use the event metadata, and create tags for the corresponding AWS service backed up. These tags can be integrated with CSPM tools, helping meeting the organizational compliance and security needs.

Solution overview

In this post, we walk you through the process of using this event-driven framework (depicted in the following picture) using Amazon S3 as an example AWS service backed up using AWS Backup. However, the framework can be extended to the AWS services supported by AWS Backup. This is a list of the services supported by AWS Backup.

Figure 1: Overview of event-driven framework for storing AWS resource metadata as tags

Prerequisites

The following prerequisites are necessary to deploy the solution:

• An AWS account with an AWS Identity and Access Management (IAM) role and user with permissions to create and manage the necessary resources and components for the application. If you don’t have an AWS account, then observe the post How do I create and activate a new Amazon Web Services account?
• An S3 bucket with versioning enabled. For testing the solution described in this post, follow the steps to create an S3 bucket and enable versioning.

Walkthrough

In the following steps we demonstrate how the events generated from EventBridge during an AWS Backup process are used to trigger a Lambda function to update tags on the S3 bucket.

1. Create an IAM role to be associated with AWS Backup using the following steps:

1. Sign in to the AWS Management Console and open the AWS IAM console.

2. In the navigation pane of the IAM console, choose Roles, and then choose Create role.

3. For Trusted entity type, choose AWS service.

4. For Service or use case, choose S3.

5. Choose Next.

6. For Permissions policies, choose AWSBackupServiceRolePolicyForS3Backup and AWSBackupServiceRolePolicyForS3Restore.

7. Choose Next.

8. For Role name and Description. Enter a meaningful name to identify this role.

9. Review the role, and then choose Create role.

2. Create an AWS Backup “On-demand backup” using the following steps:

1. Open the AWS Backup console.

2. On the dashboard, choose Create an on-demand backup. Or, in the navigation pane, choose Protected resources and then choose Create an on-demand backup.

3. For Resource type page, choose S3 for Amazon S3.

4. Choose the Bucket name created in the Prerequisites as the S3 bucket to protect.

5. Make sure that Create backup now is selected.

6. For Total retention period, specify the number of days.

7. Choose the default Backup vault.

8. For IAM role, choose the role that you created in the previous step.

9. Choose Create on-demand backup. This opens the Jobs page, where you can observe a list of jobs and view job status.

3. Create a Lambda function that EventBridge triggers when an AWS Backup event is generated by the AWS Backup service. To do this:

1. Open the Lambda console.

2. Choose Create function.

3. Configure the following settings:

1. Function name: Enter a name for the function.

2. Runtime: Choose Python 3.10 or greater.

4. Choose Create function.

5. The console creates a Lambda function with a single source file named lambda_function. Edit this file and add the following code in the built-in code editor. To save your changes, choose Save.

import boto3
from botocore.exceptions import ClientError
import json

def lambda_handler(event, context):
    print("Received event: ", json.dumps(event, indent=2))

    try:
        # Extract relevant information from the event
        event_detail = event["detail"]
        resource_type = event_detail["resourceType"]
        resource_arn = event_detail["resourceArn"]
        state = event_detail["state"]
    except KeyError as e:
        print(f"Error extracting event information: {str(e)}")
        return {
            "statusCode": 400,
            "body": json.dumps({
                "message": "Error extracting event information"
            })
        }

    # Process the event if the resource type is S3 and the state is "COMPLETED"
    if resource_type == "S3" and state == "COMPLETED":
        try:
            # Extract the recovery point ID from the recovery point ARN
            recovery_point_id = event["resources"][0].split(':')[-1]

            # Extract the S3 bucket name from the resource ARN
            s3_bucket_name = resource_arn.split(':')[-1].split('/')[-1]

            # Update the S3 bucket tags
            s3 = boto3.client('s3')
            try:
                # Get the existing tags
                response = s3.get_bucket_tagging(Bucket=s3_bucket_name)
                existing_tags = response['TagSet']
            except ClientError as e:
                if e.response['Error']['Code'] == 'NoSuchTagSet':
                    existing_tags = []
                else:
                    raise e

            # Update the tags with the new values
            new_tags = {
                'BackupState': state,
                'BackupCompletionDate': event_detail["completionDate"],
                'recoverypointid': recovery_point_id
            }

            # Merge the new tags with the existing tags
            tags_to_update = [tag for tag in existing_tags if tag['Key'] not in new_tags]
            tags_to_update.extend([{'Key': key, 'Value': value} for key, value in new_tags.items()])

            # Update the bucket tags
            s3.put_bucket_tagging(Bucket=s3_bucket_name, Tagging={'TagSet': tags_to_update})

            print(f"Updated S3 bucket {s3_bucket_name} tags with BackupState={state}, BackupCompletionDate={event_detail['completionDate']}, and recoverypointid={recovery_point_id}")
            
            return {
                "statusCode": 200,
                "body": json.dumps({
                    "message": "Event processed successfully!"
                })
            }
        except (IndexError, ClientError) as e:
            print(f"Error extracting recovery point information or updating S3 bucket tags: {str(e)}")
            return {
                "statusCode": 400,
                "body": json.dumps({
                    "message": "Error extracting recovery point information or updating S3 bucket tags"
                })
            }
    else:
        print(f"Skipping event with resource type {resource_type} and state {state}")
        return {
            "statusCode": 200,
            "body": json.dumps({
                "message": "Event skipped as it doesn't match the criteria."
            })
        }

Lambda Code Snippet

6. Update the function timeout to 15 min. To do this:

1. Choose the Configuration tab and then choose General configuration

2. Under General configuration, choose Edit.

3. For Timeout, set 15 min.

4. Choose Save.

7. Update the function’s execution role to add the permissions needed to update tags on an S3 bucket. To do this:

1. Choose Configuration, and then choose Permissions.

2. Under Execution role, choose Edit.

3. Choose the existing role. This opens the role in the AWS IAM console.

4. Attach the AWSLambdaBasicExecutionRole AWS Managed Policy.

5. Under Permission policies, choose Create inline policy under Add Permissions.

6. Under Select a service, choose S3.

7. Under Actions allowed, choose GetBucketTagging and PutBucketTagging.

8. Under Resources, choose Specific, choose Add ARNs, and choose the Amazon Resource Name (ARN) of the S3 bucket created in the Prerequisites section.

9. Choose Next. 

10. Review Policy details and choose Create Policy.

4. Create an EventBridge rule that looks for an event pattern generated when AWS Backup runs and invokes the Lambda function. To do this:

1. Open the EventBridge console.

2. In the navigation pane, choose Rules.

3. Choose Create rule.

4. Enter a Name and, optionally, a Description for the rule.

5. For Event bus, select default. When an AWS service in your account emits an event, it always goes to your account’s default event bus.

6. For Rule type, choose Rule with an event pattern.

7. Choose Next.

8. For Event source, choose Other.

9. In the Sample events section, choose an AWS events.

10. For Creation method, choose Custom pattern (JSON editor) and enter the following JSON in the Event pattern. In this post we’re using the “Backup Job State Change” event to capture the metadata as tags. This can be customized and further expanded to monitor other AWS Backup events. This is a list of the AWS backup events.

{
    "source": [
        "aws.backup"],
    "detail-type": ["Backup Job State Change"],
    "detail": {
        "resourceType": ["S3"],
        "state": ["COMPLETED"]
    }
}

Event pattern

11. Choose Next.

12. For Target types, choose AWS service, choose Lambda function as the target, and choose the function created in the previous steps.

13. Choose Next.

14. (Optional) Enter desired tags for the rule. Choose Next.

15. Review the rule and choose Create rule.

Testing

Follow these steps to test the event-driven framework set up in the previous steps:

1. Go to the AWS Backup console and create an on-demand backup for the protected resource, in this case the S3 bucket created in the Prerequisites section. To do this:
   1. Open the AWS Backup console.
   2. Go to Protected resources in the left navigation and choose the resource ID associated with the S3 bucket protected.
   3. Choose Create on-demand backup.
   4. Choose Choose an IAM role and select the IAM role created for the AWS Backup in previous steps.
   5. Choose Create on-demand backup. This should start the AWS Backup process for the S3 bucket protected, generate AWS Backup events, trigger the Lambda function and create the following tags on the S3 bucket.
      1. recoverypointid: This is the unique recovery point ID generated during the AWS Backup process.
      2. BackupState: This conveys the state of the backup (for example COMPLETED).
      3. BackupCompletionDate: This conveys the timestamp of backup completion.

Cleaning up

If you’re experimenting using the steps in this post, then delete the resources created to avoid incurring costs. To do this:

1. Delete the S3 bucket created. Refer to these steps listed to delete the bucket.
2. Delete the Lambda function created.
   1. Open the Lambda console.
   2. Select the function created for testing, choose Actions, and choose Delete. Confirm deleting by typing “delete” in the confirmation field and choose Delete.
3. Delete the EventBridge rule created.
   1. Open the EventBridge console.
   2. In the navigation pane, choose Rules.
   3. Choose the rule created for testing, and choose Delete. Confirm deleting by typing “delete” in the confirmation field and choose Delete.
4. Delete the on-demand backup created. Refer to these steps.

Conclusion

In this post, we showcased how to set up and run an event-driven framework with AWS Backup service and use the events generated to apply resource tags to the backed up AWS resource. Then, the resource tags can be used by CSPM tools to monitor the environments for security and compliance.

Try out the solution in this post, and leave your feedback and questions in the comments section. For more information on AWS Backup, check out the developer guide.