AWS Cloud Operations Blog

Tag workloads with AWS Config conformance packs across AWS accounts

Overview

As cloud deployments get larger and more complex, Organizations struggle with managing and identifying a growing set of resources. Tags provide companies with metadata about their resources – they can use them to identify the resources for cost allocation, operations or data security.

AWS Config is an AWS service that continually assesses, audits, and evaluates the configurations and relationships of your AWS resources. It allows you to discover resources that exist in your account, record their configurations, and capture any changes, allowing you to troubleshoot operational issues quickly. In addition, by using AWS Config predefined customizable rules, you can evaluate whether your AWS resources comply with common best practices and optionally configure remediation actions for non-compliant resources using AWS Systems Manager runbooks. One such managed rule, required-tags, is a rule that checks if your resources have specific tags that you identify. For example, with this rule, you can check whether your Amazon Elastic Compute Cloud (Amazon EC2) instances have the tag-key CostCenter or not.

Solution Overview

In this post, we provide an example of a customized Automation runbook, a capability of AWS Systems Manager, that automatically remediates non-compliant resources evaluated by the AWS Config rule required-tags. This is especially useful in a multi-account environment where you have similar requirements across multiple AWS accounts. Since AWS Config and Systems Manager are regional services, you will need to deploy this solution to each region in every account that requires it.

We start by guiding you through creating a shared AWS Systems Manager Automation runbook from an AWS management account, then show you how to share it with member accounts in your AWS Organizations. We then combine the AWS Config required-tags rule and the remediation action using an automation runbook into an AWS Config conformance pack. A conformance pack is a collection of AWS Config rules and remediation actions that you can easily deploy as a single entity in an account or across an organization. Finally, we deploy it through AWS Organizations to selected AWS accounts.

Solutions Overview – The management account is the account that you use to create the organization. Member accounts make up all of the rest of the accounts in an organization. An account can be a member of only one organization at a time. This solution demonstrates how you can manage tag automation by deploying resources from a management account and shared with member accounts.

Figure 1. Solutions Overview – The management account is the account that you use to create the organization. Member accounts make up all of the rest of the accounts in an organization. An account can be a member of only one organization at a time. This solution demonstrates how you can manage tag automation by deploying resources from a management account and shared with member accounts.

Please note that this solution will only work with the services that Resource Group API supports. Please see here for a list of supported services.

Services used in this solution

AWS Config provides a detailed view of the configuration of AWS resources in your AWS account. This includes how the resources are related to one another and how they were configured in the past so that you can see how the configurations and relationships change over time.

AWS Systems Manager is a secure end-to-end management solution for hybrid cloud environments. We will use it to automatically remediate non-compliant resources using an Automation runbook.

AWS Organizations lets you create new AWS accounts at no additional charge. With accounts in an organization, you can easily allocate resources, group accounts, and apply governance policies to accounts or groups. In this post we will use AWS Organizations to show how you can easily leverage AWS Config and AWS Systems Manager in a multi-account environment.

Prerequisites

For this walkthrough, you must have AWS Config enabled in all the accounts in your AWS Organization. A quick way to enable AWS Config recording is to use Quick Setup, a capability of AWS Systems Manager that allows you to quickly create a configuration recorder.

In this post, we require that the tag key company-name and tag value CompanyXYZ are added to all EC2 instances in the selected accounts across the AWS Organization. Prior to proceeding with the walkthrough, please have the following information ready:

  1. AWS Organizations Management Account ID.
  2. (Optional) A list of member Account IDs that you want to exclude from deployment of this solution.
  3. An Amazon Simple Storage Service (Amazon S3) bucket in the AWS Organizations Management account which will be used to hold the conformance pack template.
  4. We will be using AWS CloudFormation and stacks. Please see the documentation here or this video introduction.

Deploy the IAM role through CloudFormation

The following is a CloudFormation template that we use to create the IAM role required to invoke the remediation action that we create later in the “Create the Automation runbook” section. You will first deploy this as a StackSet from the management account to the member accounts in your organization. You will then use a Stack to deploy into the management account itself because StackSets do not deploy to management accounts.

First, deploy the CloudFormation StackSet. Open the following GitHub page and download the cf-iam.yaml file.

https://github.com/aws-samples/aws-config-tagging-remediation-example/blob/main/cfn-templates/cf-iam.yaml

  1. Optionally, you can change the IAM role name and or the policy name in the CloudFormation template. For the purpose of the blog the IAM role name will be TagRemediationAutomationRole and the policy name will be TagRemediationAutomationPolicy.
  2. In your management account, go to the AWS CloudFormation console, and choose StackSets from the menu on the left.
  3. Click on the Create StackSet button.
  4. On the Choose a template page, leave the defaults for sections Permissions and Prerequisite – Prepare template. For the Specify template section, select Upload a template file and then browse to the cf-iam.yaml you downloaded earlier. Click Next.
  5. On the Specify StackSet details page, give the StackSet a name (for the purpose of this blog call it StackSetIAMRoleForRemediation). Leave the rest of the screen with the defaults. Click Next and then Next again.
  6. On the Set deployment options page, for Specify regions, select a region where you want to deploy this StackSet to. Select us-east-1 for the purpose of this blog. Leave everything else with the defaults and click Next.
  7. On the Review page, again leave all the defaults, scroll all the way down to the bottom of the page and check the box next to I acknowledge that AWS CloudFormation might create IAM resources with custom names, then choose Submit to deploy.

Next, deploy the CloudFormation Stack.

  1. In the Management Account, go to the AWS CloudFormation console. Choose Stacks from the menu on the left.
  2. Click on the Create Stack button, then choose With new resources (standard) from the dropdown.
  3. On the Create stack page, leave the defaults for Prerequisite – Prepare template. For the Specify template section, select Upload a template file and then browse to the cf-iam.yaml you downloaded earlier. Click Next.
  4. On the Specify Stack details page, give the Stack a name (for the purpose of this blog call it StackIAMRoleForRemediation) and leave the rest of the screen with the defaults. Click Next and then Next again.
  5. On the Review page, again leave all the defaults, scroll all the way down to the bottom of the page and check the box next to I acknowledge that AWS CloudFormation might create IAM resources with custom names, then choose Submit to deploy.

With this deployed in your environment, you now have the required role which can be assumed when the Automation runbook is triggered by the remediation action defined in AWS Config.

Create the Automation runbook for the remediation action and share the runbook with your member accounts

Remediation of non-compliant resources evaluated by the AWS Config rule is performed by utilizing Automation runbooks. To maintain centrally managed runbooks throughout your organization, we create this runbook using a CloudFormation stack in the management account and share the runbook with the member accounts through the AWS management console.

Open the following GitHub page and download the cf-automation-runbook.yaml file.

https://github.com/aws-samples/aws-config-tagging-remediation-example/blob/main/cfn-templates/cf-automation-runbook.yaml

  1. In the management account, go to the AWS CloudFormation console. Choose Stacks from the menu on the left.
  2. Click on the Create Stack button, then choose With new resources (standard) from the dropdown.
  3. On the Create stack page, leave the defaults for Prerequisite – Prepare template. For the Specify template section, select Upload a template file and then browse to the cf-automation-runbook.yaml you downloaded earlier. Click Next.
  4. On the Specify Stack details page, give the Stack a name (for the purpose of this blog call it StackRemediationRunbook). Leave the rest of the screen with the defaults. Click Next and then Next again.
  5. On the Review page, again leave all the defaults, scroll all the way down to the bottom of the page and choose Submit to deploy.

This template created an Automation runbook named SsmDocumentTagRemediation which uses the ResourceID passed by the AWS Config rules to query the ARNs of the non-compliant resources and then uses the AWS Resource Groups Tagging API to tag those resources.

Now that we have the Automation runbook created, we will share it with the member account(s) in the AWS Organization. To do this:

  1. In the management account, open the AWS Systems Manager console, then choose Documents in the left-navigation pane.
  2. Choose the Owned by me tab and select the SsmDocumentTagRemediation runbook created in the previous step.
  3. Select the Details tab. In the permissions section, add any account that you would like to share this runbook with and choose Save to save the changes. (Please note: there is a quota that allows a single Systems Manager document to be shared with a maximum of 1000 AWS accounts).
Share the Automation runbook with other AWS accounts

Figure 2. Share the Automation runbook with other AWS accounts.

Please note that while we are using the console to share the runbook, you can also use the ModifyDocumentPermission API or AWS CLI to programmatically add AWS account IDs in bulk.

Deploy the conformance pack in the management account

A conformance pack is a collection of AWS Config rules and remediation actions that you can easily deploy as a single entity in an account and a region or across an organization in AWS Organizations. In our sample you will deploy the conformance pack through AWS Organizations.

You can deploy an Organization Conformance Pack from the Organization management account or from a delegated administrator account for security consideration. For the purpose of this post, you will deploy it from the management account.

To deploy the conformance pack to AWS Organizations, you can use either the AWS Command Line Interface (CLI) command put-organization-conformance-pack or AWS CloudFormation. In this post, we will first create a conformance pack template and upload it to an S3 bucket and then deploy the organization conformance pack using the AWS CloudFormation resource AWS::Config::OrganizationConformancePack.

To create the conformance pack template, open the following GitHub page and download the cf-conformancepack-doc-map.yaml file.

https://github.com/aws-samples/aws-config-tagging-remediation-example/blob/main/cfn-templates/cf-conformancepack-doc-map.yaml

  1. Prior to deploying the conformance pack, there are a few modifications you may need to make to the conformance pack template to fit your needs. In this example, we specify that our resource requires the following tags and the conformance pack name:
  • Tag1key: company-name
  • Tag1value: CompanyXYZ
  • Conformance Pack Name: TagRemediationConformancePack
  1. Define the conformance pack’s name as a parameter on lines 19-22:
Parameters:
ConformancePackName:
Default: 'TagRemediationConformancePack'
Type: String
  1. The required tag/value pair is defined on lines 23-27 as a map called RequiredTags:
Mappings:
  RequiredTags:
    tag1:
      key: company-name
      value: CompanyXYZ
  1. The required tag/value we defined in the previous step will be used to provide the value of InputParameters of the ResourceTaggingCheck resource on lines 39-41:
InputParameters:
  tag1Key: !FindInMap [RequiredTags, tag1, key]
  tag1Value: !FindInMap [RequiredTags, tag1, value]
  1. The RequiredTags map is also used to provide the value for the Parameters section of the MapTaggingRemediation resource that will be passed to the remediation Automation runbook on lines 51-58:
Parameters:
  RequiredTags:
    StaticValue:
      Values:
        - !Sub
          - '{"${tag1key}":"${tag1value}"'
          - tag1key: !FindInMap [RequiredTags, tag1, key]
            tag1value: !FindInMap [RequiredTags, tag1, value]
  1. Within the Scope property of ResourceTaggingCheck, for ComplianceResourceTypes, we define which services are to be evaluated by this rule. In this post, we specify only EC2 Instances as AWS::EC2::Instance. For a list of supported AWS services, see the supported resources specified in the AWS Config User Guide topic required-tags. Please note, in order to add tags to a resource, you need the necessary permissions for the service that the resource belongs to as well as permissions for adding tags. The IAM role we created allows Automation to add tags to EC2 instances. If you need to tag resources in other AWS services, you must add the permission required by the respective service to the IAM role which was deployed during Deploy the IAM role through a CloudFormation StackSets and Stack. For more information, see the User Guide documentation for each service.
  2. Within the Conformance Pack template, replace <management-account-id> in the template with your management account ID on lines 49 and 70 and change the region if you are deploying the conformance pack in a region other than us-east-1 on line 49.
  3. Once all the modifications are complete, upload the Conformance Pack template to an S3 bucket in the management account. After uploading, make a note of the S3 URI. A sample S3 URI looks like the s3://<bucket-name>/cf-conformancepack-doc-map.yaml.

You can specify up to 6 tag/value pairs for the required-tags managed rule. You can download a complete Conformance Pack template sample with multiple required tags and sample accountID 123456789012 that deploys to us-east-1.

To deploy the Conformance Pack as an Organization conformance pack through CloudFormation, open the following GitHub page and download the cf-org-conformance-pack.yaml file.

https://github.com/aws-samples/aws-config-tagging-remediation-example/blob/main/cfn-templates/cf-org-conformance-pack.yaml

  1. (Optional) You can modify the ExcludedAccounts parameter on line 24 of the cf-org-conformance-pack.yaml CloudFormation template to specify a comma delimited list of accounts that you do not want to deploy this ConformancePack to.
  2. Modify the TemplateS3Uri on line 31 with the S3 URI of the conformance pack template you uploaded to S3 in the previous step. Please note, if you changed the ConformancePackName on line 21 of the conformancepack-doc-map.yaml, you need to change the value for OrganizationConformancePackName on line 30 of this template to match it.
  3. Use AWS CloudFormation to deploy this CloudFormation template as a Stack from your management account:
    1. Go to the AWS CloudFormation console. Choose Stacks from the menu on the left.
    2. Click on the Create Stack button, then choose With new resources (standard) from the dropdown.
    3. On the Create stack page, leave the defaults for Prerequisite – Prepare template. For the Specify template section, select Upload a template file and then browse to the cf-org-conformance-pack.yaml you downloaded earlier. Click Next.
    4. On the Specify Stack details page, give the Stack a name (for the purpose of this blog call it StackConformancePack). Leave the rest of the screen with the defaults. Click Next and then Next again.
    5. On the Review page, again leave all the defaults, scroll all the way down to the bottom of the page and choose Submit to deploy.

Once the CloudFormation deployment is complete, the conformance pack is deployed to all accounts in your organization other than the accounts you excluded.

Validate EC2 resource compliance for the conformance pack

After completing the steps above, you now have an AWS Config organization conformance pack deployed with the desired required tags specified, as well as a remediation action that will automatically tag the noncompliant resources.

Validate from the management account

  1. Open the AWS Config console and navigate to the Conformance Packs page. On the Conformance Packs page, you can then see the conformance pack previously deployed and the compliance score.
Conformance pack compliance score for tag remediation.

Figure 3. Conformance pack compliance score for tag remediation.

  1. Click on the conformance pack’s name to see more details and you can then see the rule deployed with the conformance pack and the rules compliance status.

Figure 4.

  1. Click on the rule name to validate the remediation action and see the resources in scope. You can also see the remediation execution status if they were triggered.
  2. Go to AWS System Manager to see the Automation executions history if you need to see the execution details or to troubleshoot.

(Optionally) Validate from a member account

Log in to one of the member accounts where the conformance pack was deployed and follow the above steps to verify resource compliance status.

Clean Up

To clean up the resources deployed in this post, perform the following steps in the management account:

  1. Delete the conformance pack CloudFormation stack.
  2. Un-share the Automation runbook with member accounts.
  3. Delete the Automation runbook CloudFormation stack.
  4. Delete the Automation IAM role CloudFormation stack and stackset.

Conclusion

Resources deployed in the Cloud have gotten much more complex in recent years and companies have had difficulty managing them all. In AWS, tags can provide metadata about these resources. The tags can be used to identify, manage and maintain resources, helping organizations with such things as operations and security.

In this post, we showed you how to use an AWS Config Conformance pack to automatically tag noncompliant resources which an AWS Config managed rule has evaluated. Specifically, we started by creating a customized Automation runbook for the remediation action. We then shared the runbook with member accounts in our AWS Organization. We followed that by creating an AWS Config conformance pack that combined the required-tags AWS Config managed rule with the Automation runbook and demonstrated how to deploy the organization conformance pack in a multi-account environment to selected member accounts in our AWS Organization.

To learn more about the tagging best practices and build your tagging strategy for your business needs, please reference the AWS Whitepaper Tagging Best Practices.

About the authors:

Cheng-Lun Chen

Cheng-Lun is a Solutions Architect at AWS. He is currently working with SMB Greenfield customers to help them achieve their business goals. His current focus is cloud operations, enterprise management, and control.

Erik Weber

Erik Weber is a Sr. World-wide Specialist Solutions Architect for AWS Cloud Operations services. He specializes in AWS Systems Manager, AWS Config, AWS CloudTrail, and AWS Audit Manager. Outside of work, Erik has a passion for hiking, cooking, and biking.

Oren Waldman

Oren Waldman is a Sr. Solutions Architect who supports SMB customers at AWS. In his free time, he likes to spend time with family and friends, watch TV, go on road trips and hiking.