AWS Cloud Operations & Migrations Blog

Introducing AWS Config Conformance Packs

I am very excited to introduce the latest addition to the AWS Config service: conformance packs. Conformance packs help you manage configuration compliance of your AWS resources at scale – from policy definition to auditing and aggregated reporting, using a common framework and packaging model.

What are conformance packs?

Conformance packs enable you to create packages of compliance rules, simplifying deployment at scale by packaging up both rules and remediation actions into a single entity. You can then deploy these across your estate, as well as share them with your customers and the public.

Additionally, conformance packs simplify compliance reporting, as it is now reported at a new level – the pack level – which allows you to drill down to the individual rule and resource level. All your existing reporting capabilities continue to work, and you gain a logical grouping in which specific sets of compliance are either managed by different teams or are part of a different compliance or governance regime.

Conformance packs are described using a markup similar to that of AWS CloudFormation; you only declare your rules as part of the Resources section and AWS Config does the rest. Optionally, as with AWS CloudFormation, you may also pass in inputs via the Parameters section to build more flexible conformance packs. Packs are described in YAML-formatted files.

Important note: A key feature of conformance packs is that they are immutable: individual rules cannot be changed outside of the pack in which they were deployed, regardless of access or account permissions. In addition, if the pack is deployed by an organization’s master account, it cannot be modified by the organization’s member accounts. This provides you with an additional level of security and certainty when managing compliance across your estate.

Where do conformance packs fit into my compliance regime?

Managing configuration compliance for any IT service is typically required by internal teams (Central IT or InfoSec) and external auditors (PCI, HIPAA, SOC2, etc.) to ensure security and confidentiality of customer data. It’s a multi-step process that involves reference to standards and regulatory requirements, individual policy definitions, remediation workflows, and exception procedures. Previously, this was handled by multiple tools. Each tool specialized in one area (such as HIPAA or PCI) and had its own method of defining policies and reporting compliance results. This increased management overhead, and meant you had to learn to operate different tools for different compliance regimes and interpret results from each tool individually.

With this launch, configuration compliance management is greatly simplified for cloud resources compared with previously available approaches. You can now use AWS Config conformance packs to create a conformance pack that consists of a set of AWS Config rules. Moreover, you are now able to not only monitor compliance based on your own groupings but also apply remediation automatically. Conformance packs support remediation using AWS Systems Manager Automation, a feature AWS released in September.

This post describes how to set up and use conformance packs both from the console and the command-line interface (AWS CLI).

Prerequisites

In order to use conformance packs, ensure you are meeting the following requirements:

  • AWS Config recording is turned on.
  • A service role for AWS Config conformance packs exists (AWSServiceRoleForConfigConforms).
  • An Amazon S3 bucket is available for storing and delivering the pack compliance results with access granted to the service role.
  • If you choose to add remediation actions, a service role for AWS Config remediation is also required.

Ensure AWS Config recording is on

If this is the first time you are accessing AWS Config, follow the steps in Setting Up AWS Config with the Console.

Otherwise:

  1. Sign in to AWS and open the AWS Config console.
  2. Choose Settings and ensure that Recording is on. As with other aspects of the service, you can do the same using the AWS CLI or programmatically.

Create a service role for the AWS Config conformance pack

The AWS Config conformance packs service requires an IAM role to allow it to work in your account.

To create an IAM service role via the console, follow this link or follow these steps:

  1. Visit the IAM service page.
  2. Click Create Role.
  3. Select Config as the AWS service for which to create a role and Config – Conformance Packs for the use case.

To create an IAM service role that will be used by the service, simply run the following CLI command:

aws iam create-service-linked-role --aws-service-name config-conforms.amazonaws.com --description "my service linked role for config-conforms"

Update the permissions on your Amazon S3 delivery bucket

For AWS Config to be able to store conformance pack artifacts, you will need to provide an Amazon S3 bucket and add the following permissions to it:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AWSConfigConformsBucketPermissionsCheck",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
            "arn:aws:iam::AccountId:role/aws-service-role/config-conforms.amazonaws.com/AWSServiceRoleForConfigConforms"
        ]
      },
      "Action": "s3:GetBucketAcl",
      "Resource": "arn:aws:s3:::delivery-bucket-name"
    },
    {
      "Sid": "AWSConfigConformsBucketDelivery",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
            "arn:aws:iam::AccountId:role/aws-service-role/config-conforms.amazonaws.com/AWSServiceRoleForConfigConforms"
        ]
      },
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::delivery-bucket-name/[optional] prefix/AWSLogs/AccountId/Config/*",
      "Condition": {
        "StringEquals": {
          "s3:x-amz-acl": "bucket-owner-full-control"
        }
      }
    },
    {
      "Sid": " AWSConfigConformsBucketReadAccess",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
            "arn:aws:iam::AccountId:role/aws-service-role/config-conforms.amazonaws.com/AWSServiceRoleForConfigConforms"
        ]
      },
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::delivery-bucket-name/[optional] prefix/AWSLogs/AccountId/Config/*"
    }
  ]
}

More detailed information is available on the prerequisites page of the documentation.

Deploying your first conformance pack

AWS Config allows you both to use managed rules and to write your own custom rules, backed by AWS Lambda functions. Today, AWS has more than 100 managed rules and more than 20 custom rules curated from both internal teams and customers on its public repository.

For your first conformance pack, use the DynamoDB Best Practices pack, which includes three AWS Config rules that are checked against all of your Amazon DynamoDB tables in the region. Later in this example, you add auto-remediation to help meet and monitor compliance.

The sample templates can be used directly from the console. In order to use them via the AWS CLI or programmatically, simply save them locally and either point to the saved template, or upload them to an Amazon S3 bucket and refer to it.

To deploy your first pack using the console:

  1. Navigate to the AWS Config console, choose Conformance packs and choose the Deploy conformance pack button.
  2. Choose the Use sample template option and select Operational Best Practices for Amazon DynamoDB from the list of sample templates available.
  3. On the Specify conformance pack details page, give the pack a name, add in the Amazon S3 bucket name to be used for storing the reports, and optionally, add a prefix. For your sample DynamoDB pack, no additional parameters are required. (When working with packs that require input parameters, you have to check and pass the appropriate ones, because the console does not auto-populate them.
  4. Finally, review the details and, if everything looks right, choose Deploy conformance pack. The pack will move to an In progress deployment status and, within a few minutes, to Completed status.

You can the click the pack and see a detailed breakdown of the rules comprising it and their compliance statuses, as shown in the following screenshot:

MyFirstConformancePack

To deploy your first pack using the CLI:

  1. Save the DynamoDB Best Practices pack locally and run the following command: 
    aws configservice put-conformance-pack --conformance-pack-name="MyFirstConformancePack" --template-body="file://<PATH TO YOUR TEMPLATE>/<TEMPLATE FILENAME>" --delivery-s3-bucket=<YOUR BUCKET>

    Make sure to replace the placeholders with their corresponding values. You can either specify a local source for the template, or point the service to an Amazon S3 bucket using the --template-s3-uri argument.

  2. The response you receive from the service is the conformance pack’s ARN, such as the following:arn:aws:config:us-west-2:123456789012:conformance-pack/ MyFirstConformancePack/conformance-pack-ababababa

To check the compliance of your newly created conformance pack, use the following command:

aws configservice describe-conformance-pack-compliance --conformance-pack-name="MyFirstConformancePack"

You should see an output similar to the following:

{
    "ConformancePackName": "MyFirstConformancePack", 
    "ConformancePackRuleComplianceList": [
        {
            "ComplianceType": "NON_COMPLIANT", 
            "ConfigRuleName": "DynamoDbAutoscalingEnabled-conformance-pack-l1thqp3tu"
        }, 
        {
            "ComplianceType": "COMPLIANT", 
            "ConfigRuleName": "DynamoDbTableEncryptionEnabled-conformance-pack-l1thqp3tu"
        }, 
        {
            "ComplianceType": "COMPLIANT", 
            "ConfigRuleName": "DynamoDbThroughputLimitCheck-conformance-pack-l1thqp3tu"
        }
    ]
}

Add more custom rules

You might want to add in your own custom rules, or perhaps some of those from the AWS customer-driven GitHub repository.

In order to do this, first write an AWS CloudFormation template that includes the required components: Lambda functions, IAM roles for the functions, and permissions for AWS Config to call these Lambda functions.

Finally, export each Lambda function’s ARN. Then, in your conformance rack, reference these functions using Fn::ImportValue.

For your next pack, use a custom rule and another managed rule. I have chosen the IAM Policy in Use check for the managed rule, combined with the IAM Policy Exists custom rule. As mentioned previously, set up everything but the rules in one AWS CloudFormation template and package up the two rules in a conformance pack.

The first template

This contains your Lambda function along with its IAM Role and permissions, exports the created function’s ARN, and is deployed using AWS CloudFormation:

---
Resources:
  ConfigPermissionToCallLambda:
    DependsOn: [IAMPolicyExistsLambda]
    Type: AWS::Lambda::Permission
    Properties:
      FunctionName: !GetAtt IAMPolicyExistsLambda.Arn
      Action: "lambda:InvokeFunction"
      Principal: "config.amazonaws.com"
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      AssumeRolePolicyDocument:
        Statement:
          - Action: ['sts:AssumeRole']
            Effect: Allow
            Principal:
              Service: [lambda.amazonaws.com]
        Version: '2012-10-17'
      Policies:
        - PolicyName: "PutEvaluationsPermission"
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: "Allow"
                Action:
                  - "config:PutEvaluations"
                Resource: '*'
              - Effect: "Allow"
                Action:
                  - "iam:GetPolicy"
                Resource: '*'
  IAMPolicyExistsLambda:
    DependsOn: [LambdaExecutionRole]
    Type: "AWS::Lambda::Function"
    Properties:
      Handler: "index.lambda_handler"
      Role: !GetAtt LambdaExecutionRole.Arn
      Code:
        ZipFile: !Sub |
          #
          # This file made available under CC0 1.0 Universal (https://creativecommons.org/publicdomain/zero/1.0/legalcode)
          #
          # Ensure one or several specific IAM policies exist
          # Description: Checks that defined IAM policies have been defined in AWS IAM.
          #
          # Trigger Type: Periodic
          # Scope of Changes: N/A
          # Required Parameter name: PoliciesToCheck
          # Required Parameter value example: policy-name1,policy-name2 (split multiple rule name with a ",")

          import boto3
          import json

          def evaluate_compliance(rule_parameters, account_id):
            fails = 0
            client = boto3.client("iam")
            
            if 'PoliciesToCheck' in rule_parameters:
              for policy in rule_parameters["PoliciesToCheck"].split(","):
                policyARN = "arn:aws:iam::%s:policy/%s" %(account_id, policy)
                print(policyARN)
                try:
                  response = client.get_policy(PolicyArn=policyARN)
                except:
                  fails = fails + 1
            else:
              print("No IAM policy defined in parameter")
              fails = fails + 1
            if fails == 0:
              return "COMPLIANT"
            else:
              return "NON_COMPLIANT"


          def lambda_handler(event, context):
              account_id = event['accountId']
              invoking_event = json.loads(event["invokingEvent"])
              print(invoking_event)
              rule_parameters = json.loads(event["ruleParameters"])
              result_token = "No token found."
              if "resultToken" in event:
                  result_token = event["resultToken"]

              config = boto3.client("config")
              config.put_evaluations(
                  Evaluations=[
                      {
                          'ComplianceResourceType': 'AWS::::Account',
                          'ComplianceResourceId': account_id,
                          'ComplianceType': evaluate_compliance(rule_parameters, account_id),
                          'OrderingTimestamp': invoking_event['notificationCreationTime']
                      },
                  ],
                  ResultToken=event['resultToken']
              )
      Runtime: "python3.7"
      Timeout: 120

Outputs:
  IAMPolicyExistsLambdaArn:
    Description: Export Lambda ARN for use in conformance pack template
    Export:
      Name: IAMPolicyExistsLambdaArn
    Value:
      !GetAtt IAMPolicyExistsLambda.Arn

The second template

This contains the conformance pack, imports the Lambda function, adds another managed rule, and is deployed using AWS Config:

---
Resources:
  IamPolicyInUse:
    Type: "AWS::Config::ConfigRule"
    Properties:
      ConfigRuleName: IamPolicyInUse
      Description: "Checks whether the IAM policy ARN is attached to an IAM user, or an IAM group with one or more IAM users, or an IAM role with one or more trusted entity."
      MaximumExecutionFrequency: TwentyFour_Hours
      InputParameters:
        policyARN: "arn:aws:iam::aws:policy/IAMFullAccess"
      Source:
        Owner: AWS
        SourceIdentifier: IAM_POLICY_IN_USE
  
  IamPolicyExists:
    Type: "AWS::Config::ConfigRule"
    Properties:
      ConfigRuleName: IamPolicyExists
      Description: "Checks that defined IAM policies have been defined in AWS IAM."
      MaximumExecutionFrequency: TwentyFour_Hours
      InputParameters:
        PoliciesToCheck: "arn:aws:iam::aws:policy/IAMFullAccess"
      Source:
        Owner: "CUSTOM_LAMBDA"
        SourceIdentifier:
          Fn::ImportValue: IAMPolicyExistsLambdaArn
        SourceDetails: 
        - EventSource: "aws.config"
          MessageType: "ScheduledNotification"

Just as with the first example, you can view, update, and delete a pack using either the console, API, or AWS CLI.

Adding remediation to the packs

The first step in compliance is visibility and monitoring – making sure you know the compliance state of your resources. The next step is actioning based on the results you detect. For this, use remediation actions.

Remediation actions are defined using the AWS::Config::RemediationConfiguration resource type which links an AWS Config rule with a target. The target is an SSM document with logic to remediate the non-compliant resource. To enhance your first pack with remediation actions, use the Operational Best Practices For Amazon DynamoDB with Remediation. Save the template locally, replace the <Account ID> placeholder with the AWS account ID to which you want to deploy and repeat the deployment steps you used for the first pack. This specific pack requires an Amazon SNS Topic ARN as an input parameter. This topic is used for notifications about the auto-remediation status.

You can control whether or not remediation is automatically applied or not. This is done using the Automatic parameter. You may want a notification sent but a semi-manual step to control remediation, or to enable it so that non-compliant resources get remediated.

Once deployed, the pack includes remediation actions and the compliance status of the rules included in it, as shown in the following screenshot.

DynamoPackWithRemediation

To recap, there are two components: SSM documents (created once) and conformance packs. This approach allows you to reference the same SSM documents from a variety of different packs or other AWS resources, increasing reusability.

Deploying a cross-organization conformance pack

Many customers have more than one account and leverage AWS Organizations. Conformance packs can be deployed to a single account, as described previously, or across your organization.

This functionality is currently supported using the AWS CLI and API only and is very similar to the single account deployment:

aws configservice put-organization-conformance-pack --organization-conformance-pack-name="OrgDynamoConformancePack" --template-body="file://<PATH TO YOUR TEMPLATE>/<TEMPLATE FILENAME>" --delivery-s3-bucket=<YOUR BUCKET>

Make sure to replace the placeholders with their corresponding values.

Note: When deploying conformance packs to an organization, the name of the delivery Amazon S3 bucket must start with an awsconfigconforms prefix, such as awsconfigconforms-demo.

As with the single account packs, you can:

  • Describe conformance packs (describe-organization-conformance-packs)
  • Get their statuses (describe-organization-conformance-pack-statuses)
  • See a detailed status for them (get-organization-conformance-pack-detailed-status)

For more information on AWS Organizations support, please visit the AWS Config documentation.

Summary

This blog post presented how you can get started using AWS Config conformance packs to package up multiple rules, deploy and manage their compliance in your accounts, add in remediation, and deploy these across an entire organization.

If you have written your own custom rules, I strongly encourage you to share them with AWS and your fellow users via the AWS GitHub repository. Keep your eyes on the What’s New section as AWS continues to collect feedback and add features and capabilities.

 

About the Authors

Raphael is a technical business development manager for Service Catalog & Control Tower. He enjoys tinkering with automation and code and active member of the management tools community.

 

 

 

Himanshu is a software development manager in AWS Config. He is interested in all things related to configuration management, governance and compliance management in AWS and hybrid cloud environments.