How to develop custom AWS Config rules using the Rule Development Kit
To help customers rapidly prototype, develop, and deploy their custom AWS Config rules at scale, AWS introduces a new version of the AWS Config Rule Development Kit (RDK).
The RDK is a command-line utility designed to help you to shorten your security and compliance feedback cycles when using Config. It helps you build a continuous compliance framework that your auditors will love. It can be included in an end-to-end DevSecOps pipeline or used interactively from your command line to create and atomically deploy the Config, AWS Lambda, and IAM resources necessary for your custom Config use case.
In this post, I show you how to get started with the RDK.
Config is a native service for continuous compliance. It allows you to track the changes to resources and configurations (configuration items) in your account. You can define rules that detect when configuration items have drifted into undesirable territory.
Config has a number of out-of-the-box rules that cover many the common use cases for customers. You can also create custom Config Rules backed by Lambda functions. Many enterprise customers or customers operating in highly regulated industries have special requirements that need specific definitions of compliant. I also see customers using the robust, serverless Config framework to build other housekeeping functionality such as cost and security controls.
The first steps are to make sure that you have the necessary permissions in your AWS account, and then you can install the tool itself.
To get started, you need an AWS account, and necessary permissions for modifying Config rules, Lambda functions, and various IAM resources within that account. The rdk-minimum-permissions.json sample policy document illustrates the required permissions.
RDK allows you to author Lambda functions in almost all of the supported languages. It is itself written in Python, and is compatible with both Python 2.7 and 3.6. The recommended installation method is through pip (
pip install rdk). To see the source code (or even contribute!), clone the GitHub aws-config-rdk repo.
You can make sure that it is installed correctly by typing rdk at a command prompt. You should see usage instructions and an error about required arguments.
(rdk-demo) 186590d1e043:rdk-demo USER $ rdk usage: rdk [-h] [-p PROFILE] [-k ACCESS_KEY_ID] [-s SECRET_ACCESS_KEY] [-r REGION] <command> ... rdk: error: the following arguments are required: <command>, <command arguments>
Config rules backed by Python and Node.js Lambda functions can be deployed without any additional setup. To use either the Java 8 or .NET Core runtimes, you need some additional components:
- Java: Use gradle to build your deployment package, so install it somewhere accessible from your CLI.
- .NET: Microsoft ships a CLI for .NET called “dotnet” that you need installed, as well as the latest release of the .NET Core framework v1.0. These are necessary to build and create Lambda deployment packages for your .NET code.
Now that you have the RDK installed, walk through the steps to create, deploy, and manage your custom Config rules.
RDK uses the boto3 libraries for AWS API access, so any of the standard methods of passing credentials work. For more information about ways to supply your credentials, see options 3 through 8 in Credentials in the Boto documentation. Alternately, you can pass them in with the following command-line parameters:
rdk init to create a working directory. The working directory holds your rule definitions, as well as a snapshot of template files used to build your Config rules. Ideally, your working directory is a local code repo and python virtualenv, but it’s not necessary for just playing around. The
init command also configures AWS Config in the account specified by your credentials. Config begins tracking information about your resources and their configurations.
(rdk-demo) 1234567890ab:rdk-demo USER$ rdk init Running init! Found Config Recorder: default Found Config Role: arn:aws:iam::123456789012:role/config-role Found Bucket: config-bucket-123456789012 Config Service is ON Config setup complete. Found code bucket: config-rule-code-bucket-123456789012us-west-2
Create or modify
Now it’s time to create some rules!
rdk create <rulename> --runtime <runtime> to create a local rule folder that contains your initial rule code, as well as some helper code and the parameters used to deploy the rule later. You need to specify a valid Lambda runtime. Optional flags include resource types for triggering on configuration change events, the frequency for scheduled evaluations, and any parameters that Config passes on to the backing Lambda function.
To change rule parameters later on, you can use
rdk modify <rulename> --<flag> <option> to overwrite any of the options that were previously set for your local rule definition.
(rdk-demo) 1234567890ab:rdk-demo USER$ rdk create MyTestRule --runtime python3.6 --resource-types AWS::EC2::Instance Running create! Defaulting to TwentyFour_Hours Maximum Frequency. Local Rule files created.
Navigate to the folder created by create and you see a number of files, depending on the runtime that you selected.
Config rules backed by Python or Node.js Lambda functions have a .py or .js file named the same as your rule name, which is the code template to which to add your custom logic. There are other files associated with each runtime that provide necessary functions to validate inputs, submit compliance results back to Config, and handle errors and logging. You probably don’t need to worry about these.
Go ahead and add whatever logic you want based on the CI, rule parameters, or any other data that your function should pull out of your AWS environment. Just make you return a valid compliance result. If you are writing your Lambda function in Java, make sure that you update the build.gradle file with any dependencies that you add.
Now you’re ready to deploy your custom rule to AWS!
From your working directory, use
rdk deploy <rulename> to push your changes out to AWS. This uses AWS CloudFormation behind the scenes to build your Config rule, Lambda function. It also creates the necessary policies and permissions to let them talk to one another. You can list multiple rules to deploy, or use the
–all flag to push all of the rules in your working directory. If you are using either Java or .NET, this also compiles your code and packages it appropriately for Lambda deployment.
(rdk-demo) 1234567890ab:rdk-demo USER$ rdk deploy MyTestRule Running deploy! Zipping MyTestRule Uploading MyTestRule Updating CloudFormation Stack for MyTestRule Waiting for CloudFormation stack operation to complete... Waiting for CloudFormation stack operation to complete... Waiting for CloudFormation stack operation to complete... Publishing Lambda code... Lambda code updated. Config deploy complete.
After your rule has successfully deployed, you’ll want to know what it’s actually doing!
It can be useful to instrument your Lambda function with some logging so that you can have some debugging output about the decisions that your function is making. RDK provides a quick way to get a view into your function with the
logs command. It has command-line arguments for how many log events to look back, and also supports the
–f flag for continuously polling the log group for the specified function and returning results as they happen.
The following is an example of the
(rdk-demo) 1234567890ab:rdk-demo USER$ rdk logs MyTestRule/ -f Removing trailing '/' 2018-02-13 10:55:03 - Tenancy is 'default' - returning COMPLIANT 2018-02-13 10:55:04 - END RequestId: 482e6306-1069-11e8-8696-8f2e469c4c56 2018-02-13 10:55:04 - REPORT RequestId: 482e6306-1069-11e8-8696-8f2e469c4c56 Dura tion: 325.81 ms Billed Duration: 400 ms Memory Size: 25 6 MB Max Memory Used: 30 MB 2018-02-13 10:55:04 - END RequestId: 480270e8-1069-11e8-b8de-65b91f7b7901 2018-02-13 10:55:04 - REPORT RequestId: 480270e8-1069-11e8-b8de-65b91f7b7901 Dura tion: 264.03 ms Billed Duration: 300 ms Memory Size: 25 6 MB Max Memory Used: 30 MB 2018-02-13 11:00:20 - START RequestId: 07029f1c-106a-11e8-921b-c7902649007e Version: $LATEST 2018-02-13 11:00:20 - Tenancy is 'default' - returning COMPLIANT 2018-02-13 11:00:20 - END RequestId: 07029f1c-106a-11e8-921b-c7902649007e 2018-02-13 11:00:20 - REPORT RequestId: 07029f1c-106a-11e8-921b-c7902649007e Dura tion: 272.61 ms Billed Duration: 300 ms Memory Size: 25 6 MB Max Memory Used: 31 MB
From the logs, make sure that your rule is functioning as expected. If you have a bug in your logic, iterate quickly by editing the rule code and re-deploying in seconds.
After you’ve deployed your rule and verified that it’s working correctly, you can see your compliance status in the AWS Config console.
I’m excited about the ways that the RDK can empower AWS customers to build world-class, enterprise-level compliance controls quickly and easily, leaving the heavy-lifting of the machinery and the framework to AWS Config.
Config is a powerful component in a cloud-based, end-to-end security and compliance framework. Its ability to track changes over time is a great complement for Amazon CloudWatch Events, which gives you near-real-time responsiveness to individual API calls.
To learn more about how these services can work together to provide a comprehensive “Compliance As Code” solution, check out the Building the Largest Repo for Serverless Compliance-as-Code (SID205) re:Invent 2017 session.
About the Author
Michael Borchert is a Singapore-based Sr. Cloud Architect with AWS Professional Services. Michael enjoys helping AWS customers automate security, compliance, and development to accelerate all aspects of their cloud journey. In his free time he likes to play music and explore Southeast Asia with his family.