AWS Cloud Operations Blog

Write preventive compliance rules for AWS CloudFormation templates the cfn-guard way

Continuous delivery pipelines, combined with infrastructure as code tools like AWS CloudFormation, allow our customers to manage applications in a safe and predictable way. CloudFormation helps customers model and provision their AWS and third-party application resources, with features such as rollback to provide automation and safety. Together with tools such as AWS CodeBuild, AWS CodePipeline, and AWS CodeDeploy, CloudFormation helps customers automate processes that implement popular practices like GitFlow , allowing for incremental testing and automated guardrails for fast releases. Emerging best practices, such as DevSecOps, shift-left testing and low-code tools, are further enabling security leaders to embed preventive compliance rules earlier in the development lifecycle.

These best practices inspired us to build CloudFormation Guard (cfn-guard) as an open-source tool that helps you write compliance rules using a simple, policy as code language. It will help you validate CloudFormation templates against those rules to keep your AWS resources in compliance with your company policy guidelines. Customers, engineers and AWS AWS Professional Services consultants have used cfn-guard as part of a private beta. Based on the positive feedback we received, we have made it publicly available on GitHub. You can use cfn-guard to both evaluate templates locally as you write them and after you submit them to be deployed in your CI/CD pipelines. In this article, we will show you how to use cfn-guard and offer tips to integrate it into your application management processes.

Although similar tools exist to create custom compliance rules, such as cfn-nag, cfripper, and checkov, cfn-guard uses a domain-specific language (DSL) to write rules. Learning the rule language is easier than learning a programming language like Python or Ruby, which is required to make custom rules in similar tools. Because cfn-guard is written in Rust, it can be compiled to a native binary to evaluate thousands of rules across templates.

Getting started

To prepare your environment, we will download Rust, then get and build cfn-guard, and finally verify our installation. You can install Rust in macOS, Linux, and Windows machines with rustup; we will mostly cover the instructions specific to macOS here. Refer to the GitHub repository for additional installation options.

We will use Homebrew as our macOS package manager, then rustup-init to install the compiler and the cargo package manager, along with the rest of the Rust default toolchain to create binaries. Use the following three commands in your terminal:

brew install rustup
rustup-init
rustc --version

Note: If you are using Windows, you can opt to use Chocolatey or standalone installers.

After the third command, you should get a response similar to the following:

rustc 1.44.0 (49cae5576 2020-06-01)

Next, we will get and build cfn-guard and verify the installation. Use the following four commands:

git clone https://github.com/aws-cloudformation/cloudformation-guard.git 
cd cloudformation-guard
make cfn-guard
bin/cfn-guard --version

After the fourth command, you should get a response similar to the following:

CloudFormation Guard 0.5.2

If you were able to verify both installations, you can proceed to the next section.

Your first cfn-guard rule set

Consider the following scenario: you want to allow your developers to create ephemeral environments so they can test their code using the latest Amazon Linux distribution. To control costs, you want to allow them to create Amazon EC2 instances using only small instance types. You want to restrict the size of the attached EBS volumes while ensuring consistency with other managed volumes. To comply with security requirements, you want developers to create only encrypted volumes and use a non-standard port for secure shell access to the instances.

You will provide developers with instructions on how to deploy these environments on-demand, including how to use cfn-guard rule files to author compliant templates. You can setup a controlled pipeline to enforce rules; if developers then attempt to submit a non-compliant template, the pipeline will throw an error and not deploy the resources.

Let’s inspect a sample template. It creates an EC2 instance, volume, and security group:

AWSTemplateFormatVersion: '2010-09-09'
Metadata:
  License: Apache-2.0
Description: 'AWS CloudFormation Sample Template for cfn-guard blog, for developers.
  It creates an Amazon EC2 instance running the latest Amazon Linux AMI, based on the region
  in which the stack is run, with restricted sizes. It also creates an EC2 security group
  for the instance to give you SSH access on a non-standard port. 
  **WARNING** This template creates an Amazon EC2 instance, and you will be billed for 
  it once the stack is deployed.  Delete the stack after you have completed your tests to
  avoid additional charges'
Parameters:
  KeyName:
    Description: Name of an existing EC2 KeyPair to enable SSH access to the instance
    Type: AWS::EC2::KeyPair::KeyName
    ConstraintDescription: must be the name of an existing EC2 KeyPair.
  SSHLocation:
    Description: The IP address range that can be used to SSH to the EC2 instances
    Type: String
    MinLength: 9
    MaxLength: 18
    Default: X.X.X.X/X
# must change this line to a valid private IP range
    AllowedPattern: (\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})
    ConstraintDescription: Must be a company approved, valid IP CIDR range of the form x.x.x.x/x.
  LatestAmiId:
    Type:  'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
    Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
Resources:
  EC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: t3.medium
      SecurityGroups: [!Ref 'InstanceSecurityGroup']
      KeyName: !Ref 'KeyName'
      ImageId: !Ref 'LatestAmiId'
  InstanceSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: Enable SSH access 
      SecurityGroupIngress:
      - IpProtocol: tcp
        FromPort: 22
        ToPort: 22
        CidrIp: !Ref 'SSHLocation'
  NewVolume:
    Type: AWS::EC2::Volume
    Properties:
      Size: 512
      AvailabilityZone: !GetAtt EC2Instance.AvailabilityZone
Outputs:
  InstanceId:
    Description: InstanceId of the newly created EC2 instance
    Value: !Ref 'EC2Instance'
  AZ:
    Description: Availability Zone of the newly created EC2 instance
    Value: !GetAtt [EC2Instance, AvailabilityZone]
  PublicDNS:
    Description: Public DNSName of the newly created EC2 instance
    Value: !GetAtt [EC2Instance, PublicDnsName]
  PublicIP:
    Description: Public IP address of the newly created EC2 instance
    Value: !GetAtt [EC2Instance, PublicIp]

Note that cfn-guard focuses on the Resources section of your template; however, you can still use other CloudFormation capabilities in your code, including Parameters and Outputs. For this example, you are allowing the developers to use their own (previously assigned) key pairs. Further, you are leveraging a publicly provided value via AWS Systems Manager Parameter Store that picks up the latest Amazon Linux image, regardless of the region where the template is executed.

As it stands, this template is syntactically correct: it complies with all built-in rules and warnings as evaluated by the current version of the CloudFormation linter (cfn-lint). However, similar to other tools mentioned before, cfn-lint requires you to write custom rules in Python. If you have invested in custom rules for cfn-lint or other tools, you can continue to use those rules together with cfn-guard.

This template fails all four desired compliance rules of our previously outlined scenario. We will repeat them here in a more formal format:

  • When developers create a new Linux Instance, the InstanceType must be nano, t2.micro, or t3.small.
  • When developers create a security group for their instance, an ingress rule must be created that uses the non-standard port assigned to their team, namely, port 33322.
  • When developers create a volume to use with their instance, the size must be 128GB or 256GB.
  • When developers create a volume, it must be encrypted.

Now, let’s see how the rules above translate to cfn-guard’s policy-as-code language:

AWS::EC2::Instance InstanceType == /.nano|.micro|.small/
AWS::EC2::SecurityGroup SecurityGroupIngress == [{"CidrIp":"SSHLocation","FromPort":33322,"IpProtocol":"tcp","ToPort":33322}]
AWS::EC2::Volume Size == 128 |OR| AWS::EC2::Volume Size == 256
AWS::EC2::Volume Encrypted != false << you must create encrypted volumes

Because cfn-guard uses a simple declarative syntax and follows CloudFormation conventions, it results in easy to understand yet flexible rules, following the pattern depicted in Fig. 1:

cfn-guard-rule-format

Fig. 1. CloudFormation Guard Rule Format

There are other options and additional functionality that can be used when writing rules. For example, the second rule (repeated below) explicitly checks for this literal value (33322) and syntax of the security group:

AWS::EC2::SecurityGroup SecurityGroupIngress == [{"CidrIp":"SSHLocation","FromPort":33322,"IpProtocol":"tcp","ToPort":33322}]

You can leverage wildcards to make your rule more explicit and flexible. For example, the following version of the second rule will check explicitly for having the valid port (in our context, port 33322) and explicitly for not having the default port (port 22 for SSH). This revised rule will also work if the security group has other rules, making the rule itself more reusable in other templates. To do so, it uses * as a wildcard in the Property Name:

AWS::EC2::SecurityGroup SecurityGroupIngress.* != {“CidrIp":"SSHLocation","FromPort":22,"IpProtocol":"tcp","ToPort":22}

AWS::EC2::SecurityGroup SecurityGroupIngress.* == {"CidrIp":"SSHLocation","FromPort":33322,"IpProtocol":"tcp","ToPort":33322}

Wildcards enable you to create rules for complex requirements. Say you wanted to inspect all the fields that are nested inside a particular resource’s property list. You can use dotted notation, which expands and iterates over all possible fields in the resource property. Check the documentation and examples here for more details on how to use wildcards.

Finally, look back at the third rule in the rule list above, repeated here:

AWS::EC2::Volume Size == 128 |OR| AWS::EC2::Volume Size == 256

This line has two rules separated by an |OR| operator. You can write these two rules in separate lines, or use lists. We will cover this case in more detail shortly, as we run the rule evaluation.

Running cfn-guard

Now, let’s get ready to run cfn-guard and fix our template so it complies with the rules:

  1. Copy and paste the template code above, and save into a plain text file called test.yaml. Before executing this template, you must change the default CIDR for the SSHLocation parameter to a valid private IP range. Before you continue, it is good to use cfn-lint to validate that the file’s syntax and property requirements are correct.
  2. Copy and paste the four cfn-guard rules we translated before into a text file called test.rules. Cfn-guard doesn’t require a specific file extension, so you can use your own to comply with your team’s naming conventions.
  3. Run the following command, provided that the cfn-guard binary is in your terminal environment’s path, and that both files you created are in the local directory:
cfn-guard check -t test.yaml -r test.rules

Your output should be similar to the following:

[EC2Instance] failed because [InstanceType] is [t3.medium] and the permitted pattern is [.nano|.micro|.small]
[InstanceSecurityGroup] failed because [SecurityGroupIngress] is [[{"CidrIp":"SSHLocation","FromPort":22,"IpProtocol":"tcp","ToPort":22}]] and the permitted value is [[{"CidrIp":"SSHLocation","FromPort":33322,"IpProtocol":"tcp","ToPort":33322}]]
[NewVolume] failed because [Size] is [512] and the permitted value is [128]
[NewVolume] failed because [Size] is [512] and the permitted value is [256]
Number of failures: 4

Did you expect that? We expected all rules to fail, including the encryption rule, but it didn’t. Although the encryption rule is correct in our file, it was not triggered because the AWS::EC2::Volume resource type does not require the Encrypted property, and it wasn’t specified in our template. The template is still valid, but is not compliant with all our rules. Cfn-guard will not evaluate specific property rules by default if they are omitted in the template and are not required by the resource. To change this behavior, you can use the strict flag -s to allow for rules that check resource properties that are otherwise not mandated in typical configurations. Let’s add this flag to the command to force this strict checking:

cfn-guard check -t test.yaml -r test.rules -s

Now your output should match the following, which is what we expected:

[EC2Instance] failed because [InstanceType] is [t3.medium] and the permitted pattern is [.nano|.micro|.small]
[InstanceSecurityGroup] failed because [SecurityGroupIngress] is [[{"CidrIp":"SSHLocation","FromPort":22,"IpProtocol":"tcp","ToPort":22}]] and the permitted value is [[{"CidrIp":"SSHLocation","FromPort":33322,"IpProtocol":"tcp","ToPort":33322}]]
[NewVolume] failed because [Size] is [512] and the permitted value is [128]
[NewVolume] failed because [Size] is [512] and the permitted value is [256]
[NewVolume] failed because you must create encrypted volumes
Number of failures: 5

Now all rules execute correctly, including our encryption rule with the custom failure message. Note also that we got five failures, while our rule file has four lines. In reality, cfn-guard is evaluating five rules, since the rule on the third line actually evaluates two conditions and, unlike typical coding languages, there’s no short-circuit evaluation of the |OR| expression here. The convention here is that we wrote both distinct Size rules on a single line but, if the Size value matches either of the permitted values, the rule evaluation will succeed.

There are other conventions in cfn-guard to write rules. For example, you can write the volume size rule in separate lines, or you can use the let option to add all valid values in one place in your code. Arguably, this convention improves the readability and conciseness of the rules file. It also minimizes future rule changes if other size options need to be added:

let validSizes = [64, 128, 256]
AWS::EC2::Volume Size IN validSizes

How about AND expressions? As it turns out, cfn-guard considers all lines together in a rule file to be an implicit AND operation. This is common across security tools, similar to how IAM conditions with multiple keys and values are evaluated in policies: the overall evaluation succeeds when all the rules evaluate successfully.

Let’s change the template to make all the rules compliant: try changing the InstanceType value to t3.nano, the From and To egress rule ports from 22 to 33322, the Size property of the volume to either 128 or 256, and add the Encrypted property to the volume resource. Save your updated file and confirm that it passes the cfn-lint checks. Retry the cfn-guard command; it should now run without failures.

Troubleshooting

If you encounter unexpected messages, try the following steps:

  • Before using cfn-guard, verify your template syntax with cfn-lint.
  • Verify you are using the latest public versions of cfn-lint and cfn-guard. For cfn-lint, check that you have the latest resource specification files by using the -u flag. For cfn-guard, do a git pull from the folder where you first cloned the repository. Then, build the updated binary, as you did in the Getting Started section.
  • Use the verbose tracing flags for the cfn-guard command as described here. Use -v, -vv, or -vvv for controlling the levels of verbosity. Try your previous commands with -v to see how the tool scans your template’s resources section with INFO messages and how it expands rules for evaluation.
  • If you can’t figure out what’s wrong, it might be a bug. Check the open issues on the repository and, if you can’t find a case that matches yours, please open a new bug report.

Tips and recommendations

This post covers only a few of cfn-guard’s current features. There are other features, rule conventions, and optional flags covered in the readme file in the GitHub repository here. Consider the following tips and recommendations as you learn more about cfn-guard:

  • Consider creating multiple files that are specific to storage resource types, for example, or rule files specific to a particular list of compliance guidelines. Smaller resource files will be easier to reuse, change, and maintain. Also, consider having a predictable naming convention, and version-controlling the rule files, just as you would other code assets in your organization.
  • Rule files can include resources that do not appear in the template you are evaluating. This allows you to put all your storage-related rules in a single file and evaluate templates that have some of these specific storage types defined. Cfn-guard will only evaluate rules against matching resource types in your template.
  • Say you have a large CloudFormation template with dozens of resources. Rather than writing every rule from scratch, you can use the rulegen command option. This option scans your template and generates default rules for the resources. You can them improve upon these base rules that rulegen provides, taking advantage of cfn-guard language conventions like variables and lists. As a best practice, use the rulegen command option on templates that already comply with your company’s policies.
  • By using both cfn-lint and cfn-guard as part of their authoring process, you can achieve an optimal shift-left testing practice, ensuring that compliant templates are evaluated as early in their development as possible. We also recommend you run both cfn-lint and cfn-guard as part of your deployment pipeline, in addition to instructing developers to use it while authoring templates. We provide another companion tool, cfn-guard-lambda, to make it easy to integrate cfn-guard as a build step in pipelines. It helps deploy cfn-guard as an AWS Lambda function. We plan to cover this pipeline use case in a future blog post.
  • You can use cfn-guard-lambda anywhere you want to make a Lambda call directly from another function. For example, you can build a custom AWS Config rule that calls cfn-guard-lambda so it evaluates the same rules as resources get updated, to confirm ongoing compliance. Combining cfn-guard with AWS Config gives you additional options; for example, you can make compliance notifications more detailed by repurposing the output of the cfn-guard execution.
  • If you have invested in your own custom rules with tools like cfn-nag, cfnripper, or cfn-lint custom rules, you can continue to use those in combination with cfn-guard as separate pipeline build steps. If you want to reduce the maintenance time of your rule code (versus scripting language code) and speed up performance, you may choose to migrate the rules to cfn-guard. We will provide ready-to-use rule collections on GitHub, and we welcome contributions from the open source community.

Summary

This article covers a CloudFormation compliance scenario where you can proactively evaluate resource configurations with cfn-guard without writing custom code in Python or other languages. You should consider using cfn-guard both early in the template authoring process and as part of a deployment pipeline. Because it is written in Rust, its compiled binaries offer excellent performance when compared to tools written with traditional scripting languages. Since it is available as an open-source tool, it empowers you to experiment with best practices such as shift-left testing and DevSecOps quickly. We expect the community to contribute rule collections via GitHub, further increasing cfn-guard’s value proposition. We look forward to learning how customers use cfn-guard in new scenarios, and to your continued feedback so we can improve it.

 

About the Authors

Luis Colon is a Senior Developer Advocate at AWS specializing in CloudFormation. Over the years he’s been a conference speaker, an agile methodology practitioner, open source advocate, and engineering manager. When he’s not chatting about all things related to infrastructure as code, DevOps, scrum, and data analytics, he’s golfing or mixing deep house or progressive trance music. You can find him on twitter at @luiscolon1.
Josh Joy is a Security Transformation Consultant with AWS Global Security Practice, a part of our Worldwide Professional Services Organization. He has significant field experience with the use of CloudFormation Guard, as well as many other AWS service offerings. Josh helps customers improve their security posture as they migrate their most sensitive workloads to AWS. Josh enjoys diving deep and working backwards in order to help customers achieve positive outcomes. He is based in New York City.
Raisa Hashem is a Senior DevOps Security Specialist in the AWS Professional Services team based out of Melbourne, Australia. On a day to day, Raisa works with Enterprises to transform their security posture through best practices education, developing security controls and infrastructure automation. Raisa is passionate about women in tech, is a student of Indian Classical Dance, and loves learning about intelligent technology and its application in human physiology.