AWS Cloud Operations & Migrations Blog

Introducing AWS CloudFormation Guard 2.0

In their blog post published last year, Write preventive compliance rules for AWS CloudFormation templates the cfn-guard way, Luis, Raisa, and Josh showed you how to use CloudFormation Guard, an open source tool that helps validate your AWS CloudFormation templates against a rule set to keep AWS resources in compliance with company guidelines. Since the general availability of CloudFormation Guard version 1.0 last year, customers asked for better ways to create rule sets and consume the tool. We listened to the feedback and released version 2.0 of Guard!

In this blog post, I share some of the domain-specific language (DSL) changes we made in this version. In addition to writing rules in a more concise and simple way you can take advantage of new features like rule testing and migration of your existing rules to the new format.

Template validation

I’ll start by showing you how to run Guard from the command line to validate templates against rule sets.

In Guard 1.0, to check your-test.template against your-test.ruleset, you use the check subcommand together with -t and -r flags to specify the template and rule set:

% cfn-guard check -t your-test.template -r your-test.ruleset

In Guard 2.0, we changed check to validate to emphasize the focus on verification and validation. With the new version, you can now validate multiple templates against multiple rule sets. For example, you use the -d (or --data) flag and the -r (or --rules) flag and to specify, respectively, a file name or a directory name for your JSON- or YAML-formatted templates and a rules file or a directory for your rules:

% cfn-guard validate -d your-test.template -r your-test.rules

For more information about Guard 2.0 usage, flags, and subcommands, run cfn-guard --help from the command line.

More concise and simple rules

With Guard, you write rule sets to check if resource configurations comply with your expectations. One of the rule set language enhancements in Guard 2.0 is the ability to use type blocks. In the sample template snippet, you describe an Amazon Elastic Block Store (Amazon EBS) volume. You enable the encryption property for your volume and specify 10 GiBs for the volume size and gp2 for the volume type:

---
AWSTemplateFormatVersion: '2010-09-09'
Description: Sample template
Resources:
  SampleVolume:
    Type: AWS::EC2::Volume
    Properties:
      Encrypted: true
      Size: 10
      VolumeType: gp2

Now, let’s say you want to validate that all EBS volume resources in your template have the Encrypted property set to true, that volume size is not exceeding 10 GiBs, and that volume type is set to gp2. In Guard 1.0, you create the following rule set:

AWS::EC2::Volume Encrypted == true
AWS::EC2::Volume Size <= 10
AWS::EC2::Volume VolumeType == gp2

With Guard 2.0, you can rewrite the rule set so that it’s more concise, with a type block that encloses and groups relevant properties together:

AWS::EC2::Volume {
    Properties {
        Encrypted == true
        Size <= 10
        VolumeType == 'gp2'
    }
}

To try it out for yourself, create a your-test.template file that contains the example template snippet shown earlier and a your-test.rules file that contains the example rule that uses the new 2.0 syntax. Next, run Guard 2.0 as shown here:

% cfn-guard validate -d your-test.template -r your-test.rules

You should see the following output:

Summary Report Overall File Status = PASS
PASS/SKIP rules
default    PASS

Note: When you compare syntax between the 1.0 and 2.0 versions, gp2 is quoted in the version 2.0 rule set, but not in the version 1.0 rule set. Guard 2.0 now requires that string literals are always quoted with single or double quotes. There are two reasons for this:

  • To drive consistency for string literals across let assignments and comparison statements, including array literals such as ['gp1', 'gp2'].
  • Guard 2.0 now permits property access on the right side of a comparison statement too, such as Properties.Size >= VolumeSize. Without quotes, it’s now ambiguous whether the intent is to perform a string comparison or access a property.

Guard 2.0 introduces language enhancements that give you the ability to write complex rules in a clear and unambiguous way. You can now use the Conjunctive Normal Form (CNF) to describe a set of logical AND clauses across OR clauses. For example, you write a rule with four conditions to be met: A, B, C, and D. The logical evaluation of your conditions can be summarized as follows: A AND B AND C AND D. Now, for C and D, you also want to allow two choices of values for each, such as C1, C2 for the former and D1, D2 for the latter. For C and D conditions to be satisfied, either C1 or C2 and D1 or D2 must be met. The overall logical evaluation of your conditions can now be summarized as: A AND B AND C AND D, where C = C1 OR C2 and D = D1 OR D2.

Let’s reuse the previous sample template snippet. Add a property to specify the Availability Zone in which you want to create your volume (in the following example, us-west-2b):

---
AWSTemplateFormatVersion: '2010-09-09'
Description: Sample template
Resources:
  SampleVolume:
    Type: AWS::EC2::Volume
    Properties:
      AvailabilityZone: us-west-2b
      Encrypted: true
      Size: 10
      VolumeType: gp2

Now let’s say that in addition to previous checks, you want to validate that EBS volume types are either gp2 or gp3 and the value for AvailabilityZone is either us-west-2b or us-west-2c. In a Guard 1.0 rule set you would specify the following:

AWS::EC2::Volume Encrypted == true
AWS::EC2::Volume Size <= 10
AWS::EC2::Volume VolumeType == gp2 |OR| AWS::EC2::Volume VolumeType == gp3
AWS::EC2::Volume AvailabilityZone == us-west-2b |OR| AWS::EC2::Volume AvailabilityZone == us-west-2c

This example uses the 2.0 syntax:

AWS::EC2::Volume {
    Properties {
        Encrypted == true
        Size <= 10
        VolumeType == 'gp2' OR
        VolumeType == 'gp3'
        AvailabilityZone == 'us-west-2b' OR
        AvailabilityZone == 'us-west-2c'
    }
}

In Guard 2.0, you can use the OR operator in uppercase or in lowercase form.

The IN operator

You can use the IN operator to rewrite OR-based clauses more concisely. Let’s rewrite the previous example rule where, besides encryption at rest and volume size, you validate that VolumeType and AvailabilityZone properties for AWS::EC2::Volume resource types in your template are, respectively, gp2 or gp3 and us-west-2b or us-west-2c:

AWS::EC2::Volume {
    Properties {
        Encrypted == true
        Size <= 10
        VolumeType in ['gp2', 'gp3']
        AvailabilityZone in ['us-west-2b', 'us-west-2c']
    }
}

In Guard 2.0, you can specify the IN operator in either uppercase or lowercase form.

Filtering

Guard 2.0 offers a powerful new filtering feature. With filtering, you can select and further refine your selection of target resources or values against which to evaluate clauses. The type block syntax shown above is effectively a filter that is equivalent to the following form:

let volumes = Resources.*[ Type == 'AWS::EC2::Volume' ]
when %volumes !empty {
    %volumes.Properties {
         Encrypted == true
         Size <= 10
         VolumeType in ['gp2', 'gp3']
         AvailabilityZone in ['us-west-2b', 'us-west-2c']
    }
}

Filtering is the default mode for selecting resource types. You can use filters to write conditions (combinations of AND of OR clauses) to select a given set of resource types that meets your criteria. A type block is just syntactic sugar for a filter that matches only by type, such as Resources.*[ Type == 'AWS::EC2::Volume' ].

The following example template describes a volume of type gp2 and one of type io1, with different property values for each of the two volumes:

---
AWSTemplateFormatVersion: '2010-09-09'
Description: Sample template
Resources:
  MyGp2Volume:
    Type: AWS::EC2::Volume
    Properties:
      AvailabilityZone: us-west-2b
      Encrypted: true
      Size: 10
      VolumeType: gp2
  MyIo1Volume:
    Type: AWS::EC2::Volume
    Properties:
      AvailabilityZone: us-west-2b
      Encrypted: true
      Iops: 1000
      Size: 30
      VolumeType: io1

Let’s take a look at an example of using filters to validate properties for both volumes in the previous example. First, create a filter to select all resources of type AWS::EC2::Volume. You further refine the selection of volumes into two sets: your_volume_no_iops_specified and your_volume_iops, depending on whether the Iops property has been specified (in the io1 volume use case) or not (in the gp2 use case). You also need one rule for each use case. Guard 2.0 includes another new feature, named rules, that you can use to create rules with a given name. For each rule, you first check if the variable relevant to the set for a specific use case is not empty and if so, you validate property values for the use case:

let your_volume = Resources.*[ Type == 'AWS::EC2::Volume' ]
let your_volume_no_iops_specified = %your_volume[ Properties.Iops NOT EXISTS ]
let your_volume_iops = %your_volume[ Properties.Iops EXISTS ]

rule sample_gp2_volumes {
    WHEN %your_volume_no_iops_specified !EMPTY {
         %your_volume_no_iops_specified.Properties.Size == 10
         %your_volume_no_iops_specified.Properties.VolumeType == 'gp2'
    }
}

rule sample_io1_volumes {
    WHEN %your_volume_iops !EMPTY {
         %your_volume_iops.Properties.Iops == 1000
         %your_volume_iops.Properties.Size == 30
         %your_volume_iops.Properties.VolumeType == 'io1'
    }
}

You can use NOT, EXISTS, WHEN, and EMPTY in uppercase or in lowercase form.

In the preceding example, I introduced filtering. All type blocks specified without a surrounding named rule block, as shown in the examples above, belong to a named rule called default. When you use type blocks, you should see the default rule referenced in the summary output when you run Guard 2.0.

Note: Filters and named rules are the prescribed way to express clauses in Guard 2.0. Use both features to enhance readability, support refactoring and allow for complete flexibility in recombining rules for higher order clauses. Type blocks are restrictive filtering types, confined to only filter by resource type. Unless you expect this type block behavior, use filters and named rules instead.

Named rules

With named rules, you assign a name to a set of rules to create a modular validation block that you write once and reuse with other validation rules. In the following example, you create a sample_volume_checks named rule that contains sample statements to validate property values for your volume:

let volumes = Resources.*[ Type == 'AWS::EC2::Volume' ]
rule sample_volume_checks when %volumes !empty {
    %volumes.Properties {
         Encrypted == true
         Size <= 10
         VolumeType in ['gp2', 'gp3']
         AvailabilityZone in ['us-west-2b', 'us-west-2c']
    }
}

Let’s go over an example of rule reuse and correlation. You will use an example template that describes an Amazon Simple Storage Service (Amazon S3) bucket with versioning and server-side encryption enabled. In the following example, you use AES256 for the server-side encryption algorithm:

---
AWSTemplateFormatVersion: '2010-09-09'
Description: Sample template
Resources:
  SampleBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256
      BucketName: !Sub 'sample-bucket-${AWS::Region}-${AWS::AccountId}'
      VersioningConfiguration:
        Status: Enabled
  SampleVolume:
    Type: AWS::EC2::Volume
    Properties:
      AvailabilityZone: us-west-2b
      Encrypted: true
      Size: 10
      VolumeType: gp2

Let’s use reusable rules to consolidate validation of encryption-related properties in the template. For example, create and add, to a rules file, two named rules that use filters for volume and bucket resources. Create and add another named rule to reference both rules:

let volumes = Resources.*[ Type == 'AWS::EC2::Volume' ]
rule sample_volume_encryption when %volumes !empty {
    %volumes.Properties.Encrypted == true
}

let buckets = Resources.*[ Type == 'AWS::S3::Bucket' ]
rule sample_bucket_encryption when %buckets !empty {
    %buckets.Properties {
        BucketEncryption.ServerSideEncryptionConfiguration[*] {
            ServerSideEncryptionByDefault.SSEAlgorithm == 'AES256'
        }
    }
}

rule sample_encryption_at_rest {
    sample_volume_encryption
    sample_bucket_encryption
}

With the first two rules, your intent is to validate that all volumes you describe in your template have the Encrypted property set to true and that all buckets you describe in your template use the AES256 algorithm for server-side encryption. Now add two new rules: one for your volume and one for your bucket. You use the following sample rules to validate properties for each resource type. You also reference sample rules for encryption at rest validation as shown earlier:

rule sample_volume when %volumes !empty {
    sample_volume_encryption
    %volumes.Properties {
        Size <= 10
        VolumeType in ['gp2', 'gp3']
        AvailabilityZone in ['us-west-2b', 'us-west-2c']
    }
}

rule sample_bucket when %buckets !empty {
    sample_bucket_encryption
    %buckets.Properties.VersioningConfiguration.Status == 'Enabled'
}

After you run Guard 2.0 to validate your template against these rules you should see the following output messages:

Summary Report Overall File Status = PASS
PASS/SKIP rules
sample_volume_encryption     PASS
sample_bucket_encryption     PASS
sample_encryption_at_rest    PASS
sample_volume                PASS
sample_bucket                PASS

In the preceding example, I showed you an example of reusing named rules by using references. Now let’s take a look at how the rule correlation works. For example, change the Encrypted: true volume property in the template to Encrypted: false and remove the entire BucketEncryption configuration block. When you run Guard 2.0, you should get error messages like the following:

Summary Report Overall File Status = FAIL
PASS/SKIP rules
FAILED rules
sample_volume_encryption     FAIL
sample_bucket_encryption     FAIL
sample_encryption_at_rest    FAIL
sample_volume                FAIL
sample_bucket                FAIL

If you also remove sample_volume_encryption and sample_bucket_encryption references from your sample_volume and sample_bucket rules, when you run Guard 2.0 again, you should see a mix of fail and pass results:

Summary Report Overall File Status = FAIL
PASS/SKIP rules
sample_volume                PASS
sample_bucket                PASS
FAILED rules
sample_volume_encryption     FAIL
sample_bucket_encryption     FAIL
sample_encryption_at_rest    FAIL

Why? When sample_volume_encryption, sample_bucket_encryption, and sample_encryption_at_rest named rules are evaluated against resources you describe in your template, validation fails as expected because conditions have not been met. The correlation between dependent and depending rules is lost after the removal of rule references from the sample_volume and sample_bucket named rules. The sample_volume and sample_bucket named rules are now only validating other specified properties in your template and they are passing the validation. Keep this in mind when you write reusable named rules and reference your reusable rules where needed.

Writing rules against all template sections

In Guard 1.0, you can write rules to perform validation in the Resources section of CloudFormation templates. Guard 2.0 expands validation to all CloudFormation template sections, including Description and Parameters. Take a look at the following example template snippet:

---
AWSTemplateFormatVersion: '2010-09-09'
Description: Sample template
Parameters:
  SampleVolumeAZsParameter:
    AllowedValues:
      - us-west-2b
      - us-west-2c
    Description: Please specify either us-west-2b or us-west-2c here.
    Type: String
  SampleVolumeTypeParameter:
    AllowedValues:
      - gp2
      - gp3
    Default: gp2
    Description: Please specify either gp2 or gp3 here; default is gp2.
    Type: String
Resources:
  SampleVolume:
    Type: AWS::EC2::Volume
    Properties:
      AvailabilityZone: !Ref 'SampleVolumeAZsParameter'
      Encrypted: true
      Size: 10
      VolumeType: !Ref 'SampleVolumeTypeParameter'

With the following example rules, you validate that the template description is what you expect and that the SampleVolumeAZsParameter and SampleVolumeTypeParameter parameters are both of String type and contain values you expect. You also validate that there is no default value set for SampleVolumeAZsParameter and that gp2 is set as the default value for SampleVolumeTypeParameter:

rule sample_template_description {
    Description == 'Sample template'
}

rule sample_volume_type_parameter {
    let allowed_volume_types = ['gp2', 'gp3']
    Parameters.SampleVolumeTypeParameter.Type == 'String'
    Parameters.SampleVolumeTypeParameter.AllowedValues in %allowed_volume_types
    Parameters.SampleVolumeTypeParameter.Default == 'gp2'
}

rule sample_volume_azs_parameter {
    let allowed_volume_azs = ['us-west-2b', 'us-west-2c']
    Parameters.SampleVolumeAZsParameter.Type == 'String'
    Parameters.SampleVolumeAZsParameter.AllowedValues in %allowed_volume_azs
    Parameters.SampleVolumeAZsParameter.Default == null
}

In Guard 2.0, you can now write rules for any file (not just CloudFormation templates) that are formatted in JSON or YAML.

Rule testing

You can now write unit tests against your rules to validate rules work as expected. You use the new test subcommand to test expected behavior for your rules. You create a test data file, formatted in either JSON or YAML, where you describe mock resource configurations along with pass/fail expectations for your rules. Next, you run Guard 2.0 to test the rule validation logic.

Let’s take a look at how to specify a test data file with the -t (or --test-data) flag and a rule set file with the -r (or --ruleset-file) flag:

% cfn-guard test -t your-test.yaml -r your-test.rules

Note: The name of the file that contains tests for your rules must end in .json, .JSON, .jsn, .yaml, .YAML, .yml. Otherwise, it will be ignored.

Let’s say you want to write a test for the following example rule that you describe in a your-test.rules file to verify volume size is less than or equal to 10 GiBs:

rule sample_volume {
    AWS::EC2::Volume {
        Properties.Size <= 10
    }
}

Create a your-test.yaml file with the following example test data:

---
- input:
    Resources:
      SampleVolume:
        Type: AWS::EC2::Volume
        Properties:
          Size: 100
  expectations:
    rules:
      sample_volume: FAIL

In the test data file, you create a mock SampleVolume resource, where you set the volume size to be 100 GiBs. You also describe the expectation of mock data to fail validation against the sample_volume rule, because you are specifying a volume size larger than 10 GiBs. The rule should pass the test:

% cfn-guard test -t your-test.yaml -r your-test.rules
PASS Expected Rule = sample_volume, Status = FAIL, Got Status = FAIL

You can organize your test data file by using more than one input section.

Upgrade path

For more information about Guard, including how to install it, see the GitHub repository.

If you have Rust and Cargo installed on your machine, it is easy to install Guard:

% cargo install cfn-guard

You can upgrade your existing rule sets based on the Guard 1.0 syntax by using the migrate subcommand as shown here:

% cfn-guard migrate --rules your-test.ruleset --output your-test.new-rules

Conclusion

In this blog post, I shared highlights of Guard 2.0 including syntax differences and new features you can use to write more concise rules and create tests for your rules. We look forward to your feedback so we can continue to improve Guard. We welcome your contributions to this open source project!

About the Author

Matteo Rinaudo

Matteo Rinaudo is a Senior Developer Advocate for AWS CloudFormation. He is passionate about the DevOps mindset, infrastructure-as-code and configuration management. In his spare time, Matteo enjoys spending time with his wife, reading and listening to classical music. You can find Matteo on Twitter at @mrinaudo.