Continuous Compliance at Scale in AWS CI/CD Pipelines Using Pulumi CrossGuard

By Aarthi Kannan, Partner Solutions Architect, Security – AWS
By Josh Kodroff, Sr. Solutions Architect – Pulumi
By Marina Novikova, Sr. Partner Solution Architect – AWS

Pulumi

An ever-increasing number of organizations are picking up the importance of GitOps practices and the everything-as-code approach to accelerate development processes and time to market.

To ensure cloud infrastructure is compliant with organizational policies and regulatory requirements, many platform engineering teams are embedding the policy-as-code practice into their DevSecOps toolchain.

In this post, we will show how to use Pulumi and its CrossGuard policy-as-code feature to ensure your infrastructure is compliant before it’s ever provisioned in Amazon Web Services (AWS).

Pulumi is an AWS DevOps Competency Partner and AWS Marketplace Seller whose modern infrastructure as code (IaC) platform empowers teams to manage cloud resources using their favorite languages.

We’ll also demonstrate how to define custom policies in Python, and how to create a reusable package of policies by storing them in AWS CodeArtifact. The package can then be consumed in an infrastructure-as-code pipeline to reject IaC that does not comply with the codified security policies.

This post is accompanied by sample code hosted in GitHub that provides code for the workflows. For detailed instructions on using the code, see the README file. First, we’ll give a brief explanation of policy-as-code and Pulumi.

What is Policy-as-Code?

Nearly all organizations have policies relating to cloud infrastructure. Organizational policies are typically related either to security requirements (like not allowing SSH access to an Amazon EC2 instance from unrestricted IP ranges), or regulatory requirements (like not allowing the use of non-HIPAA compliant managed services).

With policy-as-code, these organizational policies are described in plain text files, automatically enforced, and managed as any other code using version control, automated testing, and other standard software practices.

Policy-as-code is considered a part of DevSecOps, where security is incorporated into the same DevOps tools and practices as development and operations. A common phrase associated with DevSecOps is “shift left on security.” In other words, that means integrating security into development and infrastructure pipelines early on, rather than bringing in security to review application code or infrastructure that is near-complete, or worse has already been deployed.

As we’ll see in this post, policy-as-code allows organizations to radically shift left on security, providing application and infrastructure teams with fast, automated feedback. This enables them to quickly and reliably create secure and compliant infrastructure.

Preventive and Detective Controls

Compliance controls fit into one of two categories:

• Preventive controls: These analyze infrastructure for compliance before it’s deployed. In a typical use case, the controls are integrated as part of an infrastructure pipeline and will fail the pipeline if non-compliant infrastructure is found.
• Detective controls: These analyze infrastructure after it has been deployed and will optionally take a remediation action if non-compliant infrastructure is found (such as by terminating an EC2 instance).

Preventive controls have the advantage of providing feedback before infrastructure is deployed, allowing for faster remediation. Detective controls have the advantage of being able to detect non-compliant infrastructure regardless of the deployment method (like if they were created via the AWS Management Console).

Pulumi policy packs contain preventive controls which work on individual resources like an AWS Identity and Access Management (IAM) policy. They may also contain stack policies for deployment governance if your company or team has regulations which need to be addressed at that level.

Pulumi and CrossGuard

Pulumi is an infrastructure as code tool that allows teams to use real programming languages like TypeScript, Python, C#, and Go to define the desired state of infrastructure. Pulumi will figure out the necessary steps to turn infrastructure as it exists into the declared desired state.

CrossGuard is Pulumi’s policy-as-code feature and allows users to author policy packs that contain configurable compliance rules. At the time of writing, policy packs may be authored in TypeScript, JavaScript, or Python.

Pulumi policy packs may be enforced as part of a CI/CD pipeline, or can be enforced server-side as part of Pulumi’s Business Critical service offering. For full details of Pulumi’s paid offerings, visit Pulumi’s pricing page.

CrossGuard: AWSGuard and Custom Policy Packs

AWSGuard is an open-source Pulumi policy pack written in TypeScript that encapsulates common AWS best practices. All rules, both in AWSGuard or custom pack, are configurable in that each rule can be turned to be mandatory, advisory, or disabled.

Users can set the default level for all policies, and override specific ones as needed. For example, you can set the default to “advisory” so all teams and users are notified about organization practices, but strictly enforce the ones that are potential regulatory issues.

Policy packs can be run against Pulumi programs written in any supported language. Remember that Pulumi supports authoring IaC in a variety of languages, and in this case we’ll be using AWSGuard, which is written in TypeScript, to check a Pulumi program written in Python.

In addition to AWSGuard, we’ll also create our own custom policy pack written in Python. The custom policy pack contains additional policies which are security best practices recommended by AWS.

Solution Architecture

Next, we’ll explain how to implement\package your custom policies and how to enforce them. We can visualize the architecture with the following diagram.

Figure 1 – Target architecture for pre-deployment security policy checks.

The architecture here comprises two distinct workflows:

• The organization’s security engineers develop and maintain a custom Pulumi CrossGuard policy pack in Python. They push published versions of the policy pack to a PyPI repository. In addition to custom policies, the same organization utilizes Pulumi’s open-source AWSGuard policy pack.
• The infrastructure team develops and maintains infrastructure as code in Pulumi. They run checks against the security team’s custom policy pack in order to verify the infrastructure they develop are in compliance with the security team’s policies.

The architecture utilizes the following AWS resources:

• AWS CodeCommit repositories to hold custom Pulumi CrossGuard policy pack source code and IaC, respectively.
• AWS CodeArtifact repository to hold the published package of the Pulumi CrossGuard policy pack.
• AWS CodePipeline to deliver the infrastructure.
• AWS CodeBuild project to run Pulumi commands, including a check against both AWSGuard and the security team’s custom policies.

Package Pulumi CrossGuard Custom Policy Packs

As mentioned above, you will need to create a new CodeCommit repository for your custom compliance rules. You can clone the GitHub repo and copy the sample Python package to your new CodeCommit repository.

You can also follow the detailed instructions below to achieve the same.

--clone the existent GitHub package locally
gh repo clone aws-samples/preventive-security-controls-in-pulumi-iac-pipeline

--Configure your AWS credentials and region, and create CodeCommit repository for custom policies
aws configure
aws codecommit create-repository --repository-name custom-policy-crossguard-pkg --repository-description "Pulumi Custom CrossGuard Policies repo"

--Create local folder for CodeCommit repositories
mkdir CodeCommitRepo 
cd CodeCommitRepo

--Clone the empty repository you created above – make sure to specify your AWS region. In the example we used us-east-1
git clone codecommit::us-east-1://custom-policy-crossguard-pkg 
cd custom-policy-crossguard-pkg/

--Move the sample pyawsguard package from the cloned GitHub content to your CodeCommit local repo
mv ../../preventive-security-controls-in-pulumi-iac-pipeline/custom-policy-crossguard-pkg/pyawsguard pyawsguard 

--Add all new files to the local repo and commit to the CodeCommit repository
git add -A 
git commit -m "Initial commit of example Python package" 
git push

Now, you can inspect your CodeCommit repository in the AWS console.

Figure 2 – AWs CodeCommit repository for a custom policy pack.

In the pyawsguard folder, you can find and check setup.cfg. This file contains the metadata for the package including the name, version number, and other details. The sample code provided already has pre-filled values for some of these fields; update the details along with the author’s name and description, as needed.

Within the src/ folder is the project folder that will be distributed as a package to be installed via pip. You can add your own or customize existent policies according to your organization’s needs.

To store the package and use it as part of the pipeline, create a CodeArtifact domain named pulumi and repository named crossguard:

aws codeartifact create-domain --domain pulumi
aws codeartifact create-repository --domain pulumi --domain-owner <aws-account-id> --repository crossguard --description "Pulumi CrossGuard policies"

To generate the distribution packages and publish the packages to AWS CodeArtifact, you can follow detailed instructions in the sample code from the GitHub repository. Here, we’ll highlight only the commands you need for the CodeArtifact repository created above.

Navigate to the folder where pyproject.toml is located (pyawsguard folder) and run the following commands.

--build your package and source code
python3 -m pip install –-upgrade build
python3 -m build

--use Twine packager to upload the generated distribution to the CodeArtifact repository. Replace <aws-account-id> with your AWS Account ID
python3 -m pip install --upgrade twine
aws codeartifact login --tool twine --repository crossguard --domain pulumi --domain-owner <aws-account-id>
export TWINE_USERNAME=aws
export TWINE_PASSWORD=`aws codeartifact get-authorization-token --domain pulumi --domain-owner <aws-account-id> --query authorizationToken --output text`
export TWINE_REPOSITORY_URL=`aws codeartifact get-repository-endpoint --domain pulumi –domain-owner <aws-account-id> --repository  crossguard --format pypi --query repositoryEndpoint --output text`
python3 -m twine upload --repository  crossguard dist/*

Now, you have the distribution package to enforce the policies in your IaC pipeline.

Figure 3 – Custom policy pack distribution package in AWS CodeArtifact.

Enforcing Policy-as-Code in the Pipeline

Within the IaC pipeline, we’ll run Pulumi commands to ensure the Pulumi code is in compliance with the organization’s policies. This build will fail when non-compliant infrastructure is detected and no resources will be provisioned.

First, let’s run it locally. You can clone the sample IaC Python project from GitHub or find it in your cloned folder preventive-security-controls-in-pulumi-iac-pipeline and navigate to sample-code/sample-resources.

The content is organized as below:

• checks/ – Python project that will hold the references to the security policies packages.
• resources/ – Sample IaC code for demonstrating the preventive checks in action.

Within the checks/ folder, navigate to custom-policy-crossguard/requirements.txt. This file instructs pip on which packages and versions are to be installed.

Below the commented line (## Please add the name of the custom package deployed to AWS CodeArtifact) add the name of the policy-as-code package along with the version number. This change is required to install and include the preventive checks as part of the code build and deployment process.

A sample IaC Python project has been provided within the resources/ folder. This sample code contains IaC definitions for deploying the following:

• Amazon Elastic Block Store (Amazon EBS) volumes
• AWS Key Management Service (AWS KMS) keys
• Amazon Relational Database Service (Amazon RDS) instances
• Amazon Simple Storage Service (Amazon S3) buckets
• Amazon Elastic Compute Cloud (Amazon EC2) security groups
• Amazon Simple Queue Service (SQS) queues

These sample resource definitions are used to demonstrate the preventive checks in action, so it includes both secure and insecure definitions of resources.

Now, let’s run the project locally and then we’ll automate it with the CodeBuild step.

The following command is used to connect to the chosen CodeArtifact domain and repository with pip as the tool to enable downloading the policy-as-code package.

aws codeartifact login --tool pip --repository  crossguard --domain pulumi --domain-owner <aws-account-id>

The following snippet is used to setup the Python environment within the checks/ folder.

cd sample-resources/checks/custom-policy-crossguard
python3 -m venv ./venv
source ./venv/bin/activate
pip3 install --upgrade pip setuptools wheel
pip3 install -r requirements.txt

The following snippet is used to setup the Python environment within the resources/ folder.

cd sample-resources/resources/
python3 -m venv ./venv
source ./venv/bin/activate
pip3 install --upgrade pip setuptools wheel
pip3 install -r requirements.txt

The following command is used to initiate policy-as-code checks against the resources declared in the IaC project as a pre-deployment step.

pulumi preview –policy-pack ../checks/custom-policy-crossguard

You will see the violation of the defined policies as in the screenshot below.

Figure 4 – Preview policy violations with Pulumi.

Next, it’s time to include the policy packs to the DevOps tools and achieve continuous compliance. We need to include the above steps in the buildspec.yaml file of a CodeBuild project. You can find the buildspec.yaml example in GitHub.

The following figure highlights the relevant command in a sample CodeBuild buildspec.yaml file.

Figure 5 – Running policy packs preview as part of build process in AWS CodeBuild.

If non-compliant infrastructure is detected, the build will fail with a meaningful error message.

Figure 6 – Output of policy packs preview in AWS CodeBuild.

As an added touch, you can create Amazon CloudWatch metrics on non-compliant infrastructure. Refer to the GitHub repo for more details on the configuration.

Figure 7 – Amazon CloudWatch metrics showing the non-compliant resources.

​​Conclusion

Policy-as-code is a powerful tool for ensuring infrastructure is compliant with an organization’s security policies. With preventive controls using Pulumi CrossGuard, infrastructure and compliance teams can ensure cloud resources are compliant before they are created, using the same familiar and powerful programming languages as application development teams.

When teams combine Pulumi CrossGuard with CI/CD pipelines, you can ensure a smooth and automated process that allows teams to ship compliant infrastructure faster.

If you are looking to learn more, visit the Pulumi blog’s sections of AWS content and policy-as-code content. Check out Pulumi’s future events page to sign up for Pulumi workshops or see live demos showcasing real-world AWS solutions using Pulumi.

.

.

Pulumi – AWS Partner Spotlight

Pulumi is an AWS Partner whose modern infrastructure as code (IaC) platform empowers teams to manage cloud resources using their favorite languages.

Contact Pulumi | Partner Overview | AWS Marketplace | Case Studies