Multi-environment CI/CD pipelines with AWS CodePipeline and open source tools

A common scenario AWS customers face is how to automate their infrastructure deployments on AWS. Customers must create a secure, agile workflow that deploys to the cloud and uses their preferred AWS services. Customers also need a reliable, supportable deployment pattern driven by automated workflows that are not overly complex and difficult to manage. Customer organizations frequently seek the ability to hand off elements of their deployment patterns to different teams in their organizations. These requirements may seem daunting to these teams. They need time and training to understand the technology they are asked to support.

In this post, we demonstrate how to reduce complexity and increase agility by creating a workflow geared toward operational support teams. By using this simplified deployment pattern, the responsibility to support the solution does not rest solely with engineers. Rather, this solution allows both engineering teams and operational support roles across the organization to use and support it. We also show how to deploy AWS infrastructure to multiple AWS accounts using open source tools and following best practices. We explain a repeatable deployment pipeline for use throughout multiple environments.

The project we create in this post addresses when a customer must deploy an application to AWS into development, QA, and production environments. In addition to explaining how to configure the deployment pipeline, we also cover how to arrange the AWS CloudFormation templates. We do this by function so that they can deploy as a cohesive application stack.

We also show how to deploy AWS Directory Service for Microsoft Active Directory. We deploy AWS Managed Microsoft AD, the roles, security groups, and other requirements necessary to run AWS Managed Microsoft AD in AWS. We dive deep into how to manage several AWS CloudFormation templates and their dependencies.

Lastly, our goal is to provide the knowledge and confidence to adopt this pattern and use it successfully as a multi-account deployment pattern.

What are we deploying?

At a high level, we are deploying AWS Managed Microsoft AD with several other AWS services. We do this to manage AWS Managed Microsoft AD in the AWS Cloud effectively. The combination of these services illustrates how users can:

• Effectively manage AWS Managed Microsoft AD
• Secure the cloud environment
• Configure logging
• Address several other operational needs

To begin, we deploy private subnets with no internet accessibility. Some organizations may require no internet accessibility with their AWS Managed Microsoft AD solution. This demo shows how to accomplish that. By not adding an internet gateway (IGW), we reduce security concerns or overhead that may come with hosting one. AWS manages patching for AWS Managed Microsoft AD, and we have VPC endpoints to communicate to AWS services across the VPC. We deploy a VPC service endpoint that, when used with AWS PrivateLink, connects AWS accounts to one another. Using AWS PrivateLink, requests send between accounts without traversing the internet. VPC traffic that is connected through AWS Managed Microsoft AD requires using hosts joined to the domain with elevated privileges to configure and modify the domain controllers.

For encryption, this solution uses AWS Key Management Service (AWS KMS) to encrypt our logs with an encrypted key. In the AWS KMS CloudFormation template, we add all of the necessary roles that need to encrypt/decrypt the key to the AWS KMS key policy. For logging, we use VPC Flow Logs and AWS CloudTrail, which sends to an encrypted Amazon Simple Storage Service (Amazon S3) bucket for storage. We use AWS Systems Manager for securing secrets and connecting to our instances. We deploy AWS Systems Manager Parameter Store to store our secrets. We use AWS Systems Manager Session Manager to connect to our Amazon Elastic Compute Cloud (Amazon EC2) instances. Lastly, a Network Load Balancer (NLB) deploys for directing requests to our AWS Managed Microsoft AD. Users from other accounts can access this NLB by either VPC peering or connecting through VPC service endpoints.

Solution overview

The following figure is an overview of the solution we are deploying. This diagram illustrates the multi-environment deployment pattern.

The following figure illustrates, in more detail, the services that we are deploying with AWS CloudFormation:

Prerequisites

To follow the steps to provision the pipeline deployment, we must complete these prerequisites first:

• An AWS account with local credentials properly configured (typically under ~/.aws/credentials).
• Latest version of the AWS Command Line Interface (AWS CLI).
• Python installed. Note: Python 2 is deprecated and we recommend using Python 3 to build this project solution.
• A Git client to clone the source code provided, and a GitHub repository.
• AWS credentials properly configured to allow our Git client to authenticate and push code to the GitHub repo.
• An OAuth token to configure access from AWS CodePipeline to GitHub. We can configure OAuth tokens in our GitHub account. Find more information in the AWS CodePipeline user guide.

Open source tools

Stacker

Stacker is an open source tool and library created by the Remind engineering team and released to the open source community. Its use is to orchestrate and manage CloudFormation stacks across one or more accounts. Stacker can manage stack dependencies automatically and allow the output parameters of a given stack to be passed as input to other stacks. It can even manage stacks deployed in different accounts and makes sure that the stacks are created in the right order. It can also parallelize the deployment of non-dependent stacks, thus reducing deployment time significantly. Now let’s configure stacker in our local workstation.

To install stacker in our workstation, we can use pip:

pip install --user stacker

For further information, visit the stacker GitHub repo or stacker Documentation, Release 1.7.0.

InSpec

InSpec in an open source framework created by Chef for testing and auditing applications and infrastructure. InSpec allows users to create desired state tests and run them against their actual running infrastructure or application. We are using InSpec for AWS by creating a suite of tests that run against our AWS account to ensure that certain conditions ring true. In this post, we demonstrate how to use InSpec to create a set of desired state integration tests. These tests help to make users’ applications more secure and reliable.

This solution requires AWS credentials. If there are not configured credentials in the local workstation, visit the AWS CLI user guide.

To install the InSpec utility on macOS, use the following steps. For Windows and Linux, installation instructions are available on the InSpec download page.

To install the InSpec utility on a macOS:

brew cask install chef/chef/inspec

Create an InSpec profile:

inspec init profile --platform aws inspec_tests

At the prompt Do you accept the 1 product license (yes/no), enter yes. A new profile is created:

After the InSpec profile is successfully created, our terminal screen will look similar to the following image:

Run the tests:

inspec exec inspec_tests -t aws://us-east-1

After running the tests, our output should look something like the following:

To learn more about InSpec for AWS, visit the GitHub repo.

cfn-python-lint

We use cfn-python-lint for AWS CloudFormation validation, resource property value validation, and an assessment of some AWS best practices. This tool is a CloudFormation Linter created by AWS and released to the open source community under the MIT No Attribution license. An example cfn-python-lint application would include ensuring that all of the resource properties in a template actually have a value and that value is the proper data type. We use this utility in this tutorial to ensure that the CloudFormation templates we are deploying are following best practices.

Once again, we use pip to install our utility:

pip install cfn-lint

Run the following command to ensure that cfn-lint was installed properly:

cfn-lint --version

We see output similar to the following image. Our cfn-lint version may be different:

AWS services used

Let’s review the AWS services we are deploying with this project.

• AWS KMS is a fully managed service for creating and managing cryptographic keys. These keys work seamlessly with AWS services and we are using a AWS KMS key in this demo to encrypt some of our resources.
• The created Amazon Virtual Private Cloud (Amazon VPC) has two private subnets, no internet access, and VPC endpoints for communication. VPC endpoints communicate with AWS services instead of traversing the internet as an additional layer of security.
• AWS Managed Microsoft AD is an AWS service built on actual Microsoft Active Directories. AWS Managed Microsoft AD allows AWS resources and other directory-aware workloads to use Active Directory in the AWS Cloud.
• Amazon EC2 instances are required to manage AWS Managed Microsoft AD as direct access to the domain controllers is not provided. Because there is no IGW or NAT in the VPC, Systems Manager connects to our instances. AWS Systems Manager Session Manager connect to the instances so that we do not have to open up a port on the host for administrators to connect. The AWS Systems Manager Agent (SSM Agent) runs on the host and we connect to the instance using the AWS Management Console. AWS Managed Windows AMI 2016 and after come with the SSM Agent installed. A default Windows AMI is used for the deployment.
• CloudTrail enables governance, compliance, operational auditing, and risk auditing of AWS accounts. Enabling CloudTrail increases visibility into user and resource activity by recording the console actions and API calls. We enable VPC Flow Logs to capture information on the IP traffic going to and from network interfaces in the VPC.

Infrastructure overview

What we deploy first:

In the solution, we use CodePipeline to create and deploy the following:

• Stack 1: Create CodePipeline and name it cassis.
• Stack 2: Create the stacker execution roles.

Steps

1. Clone the provided source code:

• Step 1: Download the repository.
• Step 2: From our local workstation, switch to the directory into which we want to copy the cloned directory.
• Step 3: Enter git clone and paste the URL we copied to the clipboard and push Enter on the keyboard:

$ git clone https://github.com/YOUR-USERNAME/YOUR-REPOSITORY

Let’s go through each of the directories to understand their purpose.

• codepipeline/: CloudFormation templates to deploy the CI/CD pipeline, including CodePipeline, AWS CodeCommit, AWS CodeBuild, and open source software cfn-nag, cfn-lint, and stacker.
• stacker: Configuration files required by stacker to perform CloudFormation stack deployments.
  • stacker/buildspec.yaml: CodeBuild buildspec that installs and invokes stacker for CFN provisioning.
  • stacker/stacker-config.yaml: Stacker config file containing stack descriptions and input parameters.
  • stacker/stacker-profiles: AWS profiles file stacker uses to deploy the various stacks.
  • stacker/stacker-env/*: Configuration files in which the parameter values for our CloudFormation templates are set based on environment.
• templates/*: The sample templates for the stacks mentioned that are validated and deployed by the pipeline.

2. Navigate to the GitHub repository GitHub console.

3. Follow the steps in “Configure your pipeline to use a personal access token (GitHub and CLI)” to configure the OAuth token, if not already done. We will be using this token to authenticate from CodePipeline to our GitHub repository.

4. At this point, we should have the provided artifact cloned locally with the stacker/ and templates/ folders (created earlier). Let’s clone the GitHub repository locally (if not already done):

git clone https://github.com/YourGitUserName/YourRepoName.git

5. Next, in the IDE of choice, open the file stacker/stacker-profiles. Update the two role_arn account values to the account deploying to:

role_arn = arn:aws:iam::123456789:role/cassis-StackerExecutionRole

role_arn = arn:aws:iam::123456789:role/cassis-StackerMasterRole

Now let’s deploy our sample pipeline to our development account.  This requires deploying two CloudFormation templates.

The first stack creates the entire pipeline infrastructure, including:

• CodePipeline
• CodeBuild
• Git configuration.
• IAM roles and policies that comprise the pipeline

Creating stack 1

The first stack provisions the infrastructure pipeline referenced previously. The stack will provision a CodeCommit repo, a CodePipeline pipeline, and four CodeBuild projects. The first CodeBuild project runs cfn-nag against our CloudFormation templates using a built-in buildspec configuration. CodeBuild project number two similarly uses a built-in buildspec configuration and runs in parallel with CodeBuild project number one.

CodeBuild projects reference by number, as they are numbered in the CodePipeline configuration (see CodePipeline/code-pipeline-template.yaml). The next CodeBuild project will run the InSpec integration tests using a built-in buildspec file.

After these steps run successfully, then the deployment of the CloudFormation templates to the AWS account will occur automatically as the final step. This final step does not come with a built-in buildspec file. This CodeBuild’s buildspec pushes directly to GitHub where it instructs CodeBuild on how to configure the job.

1. Navigate to Console, Services, CloudFormation.

2. Select Create stack. From the drop-down menu under Stacks, select With new resources (standard), as shown in the screenshot:

3. On the next screen navigate to Upload a template file, then select Choose file and upload the codepipeline/stacker-execution-role-template.yaml file:

4. Select Next.

5. Update the stack details. Use the Stack name cassis-pipeline-role, Namespace is cassis, and enter the AccountID you are deploying. Then select Next. On the next page, select Next again.

6. Review the configuration. Once satisfied, navigate to the bottom of the page and select I acknowledge that AWS CloudFormation might create IAM resources with custom names. Finally, select Create stack.

7. On the next page, monitor the progress of the CloudFormation build. Once complete, review all of the deployed resources.

Creating stack 2

Now that we created the pipeline, let’s deploy the codepipeline/codepipeline-template.yaml CloudFormation template. This template will create the stacker execution roles; which we will explain in further detail after we created them. Complete the following steps to get started in creating these roles.

Steps

1. Open Console, Services, CloudFormation. Select Create stack and complete the following sections:

• Prerequisite – Prepare template: Leave the default Template is ready selected.
• Specify template: Select Upload a template file.
• Select Choose file and select the file from the private repository, codepipeline/codepipeline-template.yaml.
• Choose Next.

2. On the page Specify stack details, complete the following sections:

• Stack name: Enter the name cassis-codepipeline.
• Parameters: Leave the defaults (or change as required). Make sure to complete these two mandatory fields:
  • StackerMasterAccountID: Account ID where the StackerMasterRole lives.
  • CodePipelineArtifactsS3BucketName: Specify a unique S3 bucket name that CodePipeline uses to store build artifacts. For this project, use the prefix cassis (such as cassis-pipeline-artifacts-XXXXXX where X is a chosen random number).
  • TargetAccount: Copy and paste the AWS account number in this field.
  • StackerEnvParametersFile: Specify the environment parameter file based off the environment we are deploying to.
  • GitHubRepoName: Name of the GithHub repository our code is stored in.
  • GitHubBranchName: Leave this as the master branch
  • Namespace: This is a prefix for all provisioned resources. Leave this as cassis for this exercise.
  • OAuthToken: Enter the OAuth token used to authenticate to your GitHub account.
  • GithubOwner: GitHub account used to connect to GitHub repo.

3. Select Next.

4. On the Configure stack options page, leave the defaults in place and select Next.

5. On the Review page, take a moment to review the inputs:

• Create a unique S3 bucket name and your correct account number entered or this project will not work.
• Enter your GitHub repository name and your user token.
• Confirm that all details are entered correctly, then select I acknowledge that AWS CloudFormation might create IAM resources with custom names. This is required, as the template creates IAM roles for the various CodeBuild projects and the CodePipeline pipeline.
• Select Create stack. It takes a few minutes for the first stack to create. Once the stack has been created successfully, review the Resources that have been created.

6. Next, navigate to the IAM console and select Roles. The six roles that the stack creates are here.

7. Let’s explore the purpose of each role:

• cassis-CodeBuildCFNLintRole: IAM service role created for the cfn-python-lint CodeBuild environment.
• cassis-CodeBuildCFNNagRole: IAM service role created for the cfn-nag CodeBuild environment.
• cassis-CodeBuildDeployerRole: IAM service role created for the stacker CodeBuild environment.
• cassis-CodePipelineRole: IAM service role for CodePipeline to allow the pipeline to perform tasks—such as read/write artifacts from/to the artifacts S3 bucket—to trigger the various CodeBuild environments.
• cassis-CodeBuildInspecRole: IAM service role for the InSpec CodeBuild environment.
• cassis-StackerMasterRole: IAM role used to launch stacker, which allows stacker to assume the cassis-StackerExecutionRole to deploy AWS resources via AWS CloudFormation on the various target accounts.

For simplicity, we are using a single account in our example (i.e., the same account where the pipeline resides).

Securing our pipelines

Next, we demonstrate how to securely configure integration with AWS KMS to create a secure string parameter. These are parameters that have a plaintext parameter name and an encrypted parameter value. We use Parameter Store with AWS KMS to encrypt and decrypt the parameter values of the secure string parameter. This further secures our project and is ideal to use whenever managing sensitive data.

1. In the console, navigate to the Systems Manager Service. Select Parameter Store.

2. Select Create Parameter (the button found on the upper right-hand side of the screen).

• For Name, enter AdminADPassword.
• For Description, enter Admin password for managed AD.
• For Tier, select Standard.
• For Type, select SecureString.
• For Data type, select text.
• For the purposes of this demo, use the example password zd29k3k1HbQl9nb02. If you begin deploying real workloads to this solution, change the password to secure the environment.
  
• For KMS key source, select My current account.
• For KMS Key ID, select alias/aws/ssm.
• For Value enter the example password, zd29k3k1HbQl9nb02.

• Finish by selecting Create parameter. We use this secure parameter in our AWS Managed Microsoft AD CloudFormation stack.

Using the AWS CodePipeline infrastructure pipeline

Now that the pipeline is ready and we have configured the pipeline artifacts, it’s time to use the pipeline to deploy our stacks.

Deploy the following templates:

• Stack 1: CloudTrail template
• Stack 2: EC2 template
• Stack 3: VPC Endpoint template
• Stack 4: AWS KMS template
• Stack 5: Managed Active Directory template
• Stack 6: Amazon S3 template
• Stack 7: VPC template

Steps

1. Navigate to the GitHub repository. We previously provided artifact cloned locally with the stacker/ and templates/ folders (as part of the prerequisites). Let’s close the GitHub repository:

git clone https://github.com/YourGitUserName/YourRepoName.git

2. Copy over the codepipeline/, stacker/, and templates/ directories into the Git repository.

cd stacker/

3. Open stacker/stacker-dev-env.yaml.

4. Update BucketName with a unique name. For example: yourbucketname12.

5. Create a branch locally and push the change:

git branch <feature_branch>

6. Check out the newly created branch:

git checkout <feature_branch>

7. Commit changes:

git add .
git commit -m "copying over configuration files to our Git repo"

git checkout master

9. Push the changes:

git push origin <feature_branch>

10. Create a pull request and merge the changes into the master branch. This starts a deployment in the CodePipeline.

Let’s return to the console and open up the CodePipeline service to see what’s happening. We should see the following stacks deployed:

• cassis-kms
• cassis-vpc
• cassis-mad
• cassis-cloudtrail
• cassis-ec2
• cassis-endpoints

Connect to Amazon EC2 instances with Session Manager

Now, we explore how to use Session Manager to connect to our Amazon EC2 instances created by our CloudFormation stack. Sessions Manager connects to the EC2 instances for us. We do not have to open up a port on the host for the administrators to connect to those instances.

Steps

1. Navigate to the Amazon EC2 service in the console and select Running instances.
Here, we can review the instances launched with the CodePipeline. Recall that we are not connecting to these instances using traditional RDP. We use the AWS Systems Manager Session Manager service instead.

2. Now select the Dev-WidowsADManagementServer instance and select Connect.

1. Select Session Manager as the Connection method and select Connect.

A PowerShell command-line interface should appear on the screen.

We are now connected to the instance using Session Manager.

InSpec integration tests

Now that we have successfully deployed and configured the AWS Managed Microsoft AD environment, let’s write integration tests.

We use Chef InSpec to run integration tests against our environment after it deploys. InSpec helps to create a suite of tests that help us to protect our environment. It also ensures that the essential components that comprise our application are up and working.

For example, in a situation in which a developer accidentally removes a necessary firewall port, InSpec detects whether our security group resource meets the conditions required to run our application stack. If the conditions are not met, the deployment fails.

Once we deploy the test suite, we see how InSpec adds an additional security layer to our application stack and a protective layer around our application’s functionality. Let’s look at an example test:

describe aws_security_group(group_name: input("WindowsEC2SG")) do
  it { should exist }
  it { should allow_in(port: 443, ipv4_range: '10.189.0.0/20') }
  it { should allow_in(port: 80, ipv4_range: '10.189.0.0/20') }
  it { should_not allow_in(port: 389, ipv4_range: '0.0.0.0/0') }
end

In the test, we are checking that our Windows security group allows traffic into port: 443 and port: 80, and that it blocks traffic on port: 389. This ensures that if someone mistakenly removes access to a port, or adds access to an unavailable port, the integration test fails.

Imagine that for this scenario a team member made security group changes to the EC2 security group not realizing the impact that it would have. In this case, the teammate opened up port: 389 because they forgot that we are using Session Manager to connect to the instances. We have already deployed an InSpec test, which should detect this change and cause a failure in the pipeline. Let’s see how InSpec handles this event.

From the local workstation in the Git repo, create a new branch:

git branch inspec_demo_branch

Check out the newly created branch:

git checkout inspec_demo_branch

From the root of the Git repository, open up the templates/ec2-instance-template.yaml file.

Edit the EC2 Windows Security Group and add an additional inbound rule allowing traffic on port: 389. The resource should look like this when complete:

Commit changes:

git add .
git commit -m "copying over configuration files to our Git repo"

Check out the master branch:

git checkout master

Push the changes:

git push origin <feature_branch>

Now open up the CodePipeline in the console and monitor the progress.

As expected, the pipeline fails at the Run-Inspec-Tests step.

Navigate to the Run-Inspec-Tests pipeline job and choose Details to see output. The following image shows what a failed InSpec test looks like. It helps us understand what to look for when we do see a failure.

Let’s go through the exercise of creating a couple of new tests.

Check out the branch that we were just using:

git checkout inspec_demo_branch

We are using Chef inputs for storing variables in a centralized location. These inputs are set in the inspec_tests/inspec.yml file.

Next add the following inputs to the inputs section of the inspec_tests/inspec.yml file:

- name: WindowsADManagementServer
  value: WindowsADManagementServer
- name: BucketName
  value: youruniquebucketname

Add the following tests to the inspec_tests/controls/integration_tests.rb file. Confirm that the tests are updated to reflect the S3 bucket and EC2 instance names that we deployed.

describe aws_ec2_instance(name: input("WidowsADManagementServer")) do
  it { should exist }
end

describe aws_s3_bucket( input("BucketName")) do
  it { should exist }
  it { should_not be_public }
  it { should have_default_encryption_enabled }
end

Once updated, commit and push the changes to merge this branch to the master branch.

CloudFormation parameters are input values used for setting the values for Resources. Parameters are declared in the template’s Parameters Object at the top of the template.

Using Parameters is advantageous because when we set the value of a CloudFormation resource, it allows for repeated use throughout the template. Using parameters, we can override values when deploying to different environments. We demonstrate how to accomplish this using stacker later in the project.

First, let’s look at some parameters:

Here, we are using several CloudFormation templates instead of a singular template to deploy our resources. The idea is to organize templates by function so that the repository is more readable and easier to understand. By organizing Stacks by function, some of our stacks are nested and have dependencies on one another.

Let’s look at how we accomplish this using native AWS CloudFormation functionality.

CloudFormation Outputs are values output by a stack that another stack can ingest. This allows us to nest stacks and input dependencies from one stack to another.

Let’s look at an example in a CloudFormation template:

Examine the SubnetIds Parameter. Fn::ImportValue is an intrinsic function that returns the value of a CloudFormation Output from another stack. The SubnetID value populates from the VPC stack SubnetID output. By using this pattern, we can seamlessly manage dependencies between CloudFormation stacks. If we look at the CloudFormation templates that we deployed, we can see this pattern several times.

CloudFormation resolvers allow users to pass Secrets from the Systems Manager Parameter Store into CloudFormation templates securely. This allows us to pass an encrypted string value from Systems Manager and set our AWS Managed Microsoft AD password without having to hardcode the value. The preceding image shows the Password property and how we are using the resolver to set the Microsoft AD admin password.

How are we orchestrating the deployment?

In this project, notice that CloudFormation templates are split up by function. We are doing this so that the code base is organized and intuitive to follow. One of the advantages of using stacker is how well it empowers engineers to create workflows that are easy to understand and maintain for technical professionals of varying skill sets.

If we look at the stacker-config file, we see that some of the CloudFormation resources have a variable like:

"VpcId: ${output cassis-vpc::PrivateSubnet1}".

This variable is telling the CloudFormation resource to wait to build until the VPC stack has finished building. By using this pattern, we are able to coordinate the order in which our CloudFormation stacks execute.

To illustrate, think of the relationship between the VPC and the AWS Managed Microsoft AD stacks. A resource in the AWS Managed Microsoft AD stack requires a VPC ID. We accommodate this relationship in the stacker config file and then the AWS Managed Microsoft AD stack waits to build until the VPC stack has completed.

How can we orchestrate deployments to other environments?

We can also use stacker to coordinate deployments to all of the environments that make up an application landscape. For this exercise, we are coordinating the deployment to the development, QA, and production environments. To accomplish this, we make use of the stacker environment file, which is a YAML file made up of CloudFormation override parameter values. These override parameter values set the values of the CloudFormation templates when we deploy them. If the CloudFormation template parameter is set to a value in the template, and the value is different in the stacker environment file, the stacker environment file value takes precedence.

When we want to deploy to our respective environments, all we must do is to specify the stacker environment file as a CodePipeline parameter. We can then deploy the same code base to each environment.

This pattern is useful for a number of reasons. First, in our development and QA environments, we can use smaller resources to save costs. For example, we may be able to get away with a smaller instance size testing in our development environment, which optimizes costs. Let’s look at how this works with two different environment files.

In our production environment file, we are deploying a M5 instance type. This instance type size is ideal because it helps ensure high performance from the instance we are using to manage AWS Managed Microsoft AD.

In our development environment, we are less concerned with Amazon EC2 performance and can use a smaller instance size to save on cost. To do this, we have to update the InstanceType parameter before we deploy our EC2 CloudFormation stack.

Now, let’s deploy our application stack to a different environment. To do this, we must have a second AWS account to deploy the application stack to.

Note: This deployment pattern would work using a single AWS account, but for this demo, we are deploying to a separate account.

Let’s deploy the stacker-execution-role and the CodePipeline CloudFormation templates and create our QA environment. When deploying the code-pipeline stack, make sure to update the StackerEnvParametersFile parameter with the stacker-qa-env.yaml value. Also, update the TargetAccount Parameter in the codepipeline-template.yaml file and the StackerMasterAccountId Parameter in the stacker-execution-role-template.yaml file.

Before we kick off an environment build, we have to update the InSpec tests. Because Amazon S3 names must be unique, we must update our InSpec test.

Once we have completed the deployment, we have our QA environment. Take some time to review the deployed resources. This same process is repeatable for the production environment.

Clean up

To avoid incurring future charges to our AWS accounts, delete the resources created in our AWS account for this project. We can simply destroy the pipeline stacks created earlier. Remember to empty the S3 bucket created by the cassis-codepipeline-stack stack. Otherwise, AWS CloudFormation won’t delete the corresponding stack.

Conclusion

In this post, we showed how to use popular open source tools in conjunction with AWS Developer Tools services, such as CodePipeline, CodeCommit, and CodeBuild to build, validate, and deploy infrastructure stacks across multiple accounts in a secure manner. We used cfn-lint to validate the templates, InSpec for testing, and stacker to perform the deployment of the CloudFormation stacks. We explained the details of the artifacts that were pushed to the AWS CodeCommit repository. We also showed how to deploy Managed Active Directory, given it’s a popular solution for customers.