Automate Terraform Deployments with Amazon CodeCatalyst and Terraform Community action

Amazon CodeCatalyst integrates continuous integration and deployment (CI/CD) by bringing key development tools together on one platform. With the entire application lifecycle managed in one tool, CodeCatalyst empowers rapid, dependable software delivery. CodeCatalyst offers a range of actions which is the main building block of a workflow, and defines a logical unit of work to perform during a workflow run. Typically, a workflow includes multiple actions that run sequentially or in parallel depending on how you’ve configured them.

Introduction

Infrastructure as code (IaC) has become a best practice for managing IT infrastructure. IaC uses code to provision and manage your infrastructure in a consistent, programmatic way. Hashicorp Terraform is one of most common tools for IaC.

With Terraform, you define the desired end state of your infrastructure resources in declarative configuration files. Terraform determines the necessary steps to reach the desired state and provisions the infrastructure automatically. This removes the need for manual processes while enabling version control, collaboration, and reproducibility across your infrastructure.

In this blog post, we will demonstrate using the “Terraform Community Edition” action in CodeCatalyst to create resources in an AWS account.

Figure 1: Amazon CodeCatalyst Action

Prerequisites

To follow along with the post, you will need the following items:

• An AWS Builder ID for signing in to CodeCatalyst.
• A CodeCatalyst space
• Have the Space administrator role assigned in your CodeCatalyst space
• Have an AWS account associated with your space along with an associated IAM role
• A CodeCatalyst project with a source repository
• A CodeCatalyst environment configured with a connection to your target AWS account
• An Amazon S3 Bucket to store Terraform remote state file
• An Amazon DynamoDB Table to manage the locking of the state file during Terraform operations.

Walkthrough

In this walkthrough we create an Amazon S3 bucket using the Terraform Community Edition action in Amazon CodeCatalyst. The action will execute the Terraform commands needed to apply your configuration. You configure the action with a specified Terraform version. When the action runs it uses that Terraform version to deploy your Terraform templates, provisioning the defined infrastructure. This action will run terraform init to initialize the working directory, terraform plan to preview changes, and terraform apply to create the Amazon S3 bucket based on the Terraform configuration in a target AWS Account. At the end of the post your workflow will look like the following:

Figure 2: Amazon CodeCatalyst Workflow with Terraform Community Action

Create the base workflow

To begin, we create a workflow that will execute our Terraform code. In the CodeCatalyst project, click on CI/CD on left pane and select Workflows. In the Workflows pane, click on Create Workflow.

Figure 3: Creating Amazon CodeCatalyst Workflow

We have taken an existing repository my-sample-terraform-repository as a source repository.

Figure 4 : Creating Workflow from source repository

Once the source repository is selected, select Branch as main and click Create. You will have an empty workflow. You can edit the workflow from within the CodeCatalyst console. Click on the Commit button to create an initial commit:

Figure 5: Initial Workflow commit

On the Commit Workflow dialogue, add a commit message, and click on Commit. Ignore any validation errors at this stage:

Figure 6: Completing Initial Commit for Workflow

Connect to CodeCatalyst Dev Environment

For this post, we will use an AWS Cloud9 Dev Environment to edit our workflow. Your first step is to connect to the dev environment. Select Code → Dev Environments.

Figure 7 : Navigate to CodeCatalyst Dev Environments

If you do not already have a Dev Environment you can create an instance by selecting the Create Dev Environment dropdown and selecting AWS Cloud9 (in browser). Leave the options as default and click on Create to provision a new Dev Environment.

Figure 8: Create CodeCatalyst Dev Environment

Once the Dev Environment has provisioned, you are redirected to a Cloud9 instance in browser. The Dev Environment automatically clones the existing repository for the Terraform project code. We at first create a main.tf file in root of the repository with the Terraform code for creating an Amazon S3 bucket. To do this, we right click on the repository folder in the tree-pane view on the left side of the Cloud9 Console window and select New File

Figure 9: Creating a new file in Cloud9

We are presented with a new file which we will name main.tf, this file will store the Terraform code. We then edit main.tf by right clicking on the file and selecting open. We insert the code below into main.tf. The code has a Terraform resource block to create an AWS S3 Bucket. The configuration also uses Terraform AWS datasources to obtain AWS region and AWS Account ID data which is used to form part of the bucket name. Finally, we use a backend block to configure Terraform to use an AWS S3 bucket to store Terraform state data. To save our changes we select File -> Save

Figure 10: Adding Terraform Code

Now let’s start creating Terraform Workflow using Amazon CodeCatalyst Terraform Community Action. Within your repository go to .codecatalyst/workflows directory and open the <workflowname.yaml> file.

Figure 11: Creating CodeCatalyst Workflow

The below code snippet is an example workflow definition with terraform plan and terraform apply. We will enter this into our workflow file, with the relevant configuration settings for our environment.

The workflow does the following:

• When a change is pushed to the main branch, a new workflow execution is triggered. This workflow carries a Terraform plan and subsequent apply operation.

  Name: terraform-action-workflow
  Compute:
    Type: EC2
    Fleet: Linux.x86-64.Large
  SchemaVersion: "1.0"
  Triggers:
    - Type: Push
      Branches:
        -  main
  Actions: 
    PlanTerraform:
      Identifier: codecatalyst-labs/provision-with-terraform-community@v1
      Environment:
        Name: dev 
        Connections:
          - Name: codecatalyst
            Role: CodeCatalystWorkflowDevelopmentRole # The IAM role to be used
      Inputs:
        Sources:
          - WorkflowSource
      Outputs:
        Artifacts:
          - Name: tfplan # generates a tfplan output artifact
            Files:
              - tfplan.out
      Configuration:
        AWSRegion: eu-west-2
        StateBucket: tfstate-bucket # The Terraform state S3 Bucket
        StateKey: terraform.tfstate # The Terraform state file
        StateKeyPrefix: states/ # The path to the state file (optional)
        StateTable: tfstate-table # The Dynamo DB database
        TerraformVersion: ‘1.5.1’ # The Terraform version to be used
        TerraformOperationMode: plan # The Terraform operation- can be plan or apply
    ApplyTerraform:
      Identifier: codecatalyst-labs/provision-with-terraform-community@v1
      DependsOn:
        - PlanTerraform
      Environment:
        Name: dev 
        Connections:
          - Name: codecatalyst
            Role: CodeCatalystWorkflowDevelopmentRole
      Inputs:
        Sources:
          - WorkflowSource
        Artifacts:
          - tfplan
      Configuration:
        AWSRegion: eu-west-2
        StateBucket: tfstate-bucket
        StateKey: terraform.tfstate
        StateKeyPrefix: states/
        StateTable: tfstate-table
        TerraformVersion: '1.5.1'
        TerraformOperationMode: apply

• Key configuration parameters are:
  • Environment.Name: The name of our CodeCatalyst Environment
  • Environment.Connections.Name: The name of the CodeCatalyst connection
  • Environment.Connections.Role: The IAM role used for the workflow
  • AWSRegion: The AWS region that hosts the Terraform state bucket
  • Environment.Name: The name of our CodeCatalyst Environment
  • Identifier: codecatalyst-labs/provision-with-terraform-community@v1
  • StateBucket: The Terraform state bucket
  • StateKey: The Terraform statefile e.g. terraform.tfstate
  • StateKeyPrefix: The folder location of the State file (optional)
  • StateTable: The DynamoDB State table
  • TerraformVersion: The version of Terraform to be installed
  • TerraformOperationMode: The operation mode for Terraform – this can be either ‘plan’ or ‘apply’

The workflow now contains CodeCatalyst action for Terraform Plan and Terraform Apply.

To save our changes we select File -> Save, we can then commit these to our git repository by typing the following at the terminal:

git add . && git commit -m ‘adding terraform workflow and main.tf’ && git push

The above command adds the workflow file and Terraform code to be tracked by git. It then commits the code and pushes the changes to CodeCatalyst git repository. As we have a branch trigger for main defined, this will trigger a run of the workflow. We can monitor the status of the workflow in the CodeCatalyst console by selecting CICD -> Workflows. Locate your workflow and click on Runs to view the status. You will be able to observe that the workflow has successfully completed and Amazon S3 bucket is created.

Figure 12: CodeCatalyst Workflow Status

Cleaning up

If you have been following along with this workflow, you should delete the resources that you have deployed to avoid further charges. The walkthrough will create an Amazon S3 bucket named <your-aws-account-id>-<your-aws-region>-terraform-sample-bucket in your AWS account. In the AWS Console > S3, locate the bucket that was created, then select and click Delete to remove the bucket.

Conclusion

In this post, we explained how you can easily get started deploying IaC to your AWS accounts with Amazon CodeCatalyst. We outlined how the Terraform Community Edition action can streamline the process of planning and applying Terraform configurations and how to create a workflow that can leverage this action. Get started with Amazon CodeCatalyst today.