Containers

Create a CI/CD pipeline for Amazon ECS with GitHub Actions and AWS CodeBuild Tests

Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service that makes it easy to operate containerized workloads at scale.  It also integrates with other core AWS services, such as Amazon Route 53, AWS Identity and Access Management (IAM), and Amazon CloudWatch.  Establishing an effective and efficient CI/CD pipeline is critical for containerized applications, regardless of the platform you are using to manage your containers.

In this post, we explore and demo how organizations using GitHub as a source code repository can use GitHub Actions to build a complete CI/CD pipeline for applications deployed on Amazon ECS.  This post also illustrates how AWS CodeBuild can be used with GitHub Actions to execute application tests as part of a complete CI/CD pipeline.

If you are new to Amazon ECS or AWS CodeBuild, check out Getting Started with Amazon ECS and Developer Tools on AWS.

GitHub Actions

For organizations using GitHub as a source code repository, GitHub Actions provide a way to implement complex CI/CD functionality directly in GitHub by initiating a workflow on any GitHub event.

A GitHub Action is an individual unit of functionality that can be combined with other GitHub Actions to create workflows, which are triggered in response to certain GitHub events, for example, pull, push, or commit.  Workflows run inside managed environments on GitHub-hosted servers.

To support CI/CD operations for applications running on Amazon ECS, AWS has open-sourced the following JavaScript-based GitHub Actions at github.com/aws-actions:

By constructing a workflow using custom GitHub Actions, in addition to those provided by AWS and GitHub, we can easily deliver CI/CD functionality for our Amazon ECS application directly from GitHub.

Testing with AWS CodeBuild

AWS CodeBuild supports webhooks when GitHub is used as the source code repository.  This allows CodeBuild to work with source code stored in GitHub, and use webhooks to trigger a build of the source code every time a code change is pushed to the GitHub repository.

You can use CodeBuild for more than just compiling your application code or building an application container image. With CodeBuild, you can easily run a variety of tests against your code, such as unit tests, static code analysis, and integration tests. When building a CI/CD pipeline, including a “test” stage is critical to maintaining application quality standards.

In this post, we don’t use CodeBuild as the “build” action of the CI/CD pipeline. Instead, we use CodeBuild as an environment to execute tests. Using CodeBuild as a testing platform can simplify running your tests for applications that are deployed on AWS.  In addition to providing a managed environment that easily integrates with other AWS services, you can execute a CodeBuild project to perform integration tests with components that are not reachable outside of AWS.  You can also use CodeBuild to Generate Test Reports from the results of unit and integration tests.  For the example in this post, we use CodeBuild to perform simple unit tests and report the status to GitHub.

Example architecture

The following diagram shows the high-level architecture that we implement.  This architecture represents a complete CI/CD pipeline that uses a GitHub workflow to automatically coordinate building, testing, and deploying an application to ECS for every commit to the repository.  This GitHub workflow uses the AWS open-source GitHub Actions to coordinate build and deploy tasks, and uses CodeBuild to execute application tests.  We also introduce a custom GitHub Action to this pipeline to evaluate the status of CodeBuild tests.

CI/CD Pipeline for ECS Applications with GitHub Actions and CodeBuild

CI/CD Pipeline for ECS Applications with GitHub Actions and CodeBuild

Our GitHub repository consists of a simple web application and accompanying infrastructure files.  The goal of our CI/CD pipeline is to execute unit tests, build a container image, upload the container image to ECR, and update an ECS Task Definition for every commit to the GitHub repository.

Creating a GitHub Repository

To get started creating the components necessary to build our example architecture, we need a GitHub repository.  If you are unfamiliar with this process, check out Creating a Repository on GitHub to get stared.  For this example, the following files should be placed in the GitHub repository:

  • app.py: a simple Flask web application.  This application contains one endpoint that reverses and returns the requested URI
"""Main application file"""
from flask import Flask
app = Flask(__name__)

@app.route('/<random_string>')
def returnBackwardsString(random_string):
    """Reverse and return the provided URI"""
    return "".join(reversed(random_string))

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)
  • app_test.py: a unit test file for app.py to ensure that the string is reversed correctly
"""Unit test file for app.py"""
from app import returnBackwardsString
import unittest

class TestApp(unittest.TestCase):
    """Unit tests defined for app.py"""

    def test_return_backwards_string(self):
        """Test return backwards simple string"""
        random_string = "This is my test string"
        random_string_reversed = "gnirts tset ym si sihT"
        self.assertEqual(random_string_reversed, returnBackwardsString(random_string))

if __name__ == "__main__":
    unittest.main()
  • requirements.txt: dependencies for app.py 

flask==1.0.2

  • buildspec.yml: instructions for AWS CodeBuild to run unit tests.  Refer to Build Specification Reference for more information about the capabilities and syntax available for buildspec files
version: 0.2
phases:
  install:
    runtime-versions:
      python: 3.8
  pre_build:
    commands:
      - pip install -r requirements.txt
      - python app_test.py
  • Dockerfile: instructions for building the application container image

FROM python:3
# Set application working directory
WORKDIR /usr/src/app
# Install requirements
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
# Install application
COPY app.py ./
# Run application
CMD python app.py

  • task-definition.json: specifications for an ECS Task Definition. Note: in this file, replace the placeholder value with your AWS Account ID
{
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "inferenceAccelerators": [],
    "containerDefinitions": [
        {
            "name": "ecs-devops-sandbox",
            "image": "ecs-devops-sandbox-repository:00000",
            "resourceRequirements": null,
            "essential": true,
            "portMappings": [
                {
                    "containerPort": "8080",
                    "protocol": "tcp"
                }
                
            ]
        }
    ],
    "volumes": [],
    "networkMode": "awsvpc",
    "memory": "512",
    "cpu": "256",
    "executionRoleArn": "arn:aws:iam::<YOUR_AWS_ACCOUNT_ID>:role/ecs-devops-sandbox-execution-role",
    "family": "ecs-devops-sandbox-task-definition",
    "taskRoleArn": "",
    "placementConstraints": []
}

The end result of this step should be a GitHub repository containing all application, test, and deployment files.

Example GitHub Repository

Example GitHub Repository

Creating ECS Infrastructure

To build the architecture described in the solution overview, you will need the following ECS components:

  • ECR Repository: store versioned application container images
  • ECS Cluster: provides compute power to run application container instances
  • ECS Task Definition: specifies application container image version and environment considerations
  • ECS Service: specifies how task definition will be deployed onto underlying compute resources

To build this infrastructure, we will be use the AWS Cloud Development Kit (CDK). If you are new to the CDK, see Getting Started with the AWS CDK. In this post, we will be using the CDK with Python 3.7.

To create our application infrastructure:

1. Configure your AWS CLI with an IAM user that has permissions to create the resources (VPC, ECS, ECR, IAM Role) described in the template below. Refer to Managing IAM Permissions for more details on creating custom users and policies.

2. Run the following commands to initialize your CDK project

# Create a project directory
mkdir ecs-devops-sandbox-cdk
# Enter the directory
cd ecs-devops-sandbox-cdk
# Use the CDK CLI to initiate a Python CDK project
cdk init --language python 
# Activate your Python virtual environment
# NOTE: For Windows users, replace with ".env\Scripts\activate.bat"
source .env/bin/activate
# Install CDK Python general dependencies
pip install -r requirements.txt
# Install CDK Python ECS dependencies
pip install aws_cdk.aws_ec2 aws_cdk.aws_ecs aws_cdk.aws_ecr aws_cdk.aws_iam

3. Replace the contents of the file ecs_devops_sandbox_cdk/ecs_devops_sandbox_cdk_stack.py (automatically created by the CDK) with the code below

"""AWS CDK module to create ECS infrastructure"""
from aws_cdk import (core, aws_ecs as ecs, aws_ecr as ecr, aws_ec2 as ec2, aws_iam as iam)

class EcsDevopsSandboxCdkStack(core.Stack):

    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        # Create the ECR Repository
        ecr_repository = ecr.Repository(self,
                                        "ecs-devops-sandbox-repository",
                                        repository_name="ecs-devops-sandbox-repository")

        # Create the ECS Cluster (and VPC)
        vpc = ec2.Vpc(self,
                      "ecs-devops-sandbox-vpc",
                      max_azs=3)
        cluster = ecs.Cluster(self,
                              "ecs-devops-sandbox-cluster",
                              cluster_name="ecs-devops-sandbox-cluster",
                              vpc=vpc)

        # Create the ECS Task Definition with placeholder container (and named Task Execution IAM Role)
        execution_role = iam.Role(self,
                                  "ecs-devops-sandbox-execution-role",
                                  assumed_by=iam.ServicePrincipal("ecs-tasks.amazonaws.com"),
                                  role_name="ecs-devops-sandbox-execution-role")
        execution_role.add_to_policy(iam.PolicyStatement(
            effect=iam.Effect.ALLOW,
            resources=["*"],
            actions=[
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
                ]
        ))
        task_definition = ecs.FargateTaskDefinition(self,
                                                    "ecs-devops-sandbox-task-definition",
                                                    execution_role=execution_role,
                                                    family="ecs-devops-sandbox-task-definition")
        container = task_definition.add_container(
            "ecs-devops-sandbox",
            image=ecs.ContainerImage.from_registry("amazon/amazon-ecs-sample")
        )

        # Create the ECS Service
        service = ecs.FargateService(self,
                                     "ecs-devops-sandbox-service",
                                     cluster=cluster,
                                     task_definition=task_definition,
                                     service_name="ecs-devops-sandbox-service")

4. Run the following command from the root directory of your CDK project

# Create the CloudFormation stack
cdk deploy

This code uses the CDK to build and deploy a CloudFormation stack that contains the ECS infrastructure required for our application.  In the code above, we are initially specifying the Task Definition to run with an example container from a public AWS sample registry.  This sample container is replaced with our application container when our CI/CD pipeline updates the Task Definition.  We are using the container from the sample registry to allow the Service to stabilize before any application container images are added to our ECR repository.

The end result of this step should be a Cluster, ecs-devops-sandbox-cluster, running a Service, ecs-devops-sandbox-service, that consists of a Task Definition, ecs-devops-sandbox-task-definition.

Create a CodeBuild Project

CodeBuild is used to execute our application tests and provide the status of these tests to GitHub.  If the tests do not pass, CodeBuild marks the build as Failed and this status is reported to GitHub.  In our architecture, we rely on webhooks that are automatically created by CodeBuild to trigger a build of our application code every time there is a commit to the master branch of the GitHub repository.

To create a CodeBuild Project with GitHub integration:

1.     Navigate to AWS CodeBuild and select Create build project

2.     Under Project Configuration, for Project name, enter ecs-devops-sandbox 

Example CodeBuild Project Configuration

Example CodeBuild Project Configuration

3.     Under Source, for Source Provider, select GitHub

4.     Under Source, for Repository, select Connect using OAuth and select Connect to GitHub

5.     At the pop-out window, log into the GitHub account that owns the repository you wish to use

6.     Under Source, for Repository, select Repository in my GitHub account

7.     Under Source, for GitHub repository, select the repository you configured earlier

8.     Under Source – Additional configuration, for Build Status – optional, check the box

Example CodeBuild Source Configuration

Example CodeBuild Source Configuration

9.     Under Primary source webhook events, for Webhook – optional, check the box

Example CodeBuild Webhook Configuration

Example CodeBuild Webhook Configuration

10. Under Environment, for Environment image, select Managed image

11. Under Environment, for Operating system, select Ubuntu

12. Under Environment, for Runtime, select Standard

13. Under Environment, for Image, select aws/AWS CodeBuild/standard:3.0

Example CodeBuild Environment Configuration

Example CodeBuild Environment Configuration

14. Leave remaining default values and select Create build project

Create a GitHub Workflow

AWS has provided a starter GitHub workflow that takes advantage of the AWS open-source GitHub Actions to build and deploy containers on ECS for each commit to master branch of the repository.

To add the starter GitHub workflow to your GitHub repository:

1.     Under the Actions tab, select New workflow

Example Add GitHub Workflow

Example Add GitHub Workflow

2.     Search for Deploy to Amazon ECS and select Set up this workflow

Example Add AWS ECS Starter Workflow

Example Add AWS ECS Starter Workflow

3.     Follow the instructions outlined in the starter workflow file for using this workflow with your application.  Note that we have already created an ECR repository, ECS task definition, ECS cluster, and ECS service

Example GitHub Workflow for ECS

Example GitHub Workflow for ECS

In practice, we want our GitHub workflow to build and deploy a new container to ECS only if the tests executed in CodeBuild were successful.  The CodeBuild project we created earlier reports the success or failure of its build execution to GitHub for every commit.  If the CodeBuild project fails for a commit, GitHub marks this commit with a status of failure.  If the CodeBuild project has not yet reported its success or failure, GitHub marks this commit with a status of pending (learn more about GitHub commit statuses).

In this example, we are executing the GitHub workflow and the CodeBuild project in parallel on each commit to the master branch.  To ensure that our workflow does not deploy a commit that does not pass the required tests, we introduce an additional step into the starter GitHub workflow to check the status of the commit.

The code below shows the complete GitHub workflow (based on the starter workflow provided by AWS) that includes a new GitHub Action, “Check commit status.”  This new GitHub Action uses the environment and variables provisioned by GitHub to query the status of the current commit from GitHub APIs.  If the current commit is pending, we check again after 10 seconds.  If the commit status is failure, i.e. the CodeBuild tests did not pass, then the GitHub workflow fails.

on:
  push:
    branches:
      - master

name: Deploy to Amazon ECS

jobs:
  deploy:
    name: Deploy
    runs-on: ubuntu-latest

    steps:
    - name: Check commit status
      id: commit-status
      run: |
        # Check the status of the Git commit
        CURRENT_STATUS=$(curl --url https://api.github.com/repos/${{ github.repository }}/commits/${{ github.sha }}/status --header 'authorization: Bearer ${{ secrets.GITHUB_TOKEN }}' | jq -r '.state');
        echo "Current status is: $CURRENT_STATUS"
        while [ "${CURRENT_STATUS^^}" = "PENDING" ]; 
          do sleep 10; 
          CURRENT_STATUS=$(curl --url https://api.github.com/repos/${{ github.repository }}/commits/${{ github.sha }}/status --header 'authorization: Bearer ${{ secrets.GITHUB_TOKEN }}' | jq -r '.state');
        done;
        echo "Current status is: $CURRENT_STATUS"
        if [ "${CURRENT_STATUS^^}" = "FAILURE" ]; 
          then echo "Commit status failed. Canceling execution"; 
          exit 1; 
        fi
        
    - name: Checkout
      uses: actions/checkout@v1

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: <YOUR_AWS_REGION>

    - name: Login to Amazon ECR
      id: login-ecr
      uses: aws-actions/amazon-ecr-login@v1

    - name: Build, tag, and push image to Amazon ECR
      id: build-image
      env:
        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
        ECR_REPOSITORY: ecs-devops-sandbox-repository
        IMAGE_TAG: ${{ github.sha }}
      run: |
        # Build a docker container and
        # push it to ECR so that it can
        # be deployed to ECS.
        docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
        echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"
        
    - name: Fill in the new image ID in the Amazon ECS task definition
      id: task-def
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        task-definition: task-definition.json
        container-name: ecs-devops-sandbox
        image: ${{ steps.build-image.outputs.image }}

    - name: Deploy Amazon ECS task definition
      uses: aws-actions/amazon-ecs-deploy-task-definition@v1
      with:
        task-definition: ${{ steps.task-def.outputs.task-definition }}
        service: ecs-devops-sandbox-service
        cluster: ecs-devops-sandbox-cluster
        wait-for-service-stability: true

Results

With the AWS CodeBuild project and GitHub workflow in place, we can add new features to our application that automatically deploy to Amazon ECS if all tests are passed.

This example shows a modification to the application code (app.py) that does not pass the unit test.

"""
Main application file
"""
from flask import Flask
app = Flask(__name__)

@app.route('/<random_string>')
def returnBackwardsString(random_string):
    """Reverse and return the provided URI"""
    return "Breaking the unit test"

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

When this code is checked into the repository, we can see that CodeBuild is automatically invoked via webhook and executes the application tests.

Example Failed CodeBuild Test Execution

Example Failed CodeBuild Test Execution

CodeBuild then reports the unsuccessful test execution to GitHub.

Example Failed GitHub Commit Status

Example Failed GitHub Commit Status

Because our GitHub workflow detected the status of the commit to be failure, the workflow did not proceed past the “Check commit status” step.

Example Failed GitHub Workflow

Example Failed GitHub Workflow

This next example shows a modification to the application code (app.py) that introduces some additional logging statements, but still passes the unit test.

"""
Main application file
"""
from flask import Flask
import logging
app = Flask(__name__)

# Initialize Logger
LOGGER = logging.getLogger()
LOGGER.setLevel(logging.INFO)

@app.route('/<random_string>')
def returnBackwardsString(random_string):
    """Reverse and return the provided URI"""
    LOGGER.info('Received a message: %s', random_string)
    return "".join(reversed(random_string))

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

When this code is checked into the repository, we can see that CodeBuild is again invoked via webhook, and successfully executes the application tests.

Example Successful CodeBuild Test Execution

Example Successful CodeBuild Test Execution

CodeBuild then reports its successful test execution to GitHub.

Example Successful GitHub Commit Status

Example Successful GitHub Commit Status

The GitHub workflow detects the status of the commit to be success, and the remaining steps in the workflow successfully execute the deployment of the application to ECS.

Example Successful GitHub Workflow

Example Successful GitHub Workflow

Finally, we can see that the ECS Task Definition has been updated to reflect the new application container image.  Note that the Git commit hash is used as the container image tag.

Example ECS Task Definition with Git Commit Hash

Example ECS Task Definition with Git Commit Hash

Conclusion

In this post, you saw how GitHub Actions can be used to support applications running on Amazon ECS.  For organizations seeking to use GitHub Actions as a part of their DevOps infrastructure, combining GitHub Actions with other CI/CD services (like AWS CodeBuild) can be a simple and powerful way to deliver feature-rich CI/CD pipelines.  For more information about GitHub Actions, see GitHub Actions Features.

TAGS: , , , , ,
Bryant Bost

Bryant Bost

Bryant Bost is an Application Consultant for AWS Professional Services, based out of Washington, DC. As a consultant, he supports customers with architecting, developing, and operating new applications, as well as migrating existing applications to AWS. In addition to web application development, Bryant specializes in serverless and container architectures, and has authored several blog posts on these topics.