Amazon ECS Multi-region Deployment with Amazon CodeCatalyst

Many AWS customers run their mission-critical workloads across multiple AWS regions to serve geographically dispersed customer base, meet disaster recovery objectives or address local laws and regulations. Amazon CodeCatalyst is a unified software development service designed to streamline and accelerate the process of building and delivering applications on AWS. It is an all-in-one platform for managing your entire development lifecycle, from planning and collaboration to continuous integration, deployment, and scaling. Amazon CodeCatalyst aims to boost developer productivity, ensure consistency, and improve the overall software development experience on AWS. By leveraging Amazon CodeCatalyst for multi-region deployments, AWS customers can ensure high availability and disaster recovery and comply with various regulatory requirements, all while improving their development and deployment process.

In this post, we will walk you through a solution which allows you to easily control updates to applications that are deployed across multiple AWS regions using Amazon CodeCatalyst.

Architecture

In this post, we are going to consider a containerized application running on Amazon Elastic Container Service (Amazon ECS) deployed in two different regions us-east-1 and us-west-2. We will walk you through how to configure an Amazon CodeCatalyst workflow to perform the deployment in stages, limiting the deployment scope to one region at a time (Figure 1).

Figure 1: Amazon ECS Multi Region deployment

Here are the high-level steps in the multi-region deployment process.

1. The developer makes updates to the application code base and pushes the code changes to the source repository hosted in Amazon CodeCatalyst
2. This code push invokes an Amazon CodeCatalyst workflow for the multi-region deployment. In this example, the workflow deploys changes to containerized application running on Amazon ECS in two regions.
3. The deployment to different regions happens in stages. In Step 3, the updates get deployed to region 1. This staged approach allows for initial testing and validation in one AWS region before proceeding.
4. Once the deployment (and any associated validation steps) is completed successfully in region 1, the workflow proceeds with deployment to the second AWS region.

Limiting the scope of each individual deployment limits the potential impact on customers from failed production deployments and prevents a multi-region impact.

Prerequisites

• You need access to an AWS account. If you don’t have one, you can create a new AWS account.
• Follow Amazon ECS Multi-Region Workshop to deploy a simple containerized application across two different AWS regions. Clone the repository and then follow steps to deploy the foundation, data and backend stack. Note the outputs from workshop-backend-main & workshop-backend-secondary, we’ll use them later on in this post.
• Follow the Amazon ECR user guide to create an Amazon Elastic Container Registry (Amazon ECR) repository named codecatalyst-ecs-image-repo.
• Create an Amazon CodeCatalyst space, with an empty Amazon CodeCatalyst project named codecatalyst-ecs-project and an Amazon CodeCatalyst environment called codecatalyst-ecs-environment. Associate your AWS account to the CodeCatalyst space. Follow the Amazon CodeCatalyst tutorial to set these up.
• An AWS Identity and Access Management (IAM) role in the Amazon CodeCatalyst space to provide Amazon CodeCatalyst service permissions to build and deploy applications. Note the name of this role as you’ll use it later in this post.
• Create an Amazon CodeCatalyst source repository titled ecs-multi-region-repo following the instructions in the documentation.
• Local installation of Visual Studio Code & Remote Development extension pack.

Walkthrough

Step 1: Create an Amazon CodeCatalyst Dev Environment

In this step, you will create an Amazon CodeCatalyst Dev Environment directly linked to your source repository ecs-multi-region-repo allowing you to work on your Amazon ECS Multi Region application code and configuration files.

• Open Amazon CodeCatalyst and navigate to your project
• In the left navigation pane, choose Code and then choose Source repositories
• Choose the source repository ecs-multi-region-repo for the Amazon ECS multi-region application
• Choose Create Dev Environment
• Choose Visual Studio Code  from the drop-down menu
• In Create Dev Environment and open with Visual Studio Code page (Figure 2), choose Create to create a Visual Studio Code development environment

Figure 2: Create Dev Environment in Amazon CodeCatalyst

• Choose Open in Visual Studio Code when prompted (Figure 3), this establishes a remote connection to Dev Environment from your local Visual Studio Code. Keep this window open as you will need it for Step 2

Figure 3: Open Dev Environment with Visual Studio Code

Step 2: Add Source files to Amazon CodeCatalyst source repository

In this step you will add the necessary source files source files to the Amazon CodeCatalyst repository you created in the pre-requisites, including the sample Amazon ECS multi-region application and the Amazon ECS task definition file.

• Inside the Visual Studio Code IDE, choose Terminal in the top menu.
• Select New Terminal or use an existing terminal window if you prefer.
• Clone the Github project inside your project folder by running the below commands in the terminal.

  git clone https://github.com/aws-samples/amazon-ecs-multi-region.git
  rm -rf amazon-ecs-multi-region/.git
  cp -r amazon-ecs-multi-region/ ecs-multi-region-repo
  cd ecs-multi-region-repo/

• You need to create an Amazon ECS task definition file for the sample application. Create a file named task.json inside the app folder. Paste the below contents into the task.json replacing placeholder <Account_ID> with your AWS Account ID and <ecsTaskExecutionRole> with your role from workshop-backend-main outputs.

  {
  "executionRoleArn":"arn:aws:iam:<Account_ID>:role/<ecsTaskExecutionRole>",
  "containerDefinitions": [
  {
  "name": "web",
  "image": "$REPOSITORY_URI:$IMAGE_TAG",
  "essential": true,
  "portMappings": [
  {
  "hostPort": 5000,
  "protocol": "tcp",
  "containerPort": 5000
  }
  ],
  "environment": [
  {
  "name": "DYNAMODB_TABLE_NAME",
  "value": "workshop-table"
  }
  ]
  }
  ],
  "requiresCompatibilities": [
  "FARGATE"
  ],
  "networkMode": "awsvpc",
  "cpu": "256",
  "memory": "512",
  "family": "CdkEcsInfraStackTaskDef"
  }

Commit the changes to the Amazon CodeCatalyst repository by issuing the following commands inside the Visual Studio Code IDE terminal window. You will need to update the <your_email> and <your_name> with your email and name.

git config user.email "<your_email>"
git config user.name "<your_name>"
git add .
git commit -m "Initial commit"
git push

In this example, we are using a single task definition file (task.json) which Amazon CodeCatalyst will use to render task definitions in both regions. But, if your workload requires different task definition files across different regions (e.g. region specific resource requirements, compliance requirements, environment specific configurations etc), you can create multiple task definition files in Amazon CodeCatalyst repository and configure RenderAmazonECStaskdefinition action for each regions with different task definition files.

Step 3: Create Amazon CodeCatalyst Workflow for multi-region deployment

Amazon CodeCatalyst workflow is an automated procedure that describes how to build, test, and deploy your code as part of a continuous integration and continuous delivery (CI/CD) system. A workflow defines a series of steps, or actions, to be executed during a workflow run. You can group actions into action groups to keep your workflow organized and configure dependencies between different groups.

• In the navigation pane, choose CI/CD, and then choose Workflows
• Choose Create workflow. Select ecs-multi-region-repo from the Source repository dropdown
• Choose main in the branch. Select Create (Figure 4). The workflow definition file appears in the Amazon CodeCatalyst console’s YAML editor

Figure 4: Create Workflow page in Amazon CodeCatalyst

• In the YAML editor, you will replace the default content with the below provided workflow definition. Replace <Account_ID> with your AWS account ID.
• Replace <EcsRegionNameMain>, <EcsClusterNameMain>, <EcsServiceNameMain>, <EcsRegionNameSecondary>, <EcsClusterNameSecondary>, <EcsServiceNameSecondary>. For values with “main” refer to output from workshop-backend-main, and values with “secondary” refer to output from workshop-backend-secondary.
  • Otherwise use your own Amazon ECS Region, Amazon ECS Cluster ARN, Amazon ECS Service Name values.
• Replace <CodeCatalyst-Dev-Admin-Role> with the Role Name from the pre-requisite

  Name: BuildAndDeployToECS
  SchemaVersion: "1.0"
  
  # Set automatic triggers on code push.
  Triggers:
    - Type: Push
      Branches:
        - main
  
  Actions:
    Build_application_Multi_Region:
          Identifier: aws/build@v1
          Inputs:
            Sources:
              - WorkflowSource
            Variables:
              - Name: region
                Value: <EcsRegionNameMain>
              - Name: registry
                Value: <Account_ID>.dkr.ecr.<EcsRegionNameMain>.amazonaws.com
              - Name: image
                Value: codecatalyst-ecs-image-repo
          Outputs:
            AutoDiscoverReports:
              Enabled: false
            Variables:
              - IMAGE
          Compute:
            Type: EC2
          Environment:
            Connections:
              - Role: <CodeCatalyst-Dev-Admin-Role>
                Name: "<Account_ID>"
            Name: codecatalyst-ecs-environment
          Configuration:
            Steps:
              - Run: export account=`aws sts get-caller-identity --output text | awk '{ print $1
                  }'`
              - Run: aws ecr get-login-password --region ${region} | docker login --username AWS
                  --password-stdin ${registry}
              - Run: docker build -t appimage app
              - Run: docker tag appimage ${registry}/${image}:${WorkflowSource.CommitId}
              - Run: docker push --all-tags ${registry}/${image}
              - Run: export IMAGE=${registry}/${image}:${WorkflowSource.CommitId}
    build-deploy-region-one:
      Actions:
        RenderAmazonECStaskdefinition_Region_One:
          Identifier: aws/ecs-render-task-definition@v1
          Configuration:
            image: ${Build_application_Multi_Region.IMAGE}
            container-name: web
            task-definition: app/task.json
          Outputs:
            Artifacts:
              - Name: TaskDefinitionOne
                Files:
                  - task-definition*
          DependsOn:
            - Build_application_Multi_Region
          Inputs:
            Sources:
              - WorkflowSource
        DeploytoAmazonECS_Region_One:
          Identifier: aws/ecs-deploy@v1
          Configuration:
            task-definition: /artifacts/build-deploy-region-one@DeploytoAmazonECS_Region_One/TaskDefinitionOne/${RenderAmazonECStaskdefinition_Region_One.task-definition}
            service: <EcsServiceNameMain>
            cluster: <EcsClusterNameMain>
            region: <EcsRegionNameMain>
          Compute:
            Type: EC2
            Fleet: Linux.x86-64.Large
          Environment:
            Connections:
              - Role: <CodeCatalyst-Dev-Admin-Role>
                Name: "<Account_ID>"
            Name: codecatalyst-ecs-environment
          DependsOn:
            - RenderAmazonECStaskdefinition_Region_One
          Inputs:
            Artifacts:
              - TaskDefinitionOne
            Sources:
              - WorkflowSource
    build-deploy-region-two:
      DependsOn:
        - build-deploy-region-one
      Actions:
        RenderAmazonECSTaskDefinition_Region_Two:
          # Identifies the action. Do not modify this value.
          Identifier: aws/ecs-render-task-definition@v1.0.4
          # Defines the action's properties.
          Configuration:
            image: ${Build_application_Multi_Region.IMAGE}
            container-name: web
            task-definition: app/task.json
          Outputs:
            Artifacts:
              - Name: TaskDefinitionTwo
                Files:
                  - task-definition*
          DependsOn:
            - Build_application_Multi_Region
          # Specifies the source and/or artifacts to pass to the action as input.
          Inputs:
            # Optional
            Sources:
              - WorkflowSource # This specifies that the action requires this Workflow as a source
        DeployToAmazonECS_Region_Two:
          Identifier: aws/ecs-deploy@v1.1.0 # Defines the action's properties.
          Configuration:
            task-definition: /artifacts/build-deploy-region-two@DeployToAmazonECS_Region_Two/TaskDefinitionTwo/${RenderAmazonECSTaskDefinition_Region_Two.task-definition}
            service: <EcsServiceNameSecondary>
            cluster: <EcsClusterNameSecondary>
            region: <EcsRegionNameSecondary>
          # Required; You can use an environment to access AWS resources.
          Environment:
            Connections:
              - Role: <CodeCatalyst-Dev-Admin-Role>
                Name: "<Account_ID>"
            Name: codecatalyst-ecs-environment
          DependsOn:
            - RenderAmazonECSTaskDefinition_Region_Two
          # Specifies the source and/or artifacts to pass to the action as input.
          Inputs:
            Artifacts:
              - TaskDefinitionTwo
            # Optional
            Sources:
              - WorkflowSource # This specifies that the action requires this Workflow as a source

Figure 5: Amazon CodeCatalyst Workflow Screen

The workflow above (Figure 5) does the following:

• Whenever code changes are pushed to the repository, a Build action is invoked automatically. The Build action builds a container image and pushes the image to the Amazon Elastic Container Registry (Amazon ECR) repository in the primary region. In this example, we are storing the container image only within the primary region. If you are implementing multi-region for disaster recovery, enable cross-region replication on Amazon ECR to automatically replicate images to repositories in other regions. You will also need to update the task definition files to reference the Amazon ECR repository in the same region where the task will run
• Once the Build stage is complete, the Amazon ECS task definition is updated with the new Amazon ECR repository image
• The DeployToECS action then deploys the new image to Amazon ECS in the first region
• Once the first action group execution succeeds, the Amazon CodeCatalyst workflow invokes second action group repeating the last two steps (Render Task Definition, Deploy) for the second region.

As you may have noticed, the build action is separated from the deployment actions in this example. This way, we are building the container image only once and deploying the same image across multiple regions. But, if you have specific build steps that are region-specific, you can include those actions in the region-specific action groups. This allows for customizations based on regional requirements while maintaining overall consistency.

To check the syntax and structure of your workflow definition:

• Choose the Validate button. It should add a green banner with “The workflow definition is valid” at the top
• Select Commit to add the workflow to the repository (Figure 6)

Figure 6: Commit workflow page in Amazon CodeCatalyst

The workflow file is stored in a ~/.codecatalyst/workflows/ folder in the root of your source repository. The file can have a .yml or .yaml extension.

Using the URL of the Application Load Balancer you noted from the pre-requisite from either of the two regions, add /healthcheck to load the health check page in your browser. You’ll to see the message in the health check page as shown in figure 7.

Figure 7: ECS Multi Region Application (US-West-1)

Step 4: Validate the setup

To validate the setup, you will make a small change to the Health check of the sample application.

• • Open Amazon CodeCatalyst dev environment (Visual Studio Code) that you created in Step 1.
  • Update your local copy of the repository. In the terminal run the below command

    git pull

  • Inside the Visual Studio Code IDE, open app.py present inside the app folder.
  • Inside healthcheck() method, on line 13, update the string from ok to ok v1
  • Commit the changes to the repository using the below commands:

    git add . 
    git commit -m “Updating health check static text”
    git push

  After the change is commit, the Amazon CodeCatalyst workflow should start running automatically. Once the Amazon CodeCatalyst workflow finishes execution, paste the Application Load Balancer URL for region and add /healthcheck to reach the check page. You will be able to see the updated message in the health check page as shown in figure 8 and 9.

  

  Figure 8: ECS Multi Region Application (US-East-1)

  

  Figure 9: ECS Multi Region Application (US-West-1)

  Considerations for multi-region deployments

  In this post, we considered a deployment scenario across two regions. Many organizations have workload running across many regions, serving customers across the globe. The Amazon CodeCatalyst workflow, that we created in this post, can be extended to more than two regions.

  Amazon CodeCatalyst allows fine-grained control for progressive wave-based deployments across multiple regions. This is achieved by using multiple action groups and sequencing those action groups using dependencies in the Amazon CodeCatalyst workflow. For example, in the workflow discussed in Step 3, you defined two action groups build-deploy-region-one and build-deploy-region-two. We setup build-deploy-region-two  to depend on build-deploy-region-one using DependsOn: property, so that the deployment to the second region starts only after the completion of the first region. This approach allows for staggered deployments, mitigating risks by preventing issues in one AWS region from impacting others.

  For workloads spanning multiple regions, the same staggering deployment approach can be extended with more action groups. Each action group can contain a list of regions to deploy to in parallel. Dependencies between action groups ensures the deployment happens sequentially. Below is a high-level architecture (Figure 10) of the setup of 3-stage deployment process for a workload running across 6 regions.

  
  Figure 10: Staggered Deployment architecture

  Cleanup

  If you have been following along with the post, you should delete the resources you deployed so you do not continue to incur charges.

  • Manually delete Amazon CodeCatalyst dev environment, source repository and project from your CodeCatalyst Space.
  • Clean up resources created with the CDK

  cdk destroy workshop-backend-secondary
  cdk destroy workshop-backend-main
  cdk destroy workshop-data
  cdk destroy workshop-foundation-secondary
  cdk destroy workshop-foundation-main

  Conclusion

  In conclusion, we demonstrated how you can setup multi-region deployments for Amazon ECS workloads using Amazon CodeCatalyst workflows. We showed how to configure the Amazon CodeCatalyst workflow to deploy to one region at a time, allowing for validation before proceeding to additional regions. The pattern can be extended to more than two AWS regions using additional action groups and dependencies. This solution addresses key challenges in multi-region deployments like maintaining consistency while ensuring high availability. Learn more about multi region in AWS Multi-Region Fundamentals Whitepaper