Replicate AWS CodeCommit Repositories between Regions using AWS Fargate

Thanks to Raja Mani, AWS Solutions Architect, for this great blog.

—

In this blog post, I’ll walk you through the steps for setting up continuous replication of an AWS CodeCommit repository from one AWS region to another AWS region using a serverless architecture. CodeCommit is a fully-managed, highly scalable source control service that stores anything from source code to binaries. It works seamlessly with your existing Git tools and eliminates the need to operate your own source control system. Replicating an AWS CodeCommit repository from one AWS region to another AWS region enables you to achieve lower latency pulls for global developers. This same approach can also be used to automatically back up repositories currently hosted on other services (for example, GitHub or BitBucket) to AWS CodeCommit.

This solution uses AWS Lambda and AWS Fargate for continuous replication. Benefits of this approach include:

The replication process can be easily setup to trigger based on events, such as commits made to the repository.
Setting up a serverless architecture means you don’t need to provision, maintain, or administer servers.
You can incorporate this solution into your own DevOps pipeline. For more information, see the blog Invoke an AWS Lambda function in a pipeline in AWS CodePipeline.

Note: AWS Fargate has a limitation of 10 GB for storage and is available in US East (N. Virginia) region. A similar solution that uses Amazon EC2 instances to replicate the repositories on a schedule was published in a previous blog and can be used if your repository does not meet these conditions.

Replication using Fargate

As you follow this blog post, you’ll set up an architecture that looks like this:

Any change in the AWS CodeCommit repository will trigger a Lambda function. The Lambda function will call the Fargate task that replicates the repository using a Git command line tool.

Let us assume a user wants to replicate a repository (Source) from US East (N. Virginia/us-east-1) region to a repository (Destination) in US West (Oregon/us-west-2) region. I’ll walk you through the steps for it:

Prerequisites

Create an AWS Service IAM role for Amazon EC2 that has permission for both source and destination repositories, IAM CreateRole, AttachRolePolicy and Amazon ECR privileges. Here is the EC2 role policy I used:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": " ",
            "Effect": "Allow",
            "Action": [
                "codecommit:*",
                "ecr:*",
                "iam:CreateRole",
                "iam:AttachRolePolicy"
            ],
            "Resource": "*"
        }
    ]
}

You need a Docker environment to build this solution. You can launch an EC2 instance and install Docker (or) you can use AWS Cloud9 that comes with Docker and Git preinstalled. I used an EC2 instance and installed Docker in it. Use the IAM role created in the previous step when creating the EC2 instance. I am going to refer this environment as “Docker Environment” in the following steps.
You need to install the AWS CLI on the Docker environment. For AWS CLI installation, refer this page.
You need to install Git, including a Git command line on the Docker environment.

Step 1: Create the Docker image

To create the Docker image, first it needs a Dockerfile. A Dockerfile is a manifest that describes the base image to use for your Docker image and what you want installed and running on it. For more information about Dockerfiles, go to the Dockerfile Reference.

1. Choose a directory in the Docker environment and perform the following steps in that directory. I used /home/ec2-user directory to perform the following steps.

2. Clone the AWS CodeCommit repository in the Docker environment. Open the terminal to the Docker environment and run the following commands to clone your source AWS CodeCommit repository (I ran the commands from /home/ec2-user directory):

$ git config --global credential.helper '!aws codecommit credential-helper $@'
$ git config --global credential.UseHttpPath true
$ git clone --mirror https://git-codecommit.us-east-1.amazonaws.com/v1/repos/Source LocalRepository
$ cd LocalRepository
$ git remote set-url --push origin https://git-codecommit.us-east2.amazonaws.com/v1/repos/Destination

Note: Change the URL marked in red to your source and destination repository URL.

3. Create a file called Dockerfile (case sensitive) with the following content (I created it in /home/ec2-user directory):

# Pull the Amazon Linux latest base image
FROM amazonlinux:latest

#Install aws-cli and git command line tools
RUN yum -y install unzip aws-cli
RUN yum -y install git

WORKDIR /home/ec2-user
RUN mkdir LocalRepository

WORKDIR /home/ec2-user/LocalRepository

#Copy Cloned CodeCommit repository to Docker container
COPY ./LocalRepository /home/ec2-user/LocalRepository
#Copy shell script that does the replication

COPY ./repl_repository.bash /home/ec2-user/LocalRepository

RUN chmod ugo+rwx /home/ec2-user/LocalRepository/repl_repository.bash
WORKDIR /home/ec2-user/LocalRepository

#Call this script when Docker starts the container 
ENTRYPOINT ["/home/ec2-user/LocalRepository/repl_repository.bash"]

4. Copy the following shell script into a file called repl_repository.bash to the DockerFile directory location in the Docker environment (I created it in /home/ec2-user directory)

#!/bin/bash

git config --global credential.helper '!aws codecommit credential-helper $@'
git config --global credential.UseHttpPath true

git fetch -p origin
git push --mirror

5. Your directory structure will look like this after completing the above steps:

-rw-r--r--   1      0    Apr  3 13:21 Dockerfile
drwxr-xr-x   2     68    Apr  3 13:21 LocalRepository
-rw-r--r--   1      0    Apr  3 13:21 repl_repository.bash

6. Verify whether the replication is working by running the repl_repository.bash script from the LocalRepository directory. Go to LocalRepository directory and run this command: . ../repl_repository.bash
If it is successful, you will get the “Everything up-to-date” at the last line of the result like this:

$ . ../repl_repository.bash
Everything up-to-date

Step 2: Build the Docker Image

1. Build the Docker image by running this command from the directory where you created the DockerFile in the Docker environment in the previous step (I ran it from /home/ec2-user directory):

$ docker build . –t ccrepl

Output: It installs various packages and set environment variables as part of steps 1 to 3 from the Dockerfile. The steps 4 to 11 from the Dockerfile should produce an output similar to the following:

Step 4/11 : WORKDIR /home/ec2-user
---> 37a5113bd2ce
Removing intermediate container ed89954b9a4f
Step 5/11 : RUN mkdir LocalRepository
---> Running in b9e4fab2b264
---> 816b553261c9
Removing intermediate container b9e4fab2b264
Step 6/11 : WORKDIR /home/ec2-user
---> 60ad564a70be
Removing intermediate container 0897cfb7dd5d
Step 7/11 : COPY ./LocalRepository/ /home/ec2-user/LocalRepository/
---> 03420e4e7d0a
Step 8/11 : COPY ./repl_repository.bash /home/ec2-user/LocalRepository
---> 7eea3f045f38
Step 9/11 : RUN chmod ugo+rwx /home/ec2-user/LocalRepository/repl_repository.bash
---> Running in 7ef77a0ad886
---> 2ff00242c190
Removing intermediate container 7ef77a0ad886
Step 10/11 : WORKDIR /home/ec2-user/LocalRepository
---> 41f2aa473cf8
Removing intermediate container 9f233487943b
Step 11/11 : ENTRYPOINT /home/ec2-user/LocalRepository/repl_repository.bash
---> Running in 0aaff1cc2e29
---> 2066cf6a9c7d
Removing intermediate container 0aaff1cc2e29
Successfully built 2066cf6a9c7d
Successfully tagged ccrepl:latest

2. Run the following command to verify that the image was created successfully. It will display “Everything up-to-date” at the end if it is successful.

[ec2-user@ip-172-1-1-210 LocalRepository]$ docker run ccrepl
Everything up-to-date

Step 3: Push the Docker Image to Amazon Elastic Container Registry (ECR)

Perform the following steps in the Docker Environment.

1. Run the AWS CLI configure command and set default region as your source repository region (I used us-east-1).

$ aws configure set default.region <Source Repository Region>

2. Create an Amazon ECR repository using this command to store your ccrepl image (Note the repositoryUri in the output):

$ aws ecr create-repository --repository-name ccrepl

Output:

3. Tag the ccrepl image with repositoryUri value from the previous step using this command:

$ docker tag ccrepl:latest aws_account_id.dkr.ecr.us-east-1.amazonaws.com/ccrepl

4. Get the Docker login credentials using the following command:

$ aws ecr get-login --no-include-email

5. Run the Docker login command returned from the previous step. If the command is successful, you will get a message “Login Succeeded”

6. Push the docker image to Amazon ECR with the repositoryUri from step 2 using this command:

$ docker push aws_account_id.dkr.ecr.us-east-1.amazonaws.com/ccrepl

Step 4: Create Fargate Task Definition and cluster

1. Create a text file called trustpolicyforecs.json with the following content in the Docker Environment:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ecs-tasks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

2. Create a role called AccessRoleForCCfromFG using the following command in the DockerEnvironment:

$ aws iam create-role --role-name AccessRoleForCCfromFG --assume-role-policy-document file://trustpolicyforecs.json

3. Assign CodeCommit service full access to the above role using the following command in the DockerEnvironment:

$ aws iam attach-role-policy --policy-arn arn:aws:iam::aws:policy/AWSCodeCommitFullAccess --role-name AccessRoleForCCfromFG

4. In the Amazon ECS Console, choose Repositories and select the ccrepl repository that was created in the previous step. Copy the Repository URI.

5. In the Amazon ECS Console, choose Task Definitions and click Create New Task Definition.

6. Select launch type compatibility as FARGATE and click Next Step.

7. In the create task definition screen, do the following:

In Task Definition Name, type ccrepl
In Task Role, choose AccessRoleForCCfromFG
In Task Memory, choose 2GB
In Task CPU, choose 1 vCPU
Click Add Container under Container Definitions in the same screen. In the Add Container screen, do the following:
- Enter Container name as ccreplcont
- Enter Image URL copied from step 4
- Enter Memory Limits as 128 and click Add.

Note: Select TaskExecutionRole as “ecsTaskExecutionRole” if it already exists. If not, select create new role and it will create “ecsTaskExecutionRole” for you.

8. Click the Create button in the task definition screen to create the task. It will successfully create the task, execution role and AWS CloudWatch Log groups.

9. In the Amazon ECS Console, click Clusters and create cluster. Select template as “Networking only, Powered by AWS Fargate” and click next step.

10. Enter cluster name as ccreplcluster and click create.

Step 5: Create the Lambda Function

In this section, I used Amazon Elastic Container Service (ECS) run task API from Lambda to invoke the Fargate task.

1. In the IAM Console, create a new role called ECSLambdaRole with the permissions to AWS CodeCommit, Amazon ECS as well as pass roles privileges needed to run the ECS task. Your statement should look similar to the following (replace <your account id>):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "codecommit:*"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ecs:RunTask"
            ],
            "Resource": [
                "*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:PassRole"
            ],
            "Resource": [
                "arn:aws:iam::<your account id>:role/ecsTaskExecutionRole",
                "arn:aws:iam::<your account id>:role/AccessRoleForCCfromFG"
            ]
        }
    ]
}

2. In AWS management console, select VPC service and click subnets in the left navigation screen. Note down the Subnet IDs that you want to run the Fargate task in.

3. Create a new Lambda Node.js function called FargateTaskExecutionFunc and assign the role ECSLambdaRole with the following content:

Note: Replace subnets values (marked in red color) with the subnet IDs you identified as the subnets you wanted to run the Fargate task on in Step 2 of this section.

var AWS = require('aws-sdk');
var ecs = new AWS.ECS();
exports.handler = (event, context, callback) => {
    var params = {
        cluster: "ccreplcluster", 
        launchType: 'FARGATE',
        taskDefinition: "ccrepl:1",
        count: 1,
        platformVersion: 'LATEST',
        networkConfiguration: {
            awsvpcConfiguration: {
            subnets: [ /* required */
                '<subnet ID1>',
	        '<subnet ID2>'
              ],
            assignPublicIp: 'ENABLED',
            }      
        }
    };
    ecs.runTask(params, function(err, data) {
        if (err) console.log(err, err.stack); // an error occurred
         else     console.log(data);           // successful response
    });
};

Step 6: Assign Lambda to CodeCommit Trigger

1. In the Lambda Console, click FargateTaskExecutionFunc under functions.

2. Under Add triggers in the Designer, select CodeCommit

3. In the Configure triggers screen, do the following:

Enter Repository name as Source (your source repository name)
Enter trigger name as LambdaTrigger
Leave the Events as “All repository events”
Leave the Branch names as “All branches”
Click Add button
Click Save button to save the changes

Step 6: Verification

To test the application, make a commit and push the changes to the source repository in AWS CodeCommit. That should automatically trigger the Lambda function and replicate the changes in the destination repository. You can verify this by checking CloudWatch Logs for Lambda and ECS, or simply going to the destination repository and verifying the change appears.

Conclusion

Congratulations! You have successfully configured repository replication of an AWS CodeCommit repository using AWS Lambda and AWS Fargate. You can use this technique in a deployment pipeline. You can also tweak the trigger configuration in AWS CodeCommit to call the Lambda function in response to any supported trigger event in AWS CodeCommit.

For more information about CodeCommit, see the AWS CodeCommit documentation.

AWS DevOps & Developer Productivity Blog