AWS DevOps Blog

Use AWS CloudFormation to Automate the Creation of an S3 Bucket with Cross-Region Replication Enabled

At the request of many of our customers, in this blog post, we will discuss how to use AWS CloudFormation to create an S3 bucket with cross-region replication enabled. We’ve included a CloudFormation template with this post that uses an AWS Lambda-backed custom resource to create source and destination buckets.

What is S3 cross-region replication?

Cross-region replication is a bucket-level feature that enables automatic, asynchronous copying of objects across buckets in different AWS regions. You can create two buckets in two different regions and use the ReplicationConfiguration property to replicate the objects from one bucket to the other. For example, you can have a bucket in us-east-1 and replicate the bucket objects to a bucket in us-west-2.

For more information, see What Is and Is Not Replicated in Cross-Region Replication.

Challenge

When you enable cross-region replication, the replicated objects will be stored in only one destination (an S3 bucket). The destination bucket must already exist and it must be in an AWS region different from your source bucket.

Using CloudFormation, you cannot create the destination bucket in a region different from the region in which you are creating your stack. To create the destination bucket, you can:

Solution overview

The CloudFormation template provided with this post uses an AWS Lambda-backed custom resource to create an S3 destination bucket in one region and a source S3 bucket in the same region as the CloudFormation endpoint.

Note: In this scenario, CloudFormation is not aware of the destination bucket created by AWS Lambda. For this reason, CloudFormation will not delete this resource when the stack is deleted.

How does it work?

Launch the stack and provide the following custom values to the CloudFormation template. These (user input) values will be passed as parameters to the template.

  • OriginalBucketName
  • ReplicationBucketName
  • ReplicationRegion (different from the source region from which you are launching the stack)

After the parameters are received by the template, the CloudFormation stack creates these IAM roles:

  • A Lambda execution role with access to Amazon CloudWatch Logs, Amazon EC2, and Amazon S3
  • An S3 role with AmazonS3FullAccess

The AWS Lambda function is created after the roles are created. Lambda triggers the creation of the S3 destination bucket in the region specified in the CloudFormation template. Versioning is enabled on the bucket.

When the destination bucket is available, CloudFormation initiates the creation of the source bucket with cross-region replication enabled. The destination bucket is the target for cross-region replication.

Note: The creation of the IAM role and Lambda function is automated in the template. You do not need not create them manually.

Automated deployment

The step-by-step instructions in this section show you how you can automate the creation of an S3 bucket with cross-region replication enabled. After you click the button, the bucket will be created in approximately two minutes.

Note: Running this solution may result in charges to your AWS account. These include possible charges for Amazon S3 and AWS Lambda.

1. Sign in to the AWS Management Console and open the AWS CloudFormation console. Choose the Launch Stack button to create the AWS CloudFormation stack (S3CrossRegionReplication).

Cloudformation template link here.

Download the cloudformation template from github and upload the .yml file as template source. 

upload template

2. On the Specify details page, change the stack name, if required. Provide the following custom values to the CloudFormation template. These (user input) values will be passed as parameters to the template.

  • OriginalBucketName
  • ReplicationBucketName
  • ReplicationRegion

Choose Next.

3. On the Options page, you can specify tags for your AWS CloudFormation template, if you like, and then choose Next.

Permissions are built in the template. You don’t have to choose an IAM role.

Choose Next.

4. On the Review page, review your template details. Select the acknowledgement check box, and then choose Create to create the stack.

Note: Because this solution uses AWS Lambda, which is currently available in selected regions only, be sure you launch this solution in an AWS region where Lambda is available. For more information, see AWS service offerings by region.

Conclusion

In this blog post, we showed you how to use a single AWS CloudFormation template and AWS Lambda-backed custom resources to create an S3 bucket with cross-region replication enabled.

I would like to thank my colleague Arun Tunuri for his contributions in designing the CloudFormation template.

About the author

Rajakumar Sampathkumar is a Senior Technical Account Manager for Amazon Web Services. In his spare time, he is a passionate author and likes to spend quality time with his kids and nature.