Getting Started Resource Center

Developer Tools

Replicate Data within and between AWS Regions

Using Amazon S3 Replication

Introduction

Overview

Amazon S3 Replication is an elastic, fully managed, low-cost feature that replicates objects between Amazon S3 buckets. S3 Replication gives you the ability to replicate data from one source bucket to multiple destination buckets in the same, or different, AWS Regions. Whether you want to maintain a secondary copy of your data for data protection, or have data in multiple geographies to provide users with the lowest latency, S3 Replication gives you the controls you need to meet your business needs. This Amazon S3 getting started guide shows you how to follow S3 Replication best practices with S3 Same-Region Replication (SRR), S3 Cross-Region Replication (CRR), S3 Replication Time Control (S3 RTC), and S3 Batch Replication.

With S3 Same-Region Replication (SRR), you can automatically replicate data between buckets within the same AWS Region to help aggregate logs into a single bucket, replicate between developer and test accounts, and abide by data sovereignty laws. With S3 Cross-Region Replication (CRR), you can replicate objects (and their respective metadata and object tags) into other AWS Regions for reduced latency, compliance, security, disaster recovery, and regional efficiency. You can also enable S3 Replication Time Control (S3 RTC) to help you meet compliance or business requirements for data replication. S3 RTC replicates most objects that you upload to Amazon S3 in seconds, and 99.99 percent of those objects within 15 minutes. To replicate existing objects, you can use S3 Batch Replication to backfill a newly created bucket with existing objects, retry objects that were previously unable to replicate, migrate data across accounts, or add new buckets to your data lake. For more information on S3 Replication, visit the Replicating Objects section in the Amazon S3 User Guide. By the end of this tutorial, you will be able to replicate data within and between AWS Regions using Amazon S3 Replication.

What you will accomplish

In this tutorial, you will:

Create an S3 bucket
Create an S3 Replication rule on your S3 bucket
Choose destination S3 bucket
Choose or create IAM roles for replication
Specify encryption type (optional)
Choose destination S3 storage class
Enable additional replication options (optional)

Prerequisites

To complete this tutorial, you need an AWS account. Access this support page for more information on how to create and activate a new AWS account.

Implementation

AWS experience

Beginner

Time to complete

20 minutes

Cost to complete

Less than $1 (Amazon S3 pricing page)

Services used

Amazon S3

Last updated

October 17, 2022

1. Sign in to the Amazon S3 console

If you have not already done so, create an AWS account.

From the AWS console services search bar, enter S3. Under the services search results section, select S3.

2. Create your first bucket

Choose Buckets from the Amazon S3 menu in the left navigation pane and then choose the Create bucket button.

3. Configure bucket properties

Enter a descriptive, globally unique name for your bucket. Select which AWS Region you would like your bucket created in. For this example, the EU (Frankfurt) eu-central-1 Region is selected.

4. Enable bucket versioning

S3 Replication requires Bucket Versioning to be enabled for both source and destination S3 buckets. For more information about versioning, see Using versioning in S3 buckets.

You can leave the remaining options as defaults. Navigate to the bottom of the page and choose Create bucket.

5. Create your destination bucket

Repeat the above steps to create another S3 bucket to serve as the destination bucket for replicating objects. Make sure to enable Bucket Versioning for the destination S3 bucket as well.

Create an S3 Replication on your S3 bucket

1. Choose the source bucket

From your list of S3 buckets, choose the S3 bucket that you want to configure as your source for replication.

2. Review bucket details

Once you select the source S3 bucket, the console takes you to the S3 bucket landing page, as shown in the following screenshot. Here, you can review the Objects, Properties, Permissions, Metrics, Management, and Access Points for the selected S3 bucket.

3. Create a replication rule

Choose the Management tab of the replication source bucket. Under Management, you will see Replication rules. Select Create replication rule.

Configure a replication rule

1. Enable replication

Provide a Replication rule name and enable the replication rule by selecting Enabled under the Status section. If the replication rule is disabled, it will not run.

Priority indicates which rule has precedence whenever two or more replication rules conflict. You will have the option to edit the priority of each replication rule on the replication configuration page. Amazon S3 attempts to replicate objects according to all replication rules. However, if there are two or more rules with the same destination bucket, then objects are replicated according to the rule with the highest priority. A rule with priority 1 is executed before a rule with priority 2. The lower the number, the higher the priority. For example, say you have a replication rule to replicate all objects with tag foo1 and another replication rule to replicate all objects with tag foo2. If you have one object tagged with foo1 and foo2, it will only get replicated with the replication rule with the higher priority. When you have only one replication rule going to one destination bucket, priority is not considered.

2. Set the replication scope

Narrow the scope of replication by defining a Filter type (Prefix or Tags), or choose to replicate the entire bucket. For example, if you want to only replicate objects that are in the Prefix “Finance”, specify that scope. For more information on filtering objects for replication, visit the documentation on specifying a filter in the S3 User Guide.

3. Choose the destination bucket

Choose the destination bucket by selecting the Browse S3 button. You can replicate to a destination bucket in the same or different AWS Region, and in the same or different AWS account. Note that you will need two different S3 buckets to configure replication, and both buckets (source and destination) must have S3 Versioning enabled. The S3 console does not provide you with a way to create a new S3 bucket in the replication setup process. In this example, we chose the destination bucket to be “aws-s3-replication-tutorial-destination-bucket.”

4. Set up IAM permissions

When creating new replication rules from the same source bucket, make sure that the IAM role associated with this configuration has sufficient permissions to write new objects in the new destination bucket. You can choose to create a new IAM role or select an existing IAM role with the right set of permissions. For more information, see the documentation on setting up permissions for S3 Replication.

5. (Optional) Configure encryption

Skip this step if your objects are not encrypted.

If your objects are encrypted with Amazon S3-managed encryption keys (SSE-S3) or AWS Key Management Service (AWS KMS), you will need to specify the encryption options while setting up replication. S3 Replication supports SSE-S3 (default encryption) and AWS KMS server-side encryption. If you choose AWS KMS encryption, you will need to provide the AWS KMS keys to decrypt in source and re-encrypt in destination. To save on AWS KMS costs, you also have the option to enable Amazon S3 Bucket Keys.