Getting Started / Hands-on / ...

Getting started with Amazon S3 Multi-Region Access Points

TUTORIAL

Introduction
CREATE AMAZON S3 BUCKETS
create s3 multi-region access point
configure s3 replication
Failover Configuration
IAM Permissions
use your multi-region access point
vpc endpoints
monitor s3 replication and requests
clean up resources

Module 8: Monitor S3 Replication and Requests

You will learn how to generate traffic to graph, and monitor S3 Replication metrics and requests.

Overview

In this module, you will learn how to monitor Amazon S3 Replication in the Multi-Region Access Point console, and S3 request metrics in Amazon CloudWatch.

 Time to complete

10 minutes

Implementation

8.1 - Generating traffic to graph

  • If you have only uploaded one or two objects to S3, there will not be very much to show on the graphs. If you wish to generate some traffic to graph, the following load generator bash commands in this step may be useful. You will need to set the environment variable $mrap_arn to the ARN of your Multi-Region Access Point before running it on the computer used in module 6.
  • The commands create a 1 MiB file, and then write and read a new object using your access point every 20 seconds indefinitely.

Note: Remember to stop the script when you are done testing, in order to minimize your costs.

  • Run the following commands:
#Create a 1 MiB file containing random data
dd if=/dev/urandom of=~/1MiB.file bs=1024 count=1024

#Write and read a new S3 object every 20 seconds
while true; do datetime=$(date "+%Y.%m.%d-%H.%M.%S") \
; aws s3 cp ~/1MiB.file s3://$mrap_arn/load_generator/1MiB_from_load_generator.$datetime \
; aws s3 cp s3://$mrap_arn/load_generator/1MiB_from_load_generator.$datetime ~/1MiB_from_S3 \
; sleep 20 ; done

8.2 - Replication metrics

  • Navigate back to the browser tab with your Multi-Region Access Point open. If you need to re-open this, in the Amazon S3 console, choose Multi-Region Access Points. Then, select the name of your Multi-Region Access Point to configure additional settings.
  • Choose the Replication and failover tab.
  • Find the Replication Metrics section and choose 1h.
    • The graph might display No data, until you select 1hr, because of the short period replication has been active.

Notes:

  • You can select the legend entries to hide or highlight individual lines, of which there is one per replication rule. Each one shows the Source bucket and the name of the rule.
  • You can tell which bucket is the one the Multi-Region Access Point routed your writes to, as replication latency will only be above zero for that Source bucket.
  • If you want to see the effect of failover, leave the load generator script running and change the failover configuration of your Multi-Region Access Point so that the bucket being written to is passive (see step 4). The change will take effect in about 2 minutes, and a few minutes later the graph will start to show some replication latency for the original bucket (as objects are now being replicated to it) and no latency for the bucket that is now being written to.
  • On this page you can also review graphs for Bytes pending replication and Operations pending replication.
     

8.3 - Request metrics

Amazon CloudWatch can monitor and display S3 request metrics, allowing you to see which buckets are fulfilling requests via your Multi-Region Access Point. There are three types of Amazon CloudWatch metrics for Amazon S3: storage metrics, request metrics, and replication metrics. Storage metrics are reported once per day and are provided to all customers at no additional cost. Request metrics are available at one-minute intervals after some latency for processing. Request metrics are billed at the standard CloudWatch rate. You must opt in to request metrics for each bucket by configuring them in the console or using the Amazon S3 API.
  • Once metrics have been logged (consider manually reading from and writing to each bucket directly to create a log for each request type and bucket), you can add them in CloudWatch under Metrics → All metrics → S3 → Request Metrics Per Filter. Graphing the sum of GetRequests and PutRequests over each 1-minute period provides a view of all activity (including replication activity) like this:
    • The example image on the right shows the effect of setting the us-east-1 bucket to passive when all clients were closest to the us-east-1 region. Note: PutRequests to the us-east-1 bucket continued, as new objects were replicated into us-east-1. S3 Replication is independent of Multi-Region Access Point failover controls.

Conclusion

Congratulations! You have learned how to set up and use an Amazon S3 Multi-Region Access Point to access data in multiple regional S3 buckets via a single global endpoint, and tested the failover controls capability.

We recommend that you clean up your resources to stop the accrual of associated costs.

Start Module Nine

Clean Up Resources