How do I view objects that failed replication from one Amazon S3 bucket to another?

3 minute read
0

I want to retrieve a list of objects that failed replication when setting up replication from one Amazon Simple Storage Service (Amazon S3) bucket to another bucket.

Short description

You can turn on S3 Replication Time Control (S3 RTC) to set up event notifications for eligible objects that failed replication. You can also use S3 RTC to set up notifications for eligible objects that take longer than 15 minutes to replicate. Additionally, you can get a list of objects that failed replication in one of the following ways:

  • Reviewing the Amazon S3 inventory report
  • Running the HeadObject API call

Resolution

Amazon S3 inventory report

Amazon S3 inventory reports list your objects and their metadata on a daily or weekly basis. The replication status of an object can be PENDING, COMPLETED, FAILED, or REPLICA.

To find objects that failed replication, filter a recent report for objects with the replication status of FAILED. Then, you can initiate a manual copy of the objects to the destination bucket. You can also re-upload the objects to the source bucket (after rectifying the permissions) to initiate replication.

You can also use Amazon Athena to query the inventory report for replication statuses.

HeadObject API call

For a list of the objects in the source bucket that are set for replication, you can run the HeadObject API call on the objects. HeadObject returns the PENDING, COMPLETED, or FAILED replication status of an object. In a response to a HeadObject API call, the replication status is found in the x-amz-replication-status element.

Note: To run HeadObject, you must have read access to the object that you're requesting. A HEAD request has the same options as a GET request, but without performing a GET.

After HeadObject returns the objects with a FAILED replication status, you can initiate a manual copy of the objects to the destination bucket. You can also re-upload the objects to the source bucket (after rectifying the permissions) to initiate replication.

Important: If you manually copy objects into the destination bucket, then the Amazon S3 inventory report and HeadObject API calls return a FAILED replication status. This replication status is for the objects in the source bucket. To change the replication status of an object and initiate replication, you must re-upload the object to the source bucket. If the new replication is successful, then the object's replication status changes to COMPLETED. If you must manually copy objects into the destination bucket, then be sure to note the date of the manual copy. Then, filter objects with a FAILED replication status by the last modified date. Doing this lets you to identify which objects are or aren't copied to the destination bucket.

AWS OFFICIAL
AWS OFFICIALUpdated a year ago