Can I be notified if replication of an object across Amazon S3 buckets fails?
Last updated: 2019-12-20
I set up replication from one Amazon Simple Storage Service (Amazon S3) bucket to another bucket. If replication fails, how can I get a list of objects that failed to be replicated?
You can enable S3 Replication Time Control (S3 RTC), which allows you to set up notifications for eligible objects that failed replication, or eligible objects that take longer than 15 minutes to replicate. Additionally, you can get a list of objects that failed replication in one of these ways:
- Reviewing the Amazon S3 inventory report
- Running the HeadObject API call
Amazon S3 inventory report
Amazon S3 inventory reports list your objects and their metadata on a daily or weekly basis. The replication status of an object can be PENDING, COMPLETED, FAILED, or REPLICA.
To find objects that failed replication, filter a recent report for objects with the replication status of FAILED. Then, you can initiate a manual copy of the objects to the destination bucket. Or, you can re-upload the objects to the source bucket, which triggers replication.
You can also use Amazon Athena to query the inventory report for replication statuses.
HeadObject API call
If you have a list of the objects in the source bucket that are set for replication, then you can run the HeadObject API call on those objects. HeadObject returns the PENDING, COMPLETED, or FAILED replication status of an object. In a response to a HeadObject API call, the replication status is found in the x-amz-replication-status element.
Note: To run HeadObject, you must have read access to the object that you're requesting. A HEAD request has the same options as a GET request, but without performing a GET.
After HeadObject returns the objects with a FAILED replication status, you can initiate a manual copy of the objects to the destination bucket. Or, you can re-upload the objects to the source bucket, which triggers replication.
Important: If you manually copy objects into the destination bucket, then the Amazon S3 inventory report and HeadObject API calls continue to return a FAILED replication status for the objects in the source bucket. To change the replication status of an object, you must re-upload the object to the source bucket, which triggers replication. If the new replication is successful, the object's replication status changes to COMPLETED. If you must manually copy objects into the destination bucket, then be sure to note the date of the manual copy. Then, be sure to filter objects with a FAILED replication status by the last modified date so that you can identify which objects have or haven't been copied to the destination bucket.