How can I initiate restores for a large volume of Amazon S3 objects that are currently in the S3 Glacier or S3 Glacier Deep Archive storage class?

Last updated: 2021-11-19

I have a large number of objects that are in the Amazon Simple Storage Service (Amazon S3) Glacier or Amazon S3 Glacier Deep Archive storage class. I want to initiate a restore on all of them. How can I initiate a restore in a large-scale operation?

Short description

Create an Amazon S3 batch operation job to initiate the restore for all the objects. You can run an Initiate restore object job on a custom list of objects, or an Amazon S3 inventory report.

Note: Before you create a batch operation job, be sure to review the pricing for Amazon S3 batch operations.

Resolution

Before you begin, the following requirements must be met:

  • An AWS Identity and Access Management (IAM) user or role that has permissions to initiate a restore. Additionally, the IAM user or role must have a trust policy with Amazon S3 batch operations.
  • Either a CSV list or an Amazon S3 inventory report to serve as the manifest of the objects that you want to restore. The manifest file must be stored on an S3 bucket. Manifests with server-side encryption (customer-provided keys or AWS Key Management Service keys) aren't supported. For more information on the requirements for each format, see Specifying a manifest.

Follow these steps to use the Amazon S3 console to create a batch operation job for initiating a restore:

1.    Open the Amazon S3 console.

2.    From the navigation pane, choose Batch operations.

3.    Choose Create job.

4.    For Region, select the AWS Region where you want to create the job.

5.    Under Choose manifest, enter the following:
For Manifest format, select S3 inventory report or CSV as your file format.
For Path to manifest object, enter the S3 path to the manifest file. (For example, the S3 path looks like this: "s3://awsexamplebucket/manifest.csv".)

6.    Choose Next.

7.    Under Choose operation, enter the following:
For Operation, select Restore.
For Restore source, select Glacier or Glacier Deep Archive.
For Number of days that the restored copy is available, enter the number of days for your use case.
For Restore tier, select either Bulk retrieval or Standard retrieval. For more information on each tier, see Archive retrieval options.

Note: For a faster retrieval on objects in the S3 Glacier storage class only, you can use the create-job CLI command and specify the Expedited restore tier. S3 Batch Operations does not support the Expedited retrieval tier.

8.    Choose Next.

9.    Under Configure additional options, enter the following:
For Description, you can choose to enter a description of the job. Or, you can leave this field blank.
For Priority, enter a number to indicate the job's priority.
For Generate completion report, you can choose to keep this option selected.
For Completion report scope, select Failed tasks only or All tasks depending on your use case.
For Path to completion report destination, enter the path that you want the report to be sent to.
For Permission, select Choose from existing IAM roles. Then, select the IAM role that has permissions to initiate a restore and has a trust policy with S3 batch operations.

10.    Choose Next.

11.    On the Review page, review the details of the job. Then, choose Create job.

12.    After you create the job, the job's status changes from New to Preparing. Then, the status changes to Awaiting your confirmation. To run the job, you must select the job and then choose Confirm and run. The job doesn't run until you confirm it.

13.    (Optional) If you selected Generate completion report, then you can review the report after the job completes. You can find the report at the Path to completion report destination that you specified.

For descriptions of each job status, see Job status. For more information on failed jobs, see Tracking job failure.