How can I initiate restores for a large volume of Amazon S3 objects that are currently in the S3 Glacier storage class?

Last updated: 2020-02-20

I have a large number of objects on Amazon Simple Storage Service (Amazon S3) that are currently in the S3 Glacier storage class. I want to initiate a restore on all of them. How can I initiate a restore in a large-scale operation?

Short Description

Create an Amazon S3 batch operation job to initiate the restore for all the objects. You can execute an Initiate Restore Object job on a custom list of objects, or an Amazon S3 inventory report.

Note: Before you create a batch operation job, be sure to review the pricing for Amazon S3 batch operations.

Resolution

Before you begin, be sure that you have:

  • An AWS Identity and Access Management (IAM) user or role that has permissions to initiate a restore. Additionally, the IAM user or role must have a trust policy with Amazon S3 batch operations.
  • Either a CSV list or an Amazon S3 inventory report to serve as the manifest of the objects that you want to restore. The manifest file must be stored on an S3 bucket. Manifests with server-side encryption (customer-provided keys or AWS Key Management Service keys) aren't supported. For more information on the requirements for each format, see Specifying a Manifest.

Follow these steps to use the Amazon S3 console to create a batch operation job for initiating a restore:

Note: To create a job using the AWS Command Line Interface (AWS CLI), see AWS CLI Examples for Amazon S3 Batch Operations and create-job.

  1. Open the Amazon S3 console.
  2. From the navigation pane, choose Batch operations.
  3. Choose Create job.
  4. For Region, select the AWS Region that you want to create the job in.
  5. Under Choose manifest, enter the following:
    For Manifest format, select S3 inventory report or CSV depending on the file format that you created for the list of objects that you want to restore.
    For Path to manifest object, enter the S3 path to the manifest file (for example, s3://awsexamplebucket/manifest.csv).
  6. Choose Next.
  7. Under Choose operation, enter the following:
    For Operation, select Restore.
    For Number of days that the restored copy is available, enter the number of days for your use case.
    For Restore tier, select either Bulk retrieval or Standard retrieval. For more information on each tier, see Archive Retrieval Options.
    Note: For a faster retrieval, you can use the AWS CLI to create the batch operation job and then specify the Expedited restore tier.
  8. Choose Next.
  9. Under Configure additional options, enter the following:
    For Description, you can choose to enter a description of the job. Or, you can leave this field blank.
    For Priority, enter a number to indicate the job's priority.
    For Generate completion report, you can choose to keep this option selected.
    For Completion report scope, select Failed tasks only or All tasks depending on your use case.
    For Path to completion report destination, enter the path that you want the report to be sent to.
    For Permission to access the specified resources, select Choose from existing IAM roles. Then, select the IAM role that has permissions to initiate a restore and has a trust policy with S3 batch operations.
  10. Choose Next.
  11. On the Review page, review the details of the job. Then, choose Create job.
  12. After you create the job, the job's status changes from New to Preparing. Then, the status changes to Awaiting your confirmation. To run the job, you must select the job and then choose Confirm and run. The job doesn't run until you confirm it.
  13. If you selected Generate completion report, then you can review the report after the job completes. You can find the report at the Path to completion report destination that you specified.

For descriptions of each job status, see Job Status. For more information on failed jobs, see Tracking Job Failure.