Why can't I copy an object between two Amazon S3 buckets?

Last updated: 2019-08-30

I'm trying to copy an object from one Amazon Simple Storage Service (Amazon S3) bucket to another, but it's not working. How can I troubleshoot this?

Short Description

To troubleshoot issues with copying an object between buckets, check the following:

  • Bucket policies and AWS Identity and Access Management (IAM) policies
  • Object ownership
  • AWS Key Management Service (AWS KMS) encryption
  • Amazon Simple Storage Service Glacier (Amazon S3 Glacier) storage class
  • Requester Pays enabled on bucket
  • AWS Organizations service control policy
  • Cross-Region request issues with Amazon Virtual Private Cloud (VPC) endpoints for Amazon S3

Resolution

Bucket policies and IAM policies

To copy an object between buckets, you must be sure that the correct permissions are configured. To copy an object between buckets in the same AWS account, you can set permissions using IAM policies. To copy an object between buckets in different accounts, you must set permissions on both the relevant IAM policies and bucket policies.

Note: For instructions on how to modify a bucket policy, see How Do I Add an S3 Bucket Policy? For instructions on how to modify the permissions for an IAM user, see Changing Permissions for an IAM User. For instructions on how to modify the permissions for an IAM role, see Modifying a Role.

Confirm the following required permissions:

  • At minimum, your IAM identity (user or role) must have permissions to the s3:ListBucket and s3:GetObject actions on the source bucket. If the buckets are in the same account, then you can set these permissions using your IAM identity's policies. If the buckets are in different accounts, then set these permissions using both the bucket policy and your IAM identity's policies.
  • At minimum, your IAM identity must have permissions to the s3:ListBucket and s3:PutObject actions on the destination bucket. If the buckets are in the same account, you can set these permissions using your IAM identity's policies. If the buckets are in different accounts, set these permissions using both the bucket policy and your IAM identity's policies.
  • Review the relevant bucket policies and IAM policies to confirm that there are no explicit deny statements conflicting with the permissions that you need. An explicit deny statement overrides an allow statement.
  • For specific operations, confirm that your IAM identity has permissions to all the necessary actions within the operation. For example, to run the command aws s3 cp, you need permission to s3:GetObject and s3:PutObject. To run the command aws s3 cp with the --recursive option, you need permission to s3:GetObject, s3:PutObject, and s3:ListBucket. To run the command aws s3 sync, then you need permission to s3:GetObject, s3:PutObject, and s3:ListBucket.
    Note: If you're using the AssumeRole API operation to access Amazon S3, you must also verify that the trust relationship is configured correctly.
  • For version-specific operations, confirm that your IAM identity has permissions to version-specific actions. For example, if you want to copy a specific version of an object, then you need the permission for s3:GetObjectVersion in addition to s3:GetObject.
  • Review the relevant bucket policies and IAM policies to be sure that the Resource element has the correct path. For bucket-level permissions, the Resource element must point to a bucket. For object-level permissions, the Resource element point to an object or objects.

For example, a policy statement for a bucket-level action like s3:ListBucket must specify a bucket in the Resource element, similar to the following:

"Resource": "arn:aws:s3:::awsexamplebucket"

A policy statement for object-level actions like s3:GetObject or s3:PutObject must specify an object or objects in the Resource element, similar to the following: 

"Resource": "arn:aws:s3:::awsexamplebucket/*"

Object ownership

If the bucket policies have the correct permissions and you're still having problems copying an object between buckets, check which AWS account owns the object. The bucket policy applies only to objects owned by the bucket owner. An object that's owned by a different account might have conflicting permissions on its access control list (ACL).

Note: The object ownership and ACL issue typically occurs when you copy AWS service logs across accounts. Examples of service logs include AWS CloudTrail logs and Elastic Load Balancing access logs.

Follow these steps to find the account that owns an object:

1.    Open the Amazon S3 console.

2.    Navigate to the object that you can't copy between buckets.

3.    Choose the object's Permissions tab.

4.    Review the values under Access for object owner and Access for other AWS accounts:

  • If the object is owned by your account, then the Canonical ID under Access for object owner contains (Your AWS account).
  • If the object is owned by another account and you have access to the object, then the Canonical ID under Access for object owner contains (External account). The Canonical ID under Access for other AWS accounts contains (Your AWS account).
  • If the object is owned by another account and you don't have access to the object, then the Canonical ID fields for both Access for object owner and Access for other AWS accounts are empty.

If the object that you can't copy between buckets is owned by another account, then the object owner can do one of the following:

  • The object owner can grant the bucket owner full control of the object. After the bucket owner owns the object, the bucket policy applies to the object.
  • The object owner can keep ownership of the object, but they must change the ACL to the settings that you need for your use case.

AWS KMS encryption

If the object is encrypted using an AWS KMS key, then confirm that your IAM identity has the correct permissions to the key. If your IAM identity and the AWS KMS key belong to the same account, then confirm that the key policy grants your IAM identity permissions to the following actions:

"Action": [
    "kms:Encrypt",
    "kms:Decrypt",
    "kms:ReEncrypt*",
    "kms:GenerateDataKey*",
    "kms:DescribeKey"
 ]

If your IAM identity and the AWS KMS key belong to different accounts, confirm that both the key policy and your IAM policies grant the required permissions.

For more information, see Using Key Policies in AWS KMS and Actions, Resources, and Condition Keys for AWS Key Management Service.

Amazon S3 Glacier storage class

You can't copy an object from the Amazon S3 Glacier storage class. You must first restore the object from Amazon S3 Glacier before you can copy the object. For instructions, see How Do I Restore an S3 Object That Has Been Archived?

Requester Pays enabled on bucket

If the source or destination bucket has Requester Pays enabled, and you're trying to access the bucket from another account, verify that your request includes the correct Requester Pays parameter:

  • For AWS Command Line Interface (AWS CLI) commands, include the --request-payer option.
  • For GET, HEAD, and POST requests, include x-amz-request-payer : requester.
  • For signed URLs, include x-amz-request-payer=requester.

AWS Organizations service control policy

If you're using AWS Organizations, check the service control policies to be sure that access to Amazon S3 is allowed.

For example, the following policy results in a 403 Forbidden error when you try to access Amazon S3 because it explicitly denies access: 

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Deny",
            "Action": "S3:*",
            "Resource": "*"
        }
    ]
}

For more information on the features of AWS Organizations, see Enabling All Features in Your Organization.

Cross-Region request issues with VPC endpoints for Amazon S3

VPC endpoints for Amazon S3 currently don't support cross-Region requests. For example, if you have an Amazon Elastic Cloud Compute (Amazon EC2) instance in Region A with a VPC endpoint configured in its associated route table, then the instance can't copy an object from Region B to a bucket in Region A. Instead, you receive an error message similar to the following:

"An error occurred (AccessDenied) when calling the CopyObject operation: VPC endpoints do not support cross-region requests"

To troubleshoot this cross-Region request issue, you can:

  • Remove the VPC endpoint from the route table. If you remove the VPC endpoint, the instance must be able to connect to the internet instead.
  • Run the copy command from another instance that's not using the VPC endpoint, or from an instance that's in neither Region A nor Region B.
  • If you must use the VPC endpoint, send a GET request to copy the object from the source bucket to the EC2 instance. Then, send a PUT request to copy the object from the EC2 instance to the destination bucket. 

Did this article help you?

Anything we could improve?


Need more help?