Enforcing ownership of Amazon S3 objects in a multi-account environment

Today, Amazon S3 hosts over 100 trillion objects and regularly peaks at tens of millions of requests per second. As the set of use cases for S3 has expanded, our customers have asked us for new ways to regulate access to their mission-critical buckets and objects. For example, with a data lake hosted on Amazon S3 serving as a ‘single source of truth,’ it is common to have applications in multiple accounts performing reads and writes to objects in the same S3 bucket. In this scenario, you can have an S3 bucket with many objects, where each object is owned by a different account than the bucket owner account, which can make the process of managing access control complex.

S3 Object Ownership is a feature that offers an effective way to ensure that an S3 bucket owner automatically owns all objects written to a bucket. Using S3 Object Ownership, you can simplify access control management at scale in multi-account environments, as you can govern access to objects you own through access control mechanisms like bucket policies. By doing so, you can improve operational efficiency and security by standardizing ownership of and access to objects written to your Amazon S3 buckets without manual processes.

In this blog post, we cover a few examples of accessing objects in a multi-account environment, and a solution with a CloudFormation template demonstrating the use of S3 Object Ownership to control object ownership.

Example scenario of accessing objects from multiple accounts

Let’s look at some typical scenarios where this is particularly relevant. For example, you might have a data lake workflow where you have an analytics application in account B reading the ‘raw objects’ from an S3 bucket in account A, and writing ‘processed objects’ to the same bucket in account A. In this example, the ‘raw objects’ in the bucket would be owned by account A, and the ‘processed objects’ would be owned by account B. Your workflow may now require another application in account C to read ‘processed objects’ in the account A bucket.

Another example would be if you are using a recommended pattern of centralizing all your logs within your organization, for example: AWS CloudTrail logs, VPC Flow Logs and S3 server logs to one or more buckets in a designated logging account. In this scenario, each log is owned by the account that created the log. It is common to use third-party tools like Splunk or Datadog for analytics against this aggregated data, originating from multiple accounts, in the common logging account.

In the two preceding examples, it is not trivial to configure the required access control when objects are owned by multiple accounts within a bucket. With S3 Object Ownership, you can set up access controls so that the bucket owner automatically owns objects uploaded to the bucket.

History

Before the launch of S3 Object Ownership in 2020, the object writer was the default object owner. This caused challenges because the bucket policy for a bucket applies only to objects owned by the bucket owner. It does not apply to objects owned by other AWS accounts.

A common practice is for the bucket owner to take ownership of objects uploaded by other accounts. Previously, this was accomplished by the bucket owner copying each object to itself.

Now, with the new bucket configuration called S3 Object Ownership, the process of taking ownership of objects has been simplified. This bucket configuration is by default Object Writer, which means that the current default behavior for object ownership still applies (that is, an object is owned by the account that created it). However, the bucket owner has the option of toggling this attribute to Bucket Owner Preferred. With this setting, any objects uploaded to the bucket with the bucket-owner-full-control access control list (ACL) in place would automatically be owned by the bucket owner. This eliminates the extra step of having to copy the object to itself to take ownership.

Solution overview

Now let’s see how the new object ownership attribute can help in the scenario we described earlier. Suppose you have three accounts, A, B, and C. Account A is the owner of the bucket where all the objects are stored. Account B reads objects from this bucket and writes updated objects to the same bucket. Next, account C needs to read all objects in this bucket. To provide this access to account C with a bucket policy, you must first ensure that all the objects are owned by account A, the owner of the bucket.

Prerequisites

• Three different AWS accounts (A, B, and C)
• Bucket in account A with some objects
• Cross-account access to objects using one of the three methods

Solution walkthrough

The following are the steps to ensure account A owns all the objects uploaded to the S3 buckets by another account, account B, and account C can access these objects.

1. Configure Object Ownership to Bucket owner preferred for S3 bucket in account A.

I. Select your bucket and go to Permissions tab.

II. Under Object ownership, select Edit.

III. Change permissions to Bucket owner preferred and select Save changes.

Now any new objects uploaded to this bucket with bucket-owner-full-control ACL will be owned by the bucket owner (account A). Note that the Bucket owner preferred setting does not affect ownership of existing objects.

2. Now, when account B writes data, they should include the bucket-owner-full-control ACL with the PUT or COPY. This will automatically transfer the ownership of the object to account A due to the bucket setting from step 1.

For a PUT operation:

aws s3api put-object-acl --bucket AccountA-EXAMPLE-BUCKET/--key AccountB-EXAMPLE-BUCKET/myobject --acl bucket-owner-full-control

For a COPY operation:

aws s3 cp s3://AccountB-EXAMPLE-BUCKET/myobject s3:// AccountA-EXAMPLE-BUCKET / --acl bucket-owner-full-control

or

aws s3api copy-object --bucket AccountA-EXAMPLE-BUCKET/--key AccountB-EXAMPLE-BUCKET/myobject --acl bucket-owner-full-control

For COPY operations on an entire bucket:

aws s3 cp s3://AccountB-EXAMPLE-BUCKET/ s3://AccountA-EXAMPLE-BUCKET/ --acl bucket-owner-full-control --recursive

Once objects are written with bucket-owner-full-control, account A will own these objects and can provide access to other accounts, such as account C, using bucket policies, access point policies, or AWS Identity and Access Management (IAM) roles.

Enforce object ownership in your S3 bucket

Configure the following two settings so that the bucket owner always owns all of the objects uploaded in the bucket by other accounts:

• Update the Bucket Owner Preferred setting for the Object Ownership attribute in all shared buckets.
• Enforce the use of bucket-owner-full-control ACL with condition in the bucket policy. Any PUTs that are attempted which do not include the canned ACL will fail.

Here is an example of a bucket policy that account A can use to ensure that account B can only upload an object when the object’s ACL is set to bucket-owner-full-control:

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Sid": "Only allow writes to my bucket with bucket owner full control",
         "Effect": "Allow",
         "Principal": {
            "AWS": [
               "arn:aws:iam::XXXAccountBXX:role/roleX"
            ]
         },
         "Action": [
            "s3:PutObject"
         ],
         "Resource": "arn:aws:s3:::bucketAccountA/*",
         "Condition": {
            "StringEquals": {
               "s3:x-amz-acl": "bucket-owner-full-control"
            }
         }
      }
   ]
}

Note: If your use-case requires you to delete and recreate a user or role referenced in a trust policy’s Principal element, then use the role ARN in the Condition instead. If your Principal element contains the ARN for a specific IAM role or user, then that ARN is transformed to a unique principal ID when the policy is saved. This helps mitigate the risk of someone escalating their permissions by removing and recreating the role or user. You don’t normally see this ID in the console because there is also a reverse transformation back to the ARN when the trust policy is displayed. However, if you delete the role or user, then the principal ID appears in the console because AWS can no longer map it back to an ARN. Therefore, if you delete and recreate a user or role referenced in a trust policy’s Principal element, you must edit the role to replace the ARN. For more information, refer to the documentation on how to create an IAM role.

Any attempt to upload an object without bucket-owner-full-control canned ACL will result in an ‘Access denied’ error.

Using AWS CloudFormation

You can also use the following AWS CloudFormation template to make it so that a bucket owner owns all of the objects uploaded to an S3 bucket. Use this only if you want all objects written to a bucket to be owned by the bucket owner (by default, the object writer owns the object).

Steps:

1. Download the CloudFormation template.
2. On the AWS CloudFormation console, select Upload a template file.
3. Upload the file you downloaded, then choose Next.
4. For Stack Name, enter the name of your stack. Under Parameters, update the parameters for the template with the following inputs, then choose Next.

5. On the Configure Stack Options page, choose Next, then choose Create stack.

You can view the status of the stack on the AWS CloudFormation console. You should see the status CREATE_COMPLETE in approximately 5 minutes.

Cleaning up

To clean up the environment and avoid incurring unwanted charges, follow the instructions here to delete the CloudFormation stack. Note that if you have objects that you created in your test bucket, you will need to delete them first before deleting the stack.

Conclusion

In this blog, we showed how to easily and automatically enforce “Bucket Owner=Object Owner” in a multi-account environment with many readers and writers. Previously, this could only be done with a manual step of copying an object to itself, when the object is uploaded by any account other than the bucket owner. S3 Object Ownership eliminates this manual step. Your bucket policy can now apply to all the objects in the bucket, simplifying access control at scale.

Thanks for reading this blog post. If you have any comments or questions, don’t hesitate to post a comment in the comments section.