Tag: Amazon S3 Batch Operations

As more organizations look to operate faster and at scale, they need ways to meet critical compliance requirements and improve data security. Encryption is a critical component of a defense in depth strategy, and when used correctly, can provide an additional layer of protection above basic access control. However, workloads that access millions or billions […]

UPDATE (2/10/2022): Amazon S3 Batch Replication, which is not covered in this blog post, launched on 2/8/2022, allowing you to replicate existing S3 objects and synchronize your S3 buckets. See the S3 User Guide for additional details. UPDATE (5/1/2023): Updated the comparison table to reflect the latest capabilities of the mechanisms covered in the table. […]

Update (3/4/2022): Added support for Glacier Instant Retrieval storage class. Update (4/19/2022): Included the copy destination prefix parameter in the Amazon CloudFormation template. Update (10/26/2022): Added performance guidance and best practices, and included template optimized for copying objects restored from archive to a different storage class. Update (4/23/2025): Upgraded AWS Lambda function Python runtime to […]

A challenge for many enterprises with data at the scale of petabytes is managing and taking actions on their data to migrate, improve efficiency, and drive down costs through automation. Amazon S3 buckets can hold billions of objects and exabytes of data, letting you build your applications with the ability to grow and scale as […]

UPDATE (2/10/2022): Amazon S3 Batch Replication launched on 2/8/2022, allowing you to replicate existing S3 objects and synchronize your S3 buckets. See the S3 User Guide for additional details. The need for data synchronization in Amazon S3 comes up in a number of scenarios for customers – enabling a new geographic region for end users, […]

For many organizations, securely and cost-effectively organizing massive amounts of stored data can be a daunting task. Without an easy way to organize data, companies may find themselves exerting a lot of time and energy trying to meet their data storage requirements, potentially slowing down projects and operations. For instance, certain data may need to […]

As organizations grow their use of AWS, they often find that a variety of teams and applications begin to use the data stored in Amazon S3. While customers love the agility benefits of this, they also seek to govern their data’s security, productivity, and cost. Earlier this year we announced support for job tags with […]

As customers scale their business on AWS, they can have millions to billions of objects in their Amazon S3 buckets. Customers often run operations on a large number of these objects in their buckets, including copying objects across accounts, encrypting objects, or tagging. Running operations on a large number of objects in S3 involves listing […]

Update (12/11/2023): As of November 20, 2023, Amazon S3 supports enabling S3 Object Lock on existing buckets. Amazon S3 Batch Operations now supports S3 Object Lock. In this post, I share ways you can use these two Amazon S3 features together to address common data protection needs. S3 Batch Operations is a feature that lets […]

Do you know exactly what’s in all of your Amazon Simple Storage Service (Amazon S3) buckets? Do all of your objects have the correct set of object tags applied? Have you set up lifecycle rules to ensure your current and future objects are managed automatically for you? If you answered “yes” to all of the […]