AWS Storage Blog

Tag: AWS Identity and Access Management (IAM)

s3-annotations-header-image

Analyze Amazon S3 annotations at scale with materialized views

Customers managing large volumes of objects in Amazon Simple Storage Service (Amazon S3) often need to attach rich business context like compliance classifications, processing lineage, AI-generated labels, and more. Until now, this context lived in external databases or sidecar files that were stored as separate objects, which created complexity to manage and keep it up […]

Amazon S3 Replication

Accelerate Amazon S3 Replication with automated S3 Batch Operations parallelization

As data volumes grow, organizations must move large datasets between storage locations to meet compliance requirements, optimize performance, and help meet data sovereignty requirements. However, migrating petabytes of data presents significant challenges: lengthy transfer times, complex coordination of parallel operations, data integrity verification, and substantial engineering overhead. Automation reduces resource consumption and operational complexity during […]

Simplify compliance-driven Amazon S3 data movement with multi-criteria filtering

Organizations routinely need to move a subset of their stored data from one location to another, such as a compliance audit that requires all PDFs from a specific quarter, a company reorganization that splits one tenant’s records into an isolated bucket, or a regulatory mandate that demands financial documents older than 7 years be archived […]

Amazon S3 featured image 2023

Unlocking data residency use cases with Amazon S3 in AWS Local Zones

Organizations running workloads in metros and geographies far from major cloud infrastructure need scalable, fully managed object storage, but regulations or business requirements mandate that data stays within specific national or metropolitan boundaries. This is particularly true for financial services, healthcare, and public sector, where compliance frameworks not only dictate where primary data resides but […]

Amazon S3 Storage Lens featured image

Gain workload-specific storage insights with Amazon S3 Storage Lens groups

As industries generate and store growing volumes of data, gaining meaningful insights into storage usage becomes increasingly complex. You need to understand your data growth patterns and drivers while optimizing storage investments across different business units and workloads. However, obtaining the necessary visibility by data categories, departments, or applications remains operationally difficult, limiting the ability […]

S3 logging decision tree

Amazon S3 audit logging, Part 3: Analyzing S3 Metadata journal tables for object lifecycle tracking

This is Part 3 of our three-part series on Amazon S3 audit logging. In Part 1, we covered server access logs for HTTP-level requests and performance analysis. In Part 2, we covered S3 data events in AWS CloudTrail for identity-focused security investigations. As data volumes grow and storage costs become a significant line item, organizations […]

S3 logging decision tree

Amazon S3 audit logging, Part 2: Centralized logging and analysis of S3 data events in AWS CloudTrail for security and compliance

This is Part 2 of our three-part series on Amazon S3 audit logging, focusing on identity-driven security investigations. In Part 1, we covered S3 server access logs for HTTP-level performance analysis and cost attribution. When a security incident occurs—an unauthorized download, a bulk deletion, or suspicious access from an unfamiliar location—the first question is always, […]

S3 logging decision tree

Amazon S3 audit logging, Part 1: Analyzing server access logs with Amazon Athena for performance insights

Organizations storing sensitive data must maintain complete visibility into how it’s accessed, by whom, and what changes occur over time. Regulatory frameworks demand detailed audit trails, security teams need rapid answers during investigations, and finance teams require granular cost attribution. Yet as data grows from terabytes to petabytes, the scale that makes centralized storage attractive […]

Scalable cross-cloud data migration to Amazon S3 with distributed rclone

Migrating petabytes of data across cloud providers is one of the most operationally demanding tasks an organization can take on. At this scale, simple transfer approaches break down. Teams lose track of what has been copied and what has failed. Transfers stall and require constant manual intervention to restart. In some cases, teams need to […]

Amazon S3

Implement single-exchange tokens for short-lived Amazon S3 presigned URLs with Terraform

Organizations across industries use signed URLs to grant temporary, credential-less access to private resources such as receipts, medical or financial records, legal files, or confidential reports. However, signed URLs can be reused by anyone until they expire, creating security risks if a URL is shared or inadvertently disclosed. This risk can be mitigated by vending […]