AWS Storage Blog

Tag: AWS Identity and Access Management (IAM)

Scalable cross-cloud data migration to Amazon S3 with distributed rclone

Migrating petabytes of data across cloud providers is one of the most operationally demanding tasks an organization can take on. At this scale, simple transfer approaches break down. Teams lose track of what has been copied and what has failed. Transfers stall and require constant manual intervention to restart. In some cases, teams need to […]

Amazon S3

Implement single-exchange tokens for short-lived Amazon S3 presigned URLs with Terraform

Organizations across industries use signed URLs to grant temporary, credential-less access to private resources such as receipts, medical or financial records, legal files, or confidential reports. However, signed URLs can be reused by anyone until they expire, creating security risks if a URL is shared or inadvertently disclosed. This risk can be mitigated by vending […]

Amazon S3 Tables

Enabling natural language access to structured data using Amazon S3 Tables and Amazon Bedrock Knowledge Bases

Organizations generate massive volumes of structured data from customer transactions, operational metrics, product catalogs, and compliance records. This data contains insights that can help businesses make better and timely decisions. Financial advisors need to review client transaction histories, retail analysts track inventory trends, and healthcare administrators monitor patient outcomes. Yet accessing these insights creates a […]

Migrate-to-Amazon-S3-account-regional-namespaces-Featured Image

Migrate to Amazon S3 account regional namespaces

Since its launch in 2006, Amazon S3 has used a global namespace where bucket names must be unique across all AWS accounts and AWS Regions. This design has served customers well at scale, but organizations managing multiple accounts and environments often encounter naming collisions. When a bucket is deleted, its name returns to the global […]

Building automated AWS Regional availability checks with Amazon S3

Every day, organizations expand into new markets, migrate critical workloads across geographies, and build systems that need to operate reliably in multiple locations. At the root of these efforts is a simple question: “What can I deploy, and where?” The answer shapes important architecture decisions, from which AWS Regions to expand into, to how you […]

Automatically decompress files in Amazon S3 using AWS Step Functions

Every day, AWS customers process millions of compressed files in Amazon S3, from small ZIP archives to multi-gigabyte datasets. While decompressing a single file is straightforward, processing thousands of files efficiently requires complex orchestration, error handling, and infrastructure management. Consider this scenario: Your organization receives over 10,000 compressed files daily from partners, ranging from 5 […]

Amazon S3 Object Lock

Applying Amazon S3 Object Lock at scale for petabytes of existing data

Organizations with petabytes of data in the cloud need a way to apply immutable storage protections to data that’s already been stored—whether for regulatory compliance or cyber resilience. Although you can enable write-once-read-many (WORM) controls for newly created storage, applying these protections to existing enterprise data at scale requires a systematic approach. Regulated industries have […]

Access Analyzer for Amazon S3

Advanced notice: Amazon S3 to disable the use of SSE-C encryption by default for all new buckets and select existing buckets in April 2026

Starting on April 6, 2026, we will be changing how server-side encryption with customer-provided keys (SSE-C) is enabled for Amazon S3 buckets. With this change, SSE-C will be disabled by default on all new S3 general purpose buckets. Furthermore, SSE-C will also be disabled for all existing buckets in Amazon Web Services (AWS) Accounts that […]

Amazon S3 Multi-Region Access Points

How to use Amazon S3 Multi-Region Access Points to streamline and reduce the cost of writing across AWS Regions

Large global organizations often struggle to efficiently manage data copies across different geographic regions when using distributed object storage services. Although several approaches exist for cross-region data writing, common solutions such as data replication or streaming can be costly and introduce latency issues. Many customers have core services deployed globally across multiple Amazon Web Services […]

Build intelligent ETL pipelines using AWS Model Context Protocol and Amazon Q

Data scientists and engineers spend hours writing complex data pipelines to extract, transform, and load (ETL) data from various sources into their data lakes for data integration and creating unified data models to build business insights. The process involves understanding the source and target systems, discovering schemas, mapping source and target, writing and testing ETL […]