Category: Amazon Simple Storage Service (S3)

Data is at the center of every machine learning pipeline. Whether pre-training foundation models (FMs), fine-tuning FMs with business-specific data, or serving inference queries, every step of the machine learning lifecycle needs low-cost, high-performance data storage to keep compute resources busy and performing useful work. Customers use Amazon Simple Storage Service (Amazon S3) to store training data […]

Amazon Simple Storage Service (Amazon S3) is the most popular object storage platform for modern data lakes. Organizations today evolved to adopt a lake house architecture that combines the scalability and cost effectiveness of data lakes with the performance and ease-of-use of data warehouses. Likewise, Amazon S3 plays an increasingly important role as the foundational […]

To adhere to the principle of least privilege, users define granular access to their Amazon Simple Storage Service (Amazon S3) data based on applications, personas, groups or organization units (OUs). This practice helps customers to mitigate the risk of unauthorized access, limiting potential damage in case of a security breach as employees only have access […]

Today, customers increasingly choose to store data for longer because they recognize its future value potential. Storing data longer, coupled with exponential data growth, has led to customers placing a greater emphasis on storage cost optimization and using cost-effective storage classes. However, a modern data archiving strategy not only calls for optimizing storage costs, but […]

Update 4/9/2025: The cross-account bucket policy in the blog has been updated. It was missing a required principal: “arn:aws:iam::accountID:role/AWS-Signed-In-Console-Role.” This omission causes an access denied error. As enterprises evolve their cloud governance practices, multiple teams working in separate accounts may need to share data. One team may oversee an enterprise data lake in one account, […]

Update (5/29/2025): On May 29, 2025, AWS DataSync launched Enhanced mode support for cross-cloud transfers. Enhanced mode simplifies data transfers between AWS and other clouds by removing the need for a DataSync agent. It also provides higher performance and scalability when compared to Basic mode. For more details, see the What’s New announcement or review the documentation for guidance […]

PingCAP, an AWS Partner Network (APN) Partner, is the company behind TiDB, an advanced open-source, distributed SQL database for building modern applications. TiDB is widely used and trusted by technologists around the world. In July 2023, PingCAP released TiDB Serverless, a fully managed, autonomous DBaaS offering of TiDB. However, based on TiDB’s existing architecture, PingCAP […]

Customers use data events to build applications to trigger and communicate between decoupled services. An event is a change in state of, or an update to, data. For certain applications, such as batch order processing or content management, customers may need to implement application logic to handle duplicate and out-of-order events. For these use cases, […]

Infrastructure as Code (IaC) involves managing IT infrastructure through code and automation tools to reduce manual management prone to errors, slow scaling, and overhead. For organizations implementing a hybrid cloud infrastructure, automation can ensure uniformity, scalability, and cost reduction while getting cloud resources provisioned efficiently. Automated provisioning and configuration enable organizations to adapt, innovate, and […]

SaaS providers with multi-tenant environments use cloud solutions to dynamically scale their workloads as customer demand increases. As their cloud footprint grows, having visibility into each end-customer’s storage consumption becomes important to distribute resources accordingly. An organization can use storage usage data per customer (tenant) to adjust its pricing model or better plan its budget. […]