AWS Storage Blog

Category: Thought Leadership

S3 cost optimization

Enhance savings for read-heavy workloads with Amazon S3 Bucket Keys

Organizations continue to grow their data lakes in the cloud as they build out new and innovative analytics, machine-learning, and generative AI workloads. At the same time, these workloads often access data that requires compliance with stringent data security and privacy standards. These compliance frameworks typically specify additional requirements for encryption at-rest, which leads customers […]

Amazon S3 Batch Operations featured image

Streamline data management at scale by automating the creation of Amazon S3 Batch Operations jobs

Over time, Enterprises may need to undertake operations or make modifications to their data as part of general data management, to address changing business needs, or to comply with evolving data-management regulations and best practices. As datasets being generated, stored, and analyzed continue to grow exponentially, the need for simplified, scalable, and reproduceable data management […]

Amazon S3 Express One Zone delivers cost and performance gains for ChaosSearch customers

ChaosSearch is an Amazon S3-native database built on a serverless, stateless compute architecture within AWS that delivers live search, SQL, and Generative AI analytics. At ChaosSearch, the speed and performance of our architecture is important to us and our customers because time to results is the difference between success and failure, and we rely on […]

Akridata accelerates processing of unstructured data with Amazon S3 Express One Zone

Deep learning processes often need to read full datasets, which are usually hundreds of gigabytes in size, before they can perform intelligent data processing. High data retrieval speed and low latency from storage are crucial for enterprises running these performance-critical workloads. Akridata, an AWS independent software vendor (ISV) partner, helps make artificial intelligence (AI)-assisted unstructured-data […]

lakeFS and Amazon S3 Express One Zone: Highly performant data version control for ML/AI

Machine learning presents a number of new challenges to data teams, calling for technology solutions that can support training and fine-tuning performance-critical workloads with high performance. Data version control is one of the facets of high-performing ML pipelines, as it allows efficient experimentation and full ML pipeline reproducibility at scale. lakeFS by Treeverse, an AWS […]

ClickHouse Cloud & Amazon S3 Express One Zone: Making a blazing fast analytical database even faster

ClickHouse is a columnar database management system (DBMS) designed for blazing-fast real-time analytics. It was built to address the needs of interactive analytical applications requiring up-to-the-second analytics. To do that, it must support real-time data ingestion at the rate of hundreds of millions of events per second and run complex analytical queries, such as filtering, […]

Amazon S3 featured image - new

Streamline data sharing and access control with Informatica Cloud Data Marketplace and Amazon S3 Access Grants

Organizations are modernizing their data lakes on Amazon Simple Storage Service (Amazon S3) to handle the ever-growing data volume and speed while meeting the demands of analytics, machine learning (ML), artificial intelligence (AI), and generative AI applications. To enable a data-driven culture and remain innovative, the data platform must allow for data-centric collaboration across business […]

Amazon S3 featured image - new

Accelerate Amazon S3 throughput with the AWS Common Runtime

Data is at the center of every machine learning pipeline. Whether pre-training foundation models (FMs), fine-tuning FMs with business-specific data, or serving inference queries, every step of the machine learning lifecycle needs low-cost, high-performance data storage to keep compute resources busy and performing useful work. Customers use Amazon Simple Storage Service (Amazon S3) to store training data […]

Amazon S3 featured image - new

How to enforce Amazon S3 Access Grants with Immuta

Amazon Simple Storage Service (Amazon S3) is the most popular object storage platform for modern data lakes. Organizations today evolved to adopt a lake house architecture that combines the scalability and cost effectiveness of data lakes with the performance and ease-of-use of data warehouses. Likewise, Amazon S3 plays an increasingly important role as the foundational […]

How PingCAP transformed TiDB into a serverless DBaaS using Amazon S3 and Amazon EBS

PingCAP, an AWS Partner Network (APN) Partner, is the company behind TiDB, an advanced open-source, distributed SQL database for building modern applications. TiDB is widely used and trusted by technologists around the world. In July 2023, PingCAP released TiDB Serverless, a fully managed, autonomous DBaaS offering of TiDB. However, based on TiDB’s existing architecture, PingCAP […]