AWS Storage Blog
Author: Ran Pergamin
Ran Pergamin is a Senior Specialist Solution Architect – Data and AI. He is focusing on solving large-scale storage problems. These days, he is passionate about Apache Iceberg, open table formats, data lakes, and S3 Vectors. In his free time, he is a CrossFit advocate.
How Tavily reduced AI search caching costs by 95% with Amazon S3 Express One Zone
Tavily is an AI infrastructure company building the web access layer for agents and large language models (LLMs). The company provides developer-friendly APIs that enable real-time, structured retrieval from the web. Their mission is to make information instantly accessible for intelligent systems, and they’re trusted by thousands of leading research, commercial AI teams, and enterprises […]
Optimize data management on S3 Tables with Intelligent-Tiering
Organizations are rapidly adopting Apache Iceberg for their data lakes because it supports petabyte-scale growth and performance with the flexibility to evolve schemas and partitions without costly rewrites. Its architecture enables modern data lake management via features like time travel and incremental processing. However, managing Iceberg datasets efficiently can become a challenge over time as […]
Mountpoint for Amazon S3 CSI driver v2: Accelerated performance and improved resource usage for Kubernetes workloads
Amazon S3 is the best place to build data lakes because of its durability, availability, scalability, and security. In 2023, we introduced Mountpoint for Amazon S3, an open source file client that allows Linux-based applications to access S3 objects through a file API. Shortly after, we took this one step further with the Mountpoint for […]
Design patterns for multi-tenant access control on Amazon S3
Large organizations and software as a service (SaaS) platforms often share storage resources across multiple users, groups, or tenants. The design pattern chosen to implement this shared storage can significantly impact how access permissions are managed at scale. This decision is key because it directly affects platforms’ security and ease of scale. A well thought […]
Configuring the auto-expansion of Amazon FSx for OpenZFS with Amazon CloudWatch and AWS Lambda
Today’s demanding workloads such as database, rendering farm, analytics and ML workloads, have increasingly demanding IO requirements. These workloads need a reliable storage infrastructure that provides sufficient storage capacity, IOPS, and throughput. As customers move more workloads to the cloud, they want to benefit from the agility and performance capabilities of the cloud as their […]
5 ways to simplify backup plans using AWS Backup resource assignment rules
Prior to the announcement of new resource assignment capabilities in AWS Backup, customers could assign resources to a backup plan in two ways. They could either select a specific resource using its resource ID or define a specific selection tag, which helps the service identify resources to be backed up by the backup plan. While […]