Category: Amazon Simple Storage Service (S3)

Financial institutions need to share files securely with partner banks while maintaining their existing file-based workflows. Many run applications using standard file systems such as Windows SMB or Linux NFS, but when sharing these files with external partners through SSH File Transfer Protocol (SFTP), they traditionally had to copy data between file systems and SFTP […]

Tavily is an AI infrastructure company building the web access layer for agents and large language models (LLMs). The company provides developer-friendly APIs that enable real-time, structured retrieval from the web. Their mission is to make information instantly accessible for intelligent systems, and they’re trusted by thousands of leading research, commercial AI teams, and enterprises […]

I’ve been working with data storage technologies for more than 20 years. Over that time, storage technologies have matured to keep up with exponential data growth. Solid state storage replaced spinning disk for the most critical workloads. Drive capacities grew from tens of gigabytes to tens of terabytes. On March 14, 2006, in the middle […]

Hi, I’m Buckets. The official AWS Storage mascot, self-appointed guardian of all things durable, scalable, and correctly permissioned. I’ve attended every re:Invent since 2006, and I have never once missed a storage breakout session. Not even the one scheduled at 8am on a Friday. Some call it dedication. I call it having strong consistency. re:Invent […]

In today’s cloud-first environment, protecting file storage systems against malware threats is a critical component of any robust security strategy. Amazon FSx for Windows File Server (FSx for Windows) delivers enterprise-grade Windows file storage in AWS, combining the reliability of fully managed services with native Windows file sharing capabilities. Built on Microsoft Windows file system […]

Organizations are rapidly adopting Apache Iceberg for their data lakes because it supports petabyte-scale growth and performance with the flexibility to evolve schemas and partitions without costly rewrites. Its architecture enables modern data lake management via features like time travel and incremental processing. However, managing Iceberg datasets efficiently can become a challenge over time as […]

Every day, AWS customers process millions of compressed files in Amazon S3, from small ZIP archives to multi-gigabyte datasets. While decompressing a single file is straightforward, processing thousands of files efficiently requires complex orchestration, error handling, and infrastructure management. Consider this scenario: Your organization receives over 10,000 compressed files daily from partners, ranging from 5 […]

Organizations with petabytes of data in the cloud need a way to apply immutable storage protections to data that’s already been stored—whether for regulatory compliance or cyber resilience. Although you can enable write-once-read-many (WORM) controls for newly created storage, applying these protections to existing enterprise data at scale requires a systematic approach. Regulated industries have […]

Organizations frequently upload compressed TAR files to Amazon S3 for efficient data transfer, but downstream applications often need extracted files for processing. Although AWS Glue excels at processing splittable files across worker nodes, TAR files need single-node processing, traditionally forcing teams to manually provision servers, monitor extraction jobs, and manage resource cleanup. This post demonstrates […]

As enterprises scale their adoption of generative AI, one of the biggest technical challenges is connecting AI applications to the right data and making that data fast, accessible, and secure. AI agents are transforming industries through applications like customer support automation, personalized e-commerce recommendations, and research assistance in financial services and healthcare. These applications require […]