AWS Storage Blog

Organizations that receive files from external partners through SFTP face a persistent operational challenge: routing each file to the correct downstream system. Invoices, contracts, images, CSVs, and reports all arrive in a single landing zone, and each requires a different destination. The traditional approach—pattern-matching on file names with regular expressions—is inherently fragile. It relies on […]

Semiconductor verification teams run thousands of simulation jobs every night using Electronic Design Automation (EDA) tools. A large verification environment can generate logs from 100,000 or more test executions per night. A single regression cycle produces simulation logs, compilation logs, and scheduler logs. For a regression with dozens of failures, manual triage typically takes 45–60 […]

As more organizations move to remote and distributed work, builders and development teams face growing challenges in sharing files securely. Without a central file system, these teams deal with version control problems, unclear access permissions, and security risks from scattered storage locations. Central IT becomes a bottleneck when handling every permission change, yet granting full […]

Immutability is a foundational data protection control, protecting data in place against unintended changes and deletions by authorized users, and changes by unauthorized users. In many cases, however, it isn’t clear at the time data is written how long it needs to stay immutable, or when that period should begin. A signed contract might need […]

Customer Story | Amazon S3 Tables – Learn how Tubular Labs (part of Chartbeat Inc.) reclaimed 50% of engineering capacity by replacing fragile, file-based data pipelines with a Common Pipeline Runtime (CPR) built on Apache Iceberg and Amazon S3 Tables. By Tubular Labs Engineering (part of Chartbeat Inc.), in collaboration with the AWS Solution Architecture team […]

Enterprise storage underpins every critical workload in a VMware environment. It’s not just capacity, it’s the operational backbone that delivers automatic failover, instant snapshots, writable clones, inline deduplication, and multi-protocol access that production applications depend on every day. When the time comes to migrate these workloads to the cloud, teams expect those same capabilities on […]

As your Spark datasets grow, storage latency often becomes the constraint, impeding application performance. Query runtimes stretch, and the bottleneck shifts from compute to how fast each node can read from Amazon Simple Storage Service (Amazon S3). We benchmarked this directly on Amazon EMR with TPC-DS at 3 TB scale. On an 8-node Graviton4 cluster […]

Organizations that replicate data across storage locations for disaster recovery, compliance, or analytics need a way to detect when replication stalls, because without early warning, unreplicated objects accumulate silently and drive unexpected storage costs. Most teams rely on automated retention policies to clean up the source copy after a defined period, but when replication breaks […]

AWS Backup centralizes data protection in your organization across hundreds of accounts and AWS Regions for more than 20 AWS services. At that scale, some jobs fail, and triaging each failure manually can take days. Most failures trace back to something small, such as a role missing permission, an encryption key policy blocking the backup […]

Nearmap captures high-resolution aerial imagery across populated areas of the United States, Canada, Australia, and New Zealand several times a year, at resolutions as fine as 1.5 inches. More than 10,000 customers globally use these images to assess insurance portfolios, size solar arrays, and track construction sites. Since 2007, Nearmap has completed more than 35,000 […]