AWS Storage Blog

Category: Amazon FSx for Lustre

AWS Backup provides centralized data protection across your AWS resources

Today, on AWS Storage Day 2020, we are excited to announce AWS Backup supports backups of Amazon FSx file systems and Windows workloads running on EC2, adding to the growing list of features to enhance your data protection. Amazon FSx customers may now enjoy the automation, ease-of-use, and managed compliance protection service that AWS Backup […]

Optimize price and performance with Amazon FSx for Lustre

Earlier this year, I blogged about the performance characteristics of Amazon FSx for Lustre persistent file systems. As with everything AWS, we never rest on our laurels and we continue to push the envelope and drive innovation. Recently, we launched a new lower-cost Hard Disk Drive (HDD) storage type for Amazon FSx for Lustre Persistent […]

How Qubole optimizes cost and performance by managing shuffle data

Ad hoc analytics, data exploration, data engineering, and machine learning (ML) workloads are often run at a massive scale and require significant computing power, which can be costly. Amazon EC2 Spot Instances, which enable you to use unused Amazon EC2 computing capacity in the AWS Cloud, offer up to 90% in savings over On-Demand Instances. […]

Automatically import Amazon S3 object updates into Amazon FSx for Lustre

Many enterprises and other AWS customers often store their datasets and build their data lakes on Amazon S3. Since Amazon FSx for Lustre is deeply integrated with S3, customers can create a new FSx for Lustre file system linked to their S3 bucket in minutes. FSx for Lustre file systems transparently present S3 objects as […]

Online Tech Talk July 27: Explore high-performance shared file storage for compute clusters

Don’t miss our AWS Online Storage Tech Talk on July 27, where an AWS expert covers reducing the cost of running compute-intensive workloads through high-performance storage using Amazon FSx for Lustre. This Tech Talk is at 11:00 AM – 12:00 PM PT (2:00 PM – 3:00 PM ET). Fast parallel file systems can help your […]

Using high-performance storage for machine learning workloads on Kubernetes

Organizations are modernizing their applications by adopting containers and microservices-based architectures. Many customers are deploying high-performance workloads on containers to power microservices architecture, and require access to low latency and high throughput shared storage from these containers. Because containers are transient in nature, these long-running applications require data to be stored in durable storage. Amazon FSx […]

Persistent storage for high-performance workloads using Amazon FSx for Lustre

High-performance file systems are often divided into two types: scratch and persistent. Scratch file systems provide temporary storage with high-performance characteristics such as submillisecond latency, up to hundreds of gigabytes per second of throughput, and millions of IOPS for short-term workloads. By contrast, persistent file systems are designed to combine the performance levels of their […]

Using Amazon FSx for Lustre for Genomics Workflows on AWS

Genomics datasets are getting larger year-over-year. Combining data from research initiatives across the globe, and having the ability to process it quickly, has been identified by large bioinformatics and genomics communities as an enabling mechanism for making significant scientific discoveries. Collaborating on a global scale requires data storage solutions that are globally accessible, highly available, […]

New enhancements for moving data between Amazon FSx for Lustre and Amazon S3

Introduction to Amazon FSx for Lustre Amazon FSx for Lustre is a high performance file system optimized for workloads such as machine learning, high performance computing, video processing, financial modeling, electronic design automation, and analytics. Amazon FSx works natively with Amazon S3, making it easy for you to process cloud data sets with high performance […]