AWS Storage Blog

Category: Amazon FSx for Lustre

Optimize price and performance with Amazon FSx for Lustre

Earlier this year, I blogged about the performance characteristics of Amazon FSx for Lustre persistent file systems. As with everything AWS, we never rest on our laurels and we continue to push the envelope and drive innovation. Recently, we launched a new lower-cost Hard Disk Drive (HDD) storage type for Amazon FSx for Lustre Persistent […]

Read More

How Qubole optimizes cost and performance by managing shuffle data

Ad hoc analytics, data exploration, data engineering, and machine learning (ML) workloads are often run at a massive scale and require significant computing power, which can be costly. Amazon EC2 Spot Instances, which enable you to use unused Amazon EC2 computing capacity in the AWS Cloud, offer up to 90% in savings over On-Demand Instances. […]

Read More

Automatically import Amazon S3 object updates into Amazon FSx for Lustre

Many enterprises and other AWS customers often store their datasets and build their data lakes on Amazon S3. Since Amazon FSx for Lustre is deeply integrated with S3, customers can create a new FSx for Lustre file system linked to their S3 bucket in minutes. FSx for Lustre file systems transparently present S3 objects as […]

Read More

Online Tech Talk July 27: Explore high-performance shared file storage for compute clusters

Don’t miss our AWS Online Storage Tech Talk on July 27, where an AWS expert covers reducing the cost of running compute-intensive workloads through high-performance storage using Amazon FSx for Lustre. This Tech Talk is at 11:00 AM – 12:00 PM PT (2:00 PM – 3:00 PM ET). Fast parallel file systems can help your […]

Read More

Using high-performance storage for machine learning workloads on Kubernetes

Organizations are modernizing their applications by adopting containers and microservices-based architectures. Many customers are deploying high-performance workloads on containers to power microservices architecture, and require access to low latency and high throughput shared storage from these containers. Because containers are transient in nature, these long-running applications require data to be stored in durable storage. Amazon FSx […]

Read More

Persistent-50 high-performance file system packs a punch!

Like many things in life, the more choices you have the harder it is to make a decision. While running workloads in the cloud does make some decisions easier, it doesn’t eliminate them altogether, and there are still choices you must make on how to best architect for the cloud. These decisions now center more […]

Read More

Persistent storage for high-performance workloads using Amazon FSx for Lustre

High-performance file systems are often divided into two types: scratch and persistent. Scratch file systems provide temporary storage with high-performance characteristics such as submillisecond latency, up to hundreds of gigabytes per second of throughput, and millions of IOPS for short-term workloads. By contrast, persistent file systems are designed to combine the performance levels of their […]

Read More

Using Amazon FSx for Lustre for Genomics Workflows on AWS

Genomics datasets are getting larger year-over-year. Combining data from research initiatives across the globe, and having the ability to process it quickly, has been identified by large bioinformatics and genomics communities as an enabling mechanism for making significant scientific discoveries. Collaborating on a global scale requires data storage solutions that are globally accessible, highly available, […]

Read More

New enhancements for moving data between Amazon FSx for Lustre and Amazon S3

Introduction to Amazon FSx for Lustre Amazon FSx for Lustre is a high performance file system optimized for workloads such as machine learning, high performance computing, video processing, financial modeling, electronic design automation, and analytics. Amazon FSx works natively with Amazon S3, making it easy for you to process cloud data sets with high performance […]

Read More

Explore AWS file storage solutions at re:Invent 2019

In today’s business environment, data is the foundation for innovation and transformation. Using data the right way can literally propel a business to becoming a market leader. So storing data, protecting data, and preserving data is paramount for every business. This year at re:Invent, we’ll have more than 150 breakout sessions, workshops, chalk talks, and builders […]

Read More