AWS Storage Blog

Tag: Amazon FSx for Lustre

Using high-performance storage for machine learning workloads on Kubernetes

Organizations are modernizing their applications by adopting containers and microservices-based architectures. Many customers are deploying high-performance workloads on containers to power microservices architecture, and require access to low latency and high throughput shared storage from these containers. Because containers are transient in nature, these long-running applications require data to be stored in durable storage. Amazon FSx […]

Persistent storage for high-performance workloads using Amazon FSx for Lustre

High-performance file systems are often divided into two types: scratch and persistent. Scratch file systems provide temporary storage with high-performance characteristics such as submillisecond latency, up to hundreds of gigabytes per second of throughput, and millions of IOPS for short-term workloads. By contrast, persistent file systems are designed to combine the performance levels of their […]

Using Amazon FSx for Lustre for Genomics Workflows on AWS

Genomics datasets are getting larger year-over-year. Combining data from research initiatives across the globe, and having the ability to process it quickly, has been identified by large bioinformatics and genomics communities as an enabling mechanism for making significant scientific discoveries. Collaborating on a global scale requires data storage solutions that are globally accessible, highly available, […]

New enhancements for moving data between Amazon FSx for Lustre and Amazon S3

Introduction to Amazon FSx for Lustre Amazon FSx for Lustre is a high performance file system optimized for workloads such as machine learning, high performance computing, video processing, financial modeling, electronic design automation, and analytics. Amazon FSx works natively with Amazon S3, making it easy for you to process cloud data sets with high performance […]

Explore AWS file storage solutions at re:Invent 2019

UPDATE: The AWS re:Invent links in this post are outdated. Please visit the AWS Events channel on YouTube for a complete on-demand playlist of AWS Storage sessions at AWS re:Invent 2019. In today’s business environment, data is the foundation for innovation and transformation. Using data the right way can literally propel a business to becoming […]

New on the Machine Learning blog: Speed up training on Amazon SageMaker using Amazon FSx for Lustre and Amazon EFS file systems

Deploying analytics applications and machine learning models requires storage that can scale in capacity and performance to handle workload demands with high throughput and low-latency file operations. A common use case we’re seeing centers around data science teams doing some form of analytics (e.g machine learning, genomics). AWS offers two scalable, durable, highly available file […]

AWS Loft

Free AWS Loft Events: Attend “AWS Storage Days” in San Francisco or NYC

As we gear up for AWS re:Invent 2019 December 2 – 6, we want to ensure you are up to speed on the full portfolio of AWS storage services. In San Francisco September 10 – 11 and in NYC September 24- 25, we will be conducting ‘AWS Storage Days’ at the AWS Loft locations. These […]

Building an HPC cluster with AWS ParallelCluster and Amazon FSx for Lustre

High performance computing (HPC) customers love the breadth of services offered by AWS and the flexibility offered by the cloud to address their computational challenges. AWS provides you with the opportunity to innovate quickly and accelerate your workflow thanks to a virtually unlimited capacity. For more information, see the following posts: Natural language processing at Clemson […]

Getting started: Training resources for storage on AWS

With a wide variety of storage and data transfer and migration services, understanding where to start when considering AWS for your data storage can be a time consuming task. Where do you start? What resources are available to help guide you through unfamiliar territory? Where can you go if you need additional help? Whether you’ve […]