AWS HPC Blog
Tag: Amazon FSx for Lustre
Build and deploy a 1 TB/s file system in under an hour
Want to set up a high-speed shared file system for your #HPC or #AI workloads in under an hour? Learn how with this new blog post.
Renewable energy transition: examining the impacts of wind energy through simulation
As we move towards a greener future, understanding wind energy’s climate impacts is key. Check out this blog post by our friends at Whiffle, to learn how large-scale simulations reveal wind power’s effect on our atmosphere.
Protein language model training with NVIDIA BioNeMo framework on AWS ParallelCluster
In this new post, we discuss pre-training ESM-1nv for protein language modeling with NVIDIA BioNeMo on AWS. Learn how you can efficiently deploy and customize generative models like ESM-1nv on GPU clusters with ParallelCluster. Whether you’re studying protein sequences, predicting properties, or discovering new therapeutics, this post has tips to accelerate your protein AI workloads on the cloud.
Introducing a community recipe library for HPC infrastructure on AWS
Today we’re showing you our community library of HPC Recipes for AWS. It’s a public repo @github that will help you achieve feature-rich, reliable HPC deployments ready to run your workloads no matter where you’re starting from.
Multiple Availability Zones now supported in AWS ParallelCluster 3.4
In AWS ParallelCluster 3.4, you can now build HPC clusters that span multiple Amazon EC2 Availability Zones. In this post, we describe how the new feature works, how to use it, and some implications for cluster design that it raises.
Second generation EFA: improving HPC and ML application performance in the cloud
Since launch, EFA has seen continuous improvements in performance. In this post, we talk about our 2nd generation of EFA, which takes another step in improving Machine Learning and High Performance Computing in the Cloud.
Expanded filesystems support in AWS ParallelCluster 3.2
AWS ParallelCluster version 3.2 introduces support for two new Amazon FSx filesystem types (NetApp ONTAP and OpenZFS). It also lifts the limit on the number of filesystem mounts you can have on your cluster. We’ll show you how, and help you with the details for getting this going right away.
How Thermo Fisher Scientific Accelerated Cryo-EM using AWS ParallelCluster
In this blog post, we’ll walk you through the process of building a successful Cryo-EM benchmarking pilot using AWS ParallelCluster, Amazon FSx for Lustre, and cryoSPARC (from Structura Biotechnology) and explain some of our design decisions along the way.
Scalable and Cost-Effective Batch Processing for ML workloads with AWS Batch and Amazon FSx
Batch processing is a common need across varied machine learning use cases such as video production, financial modeling, drug discovery, or genomic research. The elasticity of the cloud provides efficient ways to scale and simplify batch processing workloads while cutting costs. In this post, you’ll learn a scalable and cost-effective approach to configure AWS Batch Array jobs to process datasets that are stored on Amazon S3 and presented to compute instances with Amazon FSx for Lustre.