AWS HPC Blog

Building a Scalable Predictive Modeling Framework in AWS – Part 3

In this final part of this three-part blog series on building predictive models at scale in AWS, we will use the synthetic dataset and the models generated in the previous post to showcase the model updating and sensitivity analysis capabilities of the aws-do-pm framework.

Read More

Building a Scalable Predictive Modeling Framework in AWS – Part 2

In the first part of this three-part blog series, we introduced the aws-do-pm framework for building predictive models at scale in AWS. In this blog, we showcase a sample application for predicting the life of batteries in a fleet of electric vehicles, using the aws-do-pm framework.

Read More

Building a Scalable Predictive Modeling Framework in AWS – Part 1

Predictive models have powered the design and analysis of real-world systems such as jet engines, automobiles, and powerplants for decades. These models are used to provide insights on system performance and to run simulations, at a fraction of the cost compared to experiments with physical hardware. In this first post of three, we described the motivation and general architecture of the open-source aws-do-pm framework project for building predictive models at scale in AWS.

Read More

Running large-scale CFD fire simulations on AWS for Amazon.com

In this blog post, we discuss the AWS solution that Amazon’s construction division used to conduct large-scale CFD fire simulations as part of their Fire Strategy solutions to demonstrate safety and fire mitigation strategies. We outline the five key steps taken that resulted in simulation times that were 15-20x faster than previous on-premises architectures, reducing the time to complete from up to twenty-one days to less than one day.

Read More
Expanded filesystems support in AWS ParallelCluster 3.2

Expanded filesystems support in AWS ParallelCluster 3.2

AWS ParallelCluster version 3.2 introduces support for two new Amazon FSx filesystem types (NetApp ONTAP and OpenZFS). It also lifts the limit on the number of filesystem mounts you can have on your cluster. We’ll show you how, and help you with the details for getting this going right away.

Read More
Memory aware scheduling with Slurm is now available in AWS ParallelCluster

Slurm-based memory-aware scheduling in AWS ParallelCluster 3.2

AWS ParallelCluster version 3.2 now supports memory-aware scheduling in Slurm to give you control over the placement of jobs with specific memory requirements. In this blog post, we’ll show you how it works, and explain why this will be really useful to people with memory-hungry workloads.

Read More

Call for participation: RADIUSS Tutorial Series

Lawrence Livermore National Laboratory (LLNL) and AWS are joining forces to provide a training opportunity for emerging HPC tools and application. RADIUSS (Rapid Application Development via an Institutional Universal Software Stack) is a broad suite of open-source software projects originating from LLNL. Together we are hosting a tutorial series to give attendees hands-on experience with these cutting-edge technologies. Find out how to participate in these events in this blog post.

Read More

Analyzing Genomic Data using Amazon Genomics CLI and Amazon SageMaker

In this blog post, we demonstrate how to leverage the AWS Genomics Command line and Amazon SageMaker to analyze large-scale exome sequences and derive meaningful insights. We use the bioinformatics workflow manager Nextflow, it’s open source library of pipelines, NF-Core, and AWS Batch.

Read More
Blender on Batch

Efficient and cost-effective rendering pipelines with Blender and AWS Batch

This blog post explains how to run parallel rendering workloads and produce an animation in a cost and time effective way using AWS Batch and AWS Step Functions. AWS Batch manages the rendering jobs on Amazon Elastic Compute Cloud (Amazon EC2), and AWS Step Functions coordinates the dependencies across the individual steps of the rendering workflow. Additionally, Amazon EC2 Spot instances can be used to reduce compute costs by up to 90% compared to On-Demand prices.

Read More