AWS HPC Blog
Author: Colin Bridger
Colin is a Principal HPC Specialist with AWS and gained his HPC experience with networking On-Premises vendors. After working with end users and partners in verticals such as academic and climate research, CFD, FSI, and health care life sciences, he pivoted to focus on deployment of workloads in those industries on AWS cloud. He is passionate about deploying economic HPC at scale to advance research and scientific outcome.
Running a 3.2M vCPU HPC Workload on AWS with YellowDog
OMass Therapeutics, a biotechnology company identifying medicines against highly validated target ecosystems, used Yellowdog on AWS to analyze and screen 337 million compounds in 7 hours, a task which would have taken two months using an on-premises HPC cluster. YellowDog, based in Bristol in the UK, ran the drug discovery application on an extremely large, multi-region cluster in AWS with the AWS ‘pay-as-you-go’ pricing model. It provided a central, unified interface to monitor and manage AWS Region selection, compute provisioning, job allocation and execution. The entire workload completed in 65 minutes, enabling scientists to start work on analysis the same day, significantly accelerating the drug discovery process. In this post, we’ll discuss the AWS and YellowDog services we deployed, and the mechanisms used to scale to 3.2m vCPUs using multiple EC2 instance types across multiple regions in 33 minutes, running at a 95% utilization rate.