High Performance Computing (HPC) on AWS

High Performance Computing (HPC) allows scientists and engineers to solve complex science, engineering and business problems using applications that require high bandwidth, low latency networking, and very high compute capabilities. Typically, scientists and engineers must wait in long queues to access shared clusters or acquire expensive hardware systems. Using Amazon EC2 Cluster instances, customers can expedite their HPC workloads on elastic resources as needed and save money by choosing from low-cost pricing models that match utilization needs. Customers can choose from Cluster Compute or Cluster GPU instances within a full-bisection high bandwidth network for tightly-coupled and IO-intensive workloads or scale out across thousands of cores for throughput-oriented applications.

Today, AWS customers run a variety of HPC applications on these instances including Computer Aided Engineering, molecular modeling, genome analysis, and numerical modeling across many industries including Biopharma, Oil and Gas, Financial Services and Manufacturing. In addition, academic researchers are leveraging Amazon EC2 Cluster instances to perform research in physics, chemistry, biology, computer science, and materials science.

What's New
New Cluster Compute Instance The Cluster Compute Eight Extra Large instance provides supercomputing class performance with all the advantages of Amazon EC2
Featured Case Study: AeroDynamic Solutions The U.S. Air Force Research Laboratory and AeroDynamic Solutions (ADS) used Amazon EC2 to devise an effective design simulation solution. More HPC Case Studies.
Featured Tutorial: How to Launch a Cluster on Spot Chris Dagdigian from BioTeam shows how to start a cluster from scratch in 10-15 minutes using StarCluster and Amazon EC2 Spot Instances.

Benefits

AWS offers a number of key benefits that enable customers to run HPC applications to meet critical research, design and business needs.

Low Cost: Customers can eliminate the cost and complexity of procuring, configuring and operating HPC clusters with low, pay-as-you-go pricing. Further, you can optimize costs by leveraging one of several pricing models: On Demand, Reserved or Spot Instances.
Elasticity: You can add and remove compute resources to meet the size and time requirements for your workloads.
Run Jobs Anytime, Anywhere: You can launch compute jobs using simple APIs or management tools and automate workflows for maximum efficiency and scalability. You can increase your speed of innovation by accessing compute resources in minutes instead of spending time in queues.

Solution Overview

HPC on Amazon EC2 is enabled by the Cluster family of instance types. Cluster Compute and Cluster GPU instances can be used just like any other Amazon EC2 instance but also offer the following features optimized for HPC applications:

Cluster instances can be launched within a Placement Group. All instances launched within a Placement Group have low latency, full bisection 10 Gbps bandwidth between instances. Like many other Amazon EC2 resources, Placement Groups are dynamic and can be resized if required. You can also connect multiple Placement Groups to create very large clusters for massively parallel processing.
Cluster Compute and Cluster GPU instances specify the underlying processor architecture in their definition to enable developers to tune their applications by compiling for that specific processor architecture in order to achieve optimal performance.
Cluster GPU instances allow customers to take advantage of the parallel performance of NVidia Tesla GPUs using the CUDA and OpenCL programming models for GPGPU computing.

You can read more about Cluster Instances and other instance types on the Amazon EC2 Instance Types page.

Performance at Scale

By leveraging the advanced networking and high computational capabilities of Amazon EC2 Cluster instances, customers can provision clusters that can give them supercomputing class performance without the need to build and operate their own HPC facility. For example, a 1064 instance (17024 cores) cluster of cc2.8xlarge instances was able to achieve 240.09 TeraFLOPS for the High Performance Linpack benchmark, placing the cluster at #42 in the November 2011 Top500 list.

Optimizing For Time and Cost

One of the key advantages of Amazon EC2 is the ability to optimize resources without being limited by access to a fixed-size cluster. You can launch multiple clusters concurrently without the need to submit jobs to a queue. You can choose between running a single large cluster or many smaller clusters concurrently to address application scaling limits.

You can also optimize on cost by choosing a pricing model appropriate for your applications and utilization profile. Spot Instances provide a way to receive significant price savings by bidding on unused Amazon EC2 capacity. Customers whose bids exceed the Spot price gain access to available instances and run as long as the bid exceeds the Spot Price. Learn more about Spot Instances for Scientific Computing.

Getting Started

Build an Application
- Launch a 64 core HPC cluster on Amazon EC2: In this video, AWS Technology Evangelist Matt Wood demonstrates how to build an 8-node cluster and run a molecular dynamics simulation using NAMD.
- Launch your application: Amazon EC2 makes it easy for you to build and customize Amazon Machine Images (AMIs) with the software that you need. You can also use AWS CloudFormation to construct and share full stack templates. You can also begin building your application by accessing our resources webpage.
Leverage a Vibrant HPC Ecosystem

For HPC applications, you may want to consider using cluster configuration and management tools and application optimization tools from the following supporting solution providers:

Cycle Computing Univa StackIQ

Platform Computing Adaptive Computing Bright Computing

Intel

Alternatively, a number of open source solutions enable customers to provision, launch and manage clusters on Amazon EC2, such as the following commonly used resource managers:
- MIT StarCluster: StarCluster is a utility for creating and managing general purpose computing clusters on Amazon EC2.
- CloudFlu: The CloudFlu library aims to overcome the entrance barrier for high performance parallel computing for engineers, specifically for the CFD application, OpenFOAM.
Contact an AWS Sales Representative

If you'd like to learn more about using Amazon Web Services to support your business, or if you have a licensing question about AWS offerings, let us know.