High Performance Computing (HPC) on AWS
High Performance Computing (HPC) allows scientists and engineers to solve complex science, engineering and business problems using applications that require high bandwidth, low latency networking, and very high compute capabilities. Typically, scientists and engineers must wait in long queues to access shared clusters or acquire expensive hardware systems. Using Amazon EC2 Cluster instances, customers can expedite their HPC workloads on elastic resources as needed and save money by choosing from low-cost pricing models that match utilization needs. Customers can choose from Cluster Compute or Cluster GPU instances within a full-bisection high bandwidth network for tightly-coupled and IO-intensive workloads or scale out across thousands of cores for throughput-oriented applications.
Today, AWS customers run a variety of HPC applications on these instances including Computer Aided Engineering, molecular modeling, genome analysis, and numerical modeling across many industries including Biopharma, Oil and Gas, Financial Services and Manufacturing. In addition, academic researchers are leveraging Amazon EC2 Cluster instances to perform research in physics, chemistry, biology, computer science, and materials science.
New Cluster Compute Instance
The Cluster Compute Eight Extra Large instance provides supercomputing class performance with all the advantages of Amazon EC2
Featured Tutorial: How to Launch a Cluster on Spot
Chris Dagdigian from BioTeam shows how to start a cluster from scratch in 10-15 minutes using StarCluster and Amazon EC2 Spot Instances.
AWS offers a number of key benefits that enable customers to run HPC applications to meet critical research, design and business needs.
- Low Cost: Customers can eliminate the cost and complexity of procuring, configuring and operating HPC clusters with low, pay-as-you-go pricing. Further, you can optimize costs by leveraging one of several pricing models: On Demand, Reserved or Spot Instances.
- Elasticity: You can add and remove compute resources to meet the size and time requirements for your workloads.
- Run Jobs Anytime, Anywhere: You can launch compute jobs using simple APIs or management tools and automate workflows for maximum efficiency and scalability. You can increase your speed of innovation by accessing compute resources in minutes instead of spending time in queues.
HPC on Amazon EC2 is enabled by the Cluster family of instance types. Cluster Compute and Cluster GPU instances can be used just like any other Amazon EC2 instance but also offer the following features optimized for HPC applications:
- Cluster instances can be launched within a Placement Group. All instances launched within a Placement Group have low latency, full bisection 10 Gbps bandwidth between instances. Like many other Amazon EC2 resources, Placement Groups are dynamic and can be resized if required. You can also connect multiple Placement Groups to create very large clusters for massively parallel processing.
- Cluster Compute and Cluster GPU instances specify the underlying processor architecture in their definition to enable developers to tune their applications by compiling for that specific processor architecture in order to achieve optimal performance.
- Cluster GPU instances allow customers to take advantage of the parallel performance of NVidia Tesla GPUs using the CUDA and OpenCL programming models for GPGPU computing.
You can read more about Cluster Instances and other instance types on the Amazon EC2 Instance Types page.
Performance at Scale
By leveraging the advanced networking and high computational capabilities of Amazon EC2 Cluster instances, customers can provision clusters that can give them supercomputing class performance without the need to build and operate their own HPC facility. For example, a 1064 instance (17024 cores) cluster of cc2.8xlarge instances was able to achieve 240.09 TeraFLOPS for the High Performance Linpack benchmark, placing the cluster at #42 in the November 2011 Top500 list.
Optimizing For Time and Cost
One of the key advantages of Amazon EC2 is the ability to optimize resources without being limited by access to a fixed-size cluster. You can launch multiple clusters concurrently without the need to submit jobs to a queue. You can choose between running a single large cluster or many smaller clusters concurrently to address application scaling limits.
You can also optimize on cost by choosing a pricing model appropriate for your applications and utilization profile. Spot Instances provide a way to receive significant price savings by bidding on unused Amazon EC2 capacity. Customers whose bids exceed the Spot price gain access to available instances and run as long as the bid exceeds the Spot Price. Learn more about Spot Instances for Scientific Computing.