High Performance Computing (HPC) allows scientists and engineers to solve complex science, engineering, and business problems using applications that require high bandwidth, low latency networking, and very high compute capabilities. AWS allows you to increase the speed of research by running high performance computing in the cloud and to reduce costs by providing Cluster Compute or Cluster GPU servers on-demand without large capital investments. You have access to a full-bisection, high bandwidth network for tightly-coupled, IO-intensive workloads, which enables you to scale out across thousands of cores for throughput-oriented applications.
Amazon's C3 Cluster Compute Instance Type ranks 76th in Top 500 Supercomputers
AWS created the cluster with Intel Xeon 2.8GHz processors with 26,496 cores and ran at 484.2 TFlop/s.
Tradeworx President, Arzhang Kamerai, presents during the keynote at re:Invent on how they used Amazon EC2 to help the SEC analyze 3 billion records in 2.8 seconds.
“Pfizer did not have to invest in additional hardware and software, which is only used during peak loads; that savings allowed for investments in other WRD activities.”
Dr. Michael Miller, Head of HPC for Worldwide R&D at Pfizer
High Performance Computing (HPC) on Amazon Elastic Compute Cloud (EC2) is enabled by the Cluster Compute-optimized and GPU instance types (virtual machines). You can use them just like other EC2 instances, but they also have been specifically engineered to provide high performance networking and you can scale to tens of thousands of instances on-demand.
At re:Invent 2014, AWS pre-announced the latest generation of compute-optimized Amazon EC2 instance type, the C4, which is based on the Intel Xeon E5-2666 v3 (code name Haswell) processor. This custom processor, designed specifically for Amazon EC2, runs at a base speed of 2.9 GHz, and can achieve clock speeds as high as 3.5 GHz with Intel Turbo boost. These instances are designed to deliver the highest level of processor performance on EC2.
GPU instances are ideally suited for graphical high performance computing applications, including 3D modeling and simulation; and compute workloads, including computational chemistry, financial analysis, engineering design, and general purpose GPU computing.
AWS GPU instances provide access to NVIDIA GPUs each with up to 1,536 CUDA cores and 4GB of video memory. With the latest driver releases, these GPUs provide support for OpenGL, DirectX, CUDA, OpenCL, and the GRID SDK.
Cluster instances can be launched within a Placement Group. All instances launched within a Placement Group have low latency, full bisection, 10 Gbps bandwidth between instances. Like many other Amazon EC2 resources, Placement Groups are dynamic and are elastically scalable as needed. You can also connect multiple Placement Groups to create very large high performance computing clusters for massively parallel processing.
AWS currently supports enhanced networking capabilities using SR-IOV (Single Root I/O Virtualization) for the C3 and I2 instance types. SR-IOV is a method of device virtualization that provides higher I/O performance and lower CPU utilization compared to traditional implementations. For supported Amazon EC2 instances, this feature provides higher packet per second (PPS) performance, lower inter-instance latencies, and very low network jitter.
Data has gravity; as datasets grow larger it becomes easier to move the compute closer to the data to reduce latency and increase throughput. With AWS big data storage and database services, such Amazon S3, Amazon Redshift, Amazon DynamoDB, and Amazon RDS, you have the perfect place to host your data for your high performance computing cluster. Furthermore, with Amazon Elastic Block Store (EBS) you can create large scale parallel filesystems to meet the high volume, performance, and throughput requirements of your HPC workload.
You can save time and money by leveraging Spot Instances for your HPC workloads. Spot Instances is a pricing model that enables you to bid on unused Amazon EC2 capacity at whatever price you choose. When your bid exceeds the Spot price, you gain access to the available Spot Instances and run as long as the bid exceeds the Spot Price. Historically, the Spot price has been 50% to 93% lower than the on-demand price.
AWS Marketplace is an online store that provides an easy way for developers and IT professionals to discover and use software to run in the AWS Cloud. You can find a selection of high performance computing software ready to run in your cluster, such as the Univa Grid Engine resource management system or the Intel Lustre HPC file system, with just a few clicks directly from the AWS Marketplace.