Get Started with the Project

6 Steps  |  60 Minutes

Linux_HPC_Cluster_aws-project
architectural-diagram
architecture-diagram-elastic-hpc-cluster

Amazon Web Services (AWS) makes it easy for you to access on-demand and scalable compute capacity to power your High Performance Computing (HPC) workloads. Examples of HPC workloads customers have run on AWS include CFD simulations, computational chemistry workloads, genomics analysis, weather forecasting, and more. In this project, you will deploy an HPC cluster using AWS ParallelCluster, an open-source solution that enables you to rapidly provision and manage HPC clusters in the cloud. These cloud-based clusters deliver many similar characteristics of traditional on-premises clusters while leveraging uniquely cloud-based capabilities. For example, AWS ParallelCluster clusters are fully elastic, meaning that compute nodes are only launched when there is work to be done. 

What you'll accomplish:

Install AWS ParallelCluster, an open-source tool that enables you to create and configure an elastic HPC cluster on Amazon Elastic Compute Cloud (Amazon EC2). The AWS ParallelCluster source code is published on GitHub and documentation is available at AWS Documentation.

Configure and launch an elastic HPC cluster on Amazon EC2 using AWS ParallelCluster. You can customize your first HPC cluster by selecting the deployment region, Amazon Virtual Private Cloud (Amazon VPC), instance type, volume type, minimum and maximum number of compute nodes, and preferred batch scheduler.

Submit and run a parallel MPI job. A simple parallel “hello world” code is compiled and submitted to the scheduler to launch the job, enable the compute nodes, and execute the MPI job.

Take an EBS snapshot of the shared disk for reusability with Amazon Elastic Block Store (Amazon EBS). A snapshot of the shared disk is created, the cluster is shut down, and a new cluster is launched with the “hello world” software pre-installed and ready to run.

What you'll need before starting:

An AWS Account: You will need an AWS account to set up your HPC environment. Sign up for AWS.

Skill level: Previous experience with HPC scheduling software is recommended to successfully complete this project.

AWS Experience: Familiarity with Amazon EC2, Amazon Simple Storage Service (Amazon S3), and Amazon VPC is recommended to successfully complete this project.

Cost Estimate:

The total cost to complete this Linux HPC project is estimated to be less than $1. This cost assumes that you accept the recommended configurations and that you terminate all resources within 24 hours of completing the project. 

Your use case may require different configurations that will impact your bill. For example. the AWS ParallelCluster config file calls out instance type, maximum queue size, and EBS volume sizes. When AWS ParallelCluster is launched with customized configuration settings, the costs will vary depending on set up and usage. Typically, you would launch a cluster only for the time period you need to use it, to minimize the cost of running large workloads.

To see a breakdown of the services used and their associated costs, see Services Used and Costs.

Get Started with the Project

This paper describes a range of HPC applications that are running today on AWS and provides best practices.

Learn more about HPC capabilities and find resources to help get you started

Need more resources to get started with AWS? Visit the Getting Started Resource Center to learn more.