Q: What is an elastic HPC cluster?
An elastic cluster provides an HPC environment with the same “look and feel” of a conventional HPC cluster, but with the added benefit of being scalable: jobs are submitted to a queue and nodes spin up as needed, jobs are automatically launched as they become idle, and nodes are automatically shut down when no longer needed.
Q: What are the benefits of using Amazon Web Services for HPC?
AWS is elastic. Rather than pay a large upfront capital cost, one only pays for what they use. Launch a 1000 core cluster for only one day and pay for only one day. Further, a cloud-based cluster allows for continual access to the latest hardware and the opportunity to customize each cluster to the application that will run on the hardware. Many AWS HPC customers stand up (and tear down) clusters each and every day. AWS operates across a large number of geographic regions, each populated with multiple large-scale availability zones comprised of one more data centers. This enables AWS customers to easily access effectively limitless amounts of compute capacity in the location of their choice.
Q: What are my options for running HPC applications on AWS?
There are many open-source and commercial HPC solutions that support AWS. Many HPC users choose to start with AWS ParallelCluster, customizing the software to meet their unique needs. Other HPC users may prefer to deploy compute instances manually or using their own scripts which invoke EC2 APIs using the AWS CLI. For some users, a fully-managed SaaS offering may be the preferred approach. All of these options are viable and used by AWS customers.
Q: What is AWS ParallelCluster ?
AWS ParallelCluster is a tool for creating and launching an AWS CloudFormation template. It is a python application that calls AWS services including Amazon EC2, Amazon EBS, and Amazon S3. It also uses Amazon SQS, Amazon DynamoDB, Amazon CloudWatch, and AWS CloudFormation to monitor and maintain the cluster. For more information, see AWS ParallelCluster documentation.
Q: What are the benefits of using AWS ParallelCluster to run my HPC application?
AWS ParallelCluster provides quick access to a cluster environment with a variety of software development tools, a shared disk, a selection of scheduling software, instance and run monitoring capability. AWS ParallelCluster queues software launched by multiple users. Furthermore, environments launched by AWS ParallelCluster are fully-configured to run MPI applications using AWS enhanced networking features.
Q: How much does the AWS ParallelCluster software cost?
CfnCluster is a free open-source tool built by AWS.
Q: How much will the AWS resources in my HPC cluster cost?
Once deployed, typical AWS infrastructure costs are incurred which include the cost of the master instance, compute nodes, EBS volumes and any Amazon S3 storage used. Fore more information, see Services Used and Costs.
Q. What if my application requires different infrastructure configurations? Can I easily experiment with AWS ParallelCluster?
Yes - AWS ParallelCluster is especially well suited for experimentation as your cluster can be easily re-configured based on your needs and as your needs evolve. Additionally, your pcluster configuration file can be placed under version control to document the setup used for each experiment. When seeking the best performance or the lowest cost, AWS ParallelCluster makes it easy to test different cluster configuration aspects such as the instance type, operating system, and storage configurations.