AWS HPC Blog

Jumpstart your HPC journey with the AWS Parallel Computing Service getting started kit

Last week, we announced the launch of AWS Parallel Computing Service (AWS PCS), a game-changing addition to our high performance computing offerings. AWS PCS is a managed service that makes it easier than ever for you to run and scale HPC workloads without having to focus on managing the underlying infrastructure. Whether you’re simulating weather patterns, designing next-generation vehicles, or chasing a cure for cancer, AWS PCS is designed to help accelerate your research and innovation.

As part of the AWS PCS roll-out, we’ve put together a host of resources to help you get going with our new service. Whether you prefer to read the instructions first, watch videos, or take a hand-on approach, we’ve got something for you.

Today we’ll walk you through these resources, so you can get moving quickly with PCS.

One click can mean more than many others

Our first resource is the one that might get you to your “a ha!” moment soonest. The one-click PCS recipe in our HPC Recipes Library (see later in this post for more on that) is the shortest path we could muster to give you a working HPC cluster, complete with an Amazon FSx for Lustre scratch filesystem, a small node group of x86 instances, all based around a sample AMI we already built for you.

We don’t expect this to be a production-grade cluster, but it’ll get you to a Slurm command line soonest – allowing you to figure out what will be perfect for you. And since – because of this recipe – you’re only ever 15 minutes away from a working cluster, you can afford to experiment and learn the old-fashioned way: by breaking things.

AWS PCS User Guide

Getting started documentationLike all AWS services, AWS PCS has a detailed User Guide developed by the technical writers, software engineers, and product managers involved in building it. It describes how to set up your AWS account to use PCS, and covers a wide array of topics, such as picking the right cluster size for your needs, managing your cluster day-to-day, and how PCS works with and integrates with other AWS services.

It also includes a step-by-step tutorial, Getting started with AWS PCS, that covers the basic components of a PCS cluster, including the cluster primitive, compute node groups, queues, and external filesystems. And it walks you through the steps to create a simple but functional HPC environment.

HPC Tech Shorts

HPC Tech ShortsSometimes, it’s helpful to see a demonstration and hear some wider context, rather than just reading about how to do something. That’s why we’ve released over 2 hours of builder-oriented video about PCS on the HPC Tech Shorts channel on YouTube. At launch, there are six videos, and we’ll be releasing more in the coming weeks and months.

  1. Introducing AWS Parallel Computing Service (9m) – This video introduces the value proposition for AWS PCS and describes, at a high level how it works and integrates with other AWS services.
  2. Your first AWS PCS cluster (43m) – Here, we follow along step-by-step with the Getting Started with AWS PCS tutorial from the User Guide. From setting up networking, to provisioning storage, to building your cluster, we explain everything we’re doing at each step of the way (and why we’re doing it).

These first two videos standalone. Once you’ve watched them, you’ll have a good idea of what PCS can do and you’ll have seen it in action.

When you’re ready to dive deeper, there are four more videos that cover a range of topics important for customizing PCS to meet your needs. We recommend you watch them in a sequence:

  1. Launch templates, instance profiles, security groups, and AMIs (17m) – In this video, we introduce how various AWS primitives combine in PCS to enable specific capabilities and behaviors for the instances where your HPC jobs run.
  2. Configuring Elastic Fabric Adapter (EFA) and multi-NIC instances (20m) – More than a hundred Amazon Elastic Compute Cloud (Amazon EC2) instance types can use EFA for fast, low-latency networking, some them even featuring multiple network cards for even better throughput. We cover the details of how to configure PCS to take maximum advantage of EFA, including an AWS CloudFormation template that can help you get started.
  3. Create and use a custom AMI for AWS PCS (29m) – With PCS, you can configure a super-customized software environment using a custom AMI. In this video, we take you through every step of the process, starting with a Rocky Linux 9 baseline image and ending with an AMI that uses Spack to manage your user-land software. Then, we show you how to use the AMI with PCS once it’s built.
  4. Bring your own login node (21m) – There’s two ways of providing access nodes for AWS PCS. You can let PCS manage the login instances, or you can configure your own instances to connect to your cluster’s Slurm controller. This video covers connecting a standalone EC2 instance to a PCS cluster and its shared Lustre filesystem.

There will be more tutorials like this over coming months.

HPC Recipes for AWS

The HPC Recipe LIbrary TS@2xAround this time last year, we released a community recipe library for HPC infrastructure on AWS, a library of patterns and infrastructure as code resources that you can use to learn HPC on AWS or even build up proofs-of-concepts and demonstrations. It’s grown quite a bit since then, and has been an integral part of several AWS HPC blog posts. We’ve now added a PCS-specific channel to it.

The PCS channel is growing quickly, but to give you a preview, there are recipes that support the getting started with PCS tutorial, help you set up a standalone login node, and show you how to build a simple CFD-ready cluster. The recipes in this repo are used extensively in our tutorials and videos.

We recommend you keep your eyes on HPC Recipes (consider starring it on GitHub), because it’s one of the main places where we will show you how to progressively mechanize many different aspects of running a PCS cluster.

Next Steps

AWS Parallel Computing Service (AWS PCS) is designed to accelerate your research and innovation by simplifying the management of HPC clusters on AWS. We’ve provided a comprehensive set of resources to help you get started:

These resources cater to different learning styles, whether you prefer reading docs, watching demos, or diving into hands-on examples.

Now it’s your turn to explore AWS PCS and see how it can transform your HPC workflows. Start with the getting started guide, watch our introductory videos, or jump right in with one of our HPC Recipes. As you build and scale your HPC environments on AWS PCS, you’ll discover new ways to accelerate your scientific and engineering breakthroughs.

Ready to take the next step in your HPC journey? Visit the AWS Parallel Computing Service product page to learn more and get started today.

Try it out and let us know what you think at ask-hpc@amazon.com. We’re excited to see what you’ll achieve with AWS PCS!

TAGS: , , , , ,
Matt Vaughn

Matt Vaughn

Matt Vaughn is a Principal Developer Advocate for HPC and scientific computing. He has a background in life sciences and building user-friendly HPC and cloud systems for long-tail users. When not in front of his laptop, he’s drawing, reading, traveling the world, or playing with the nearest dog.

Brendan Bouffler

Brendan Bouffler

Brendan Bouffler is the head of the Developer Relations in HPC Engineering at AWS. He’s been responsible for designing and building hundreds of HPC systems in all kind of environments, and joined AWS when it became clear to him that cloud would become the exceptional tool the global research & engineering community needed to bring on the discoveries that would change the world for us all. He holds a degree in Physics and an interest in testing several of its laws as they apply to bicycles. This has frequently resulted in hospitalization.