Posted On: Sep 10, 2021
Amazon ParallelCluster is a fully supported and maintained open source cluster management tool that makes it easier for scientists, researchers, and IT administrators to deploy and manage high performance computing (HPC) clusters on Amazon Web Services. HPC clusters are traditionally collections of tightly coupled compute, storage, and networking resources that enable customers to run large scale scientific and engineering workloads.
With its latest release, we have introduced a major update to AWS ParallelCluster. Significant feature enhancements in this version include:
- Support for cluster management via Amazon API Gateway: Customers can now manage and deploy clusters through HTTP endpoints with Amazon API Gateway. This opens up new possibilities for scripted or event-driven workflows, such as creating new clusters when datasets are ingested or when existing infrastructure is insufficient to meet dynamic compute needs. These APIs also make it easier for builders to use AWS ParallelCluster as a building block in their HPC workflows and create their own extensible solutions, services, and customized front-end interfaces. AWS ParallelCluster’s command-line interface (CLI) has also been redesigned for compatibility with this API and includes a new JSON output option. This new functionality makes it possible for customers to implement similar building block capabilities using the CLI as well.
- Improved custom AMI creation: Customers now have access to a more robust process for creating and managing custom AMIs using EC2 Image Builder. Customers can specify build components to install software, apply security hardening steps, or modify operating system configuration in a replicable way. Custom AMIs can now be managed through a separate AWS ParallelCluster configuration file, and can be created using the pcluster build-image command in the AWS ParallelCluster command-line interface
- Robust cluster configuration: Cluster configuration files have been retooled to make them more robust and easier to maintain. You can learn more about the changes to our configuration files here.
For more detail you can find the complete release notes for the latest version of Amazon ParallelCluster here.