Amplify lets you quickly and securely set up and manage a serverless UI for the HPC cluster, and Step Functions helps you visualize and control the workflow that orchestrates job steps. CloudWatch monitors the cluster’s performance through collected metrics, helping you gain insights into the operation. And by using CloudFormation, you can use infrastructure as code to provision the environment, limiting human errors and increasing the consistency of event responses. All of these services are fully managed by AWS.

Cognito provides frictionless customer identity and access management for the frontend and enables user pools as well as federated login and access. Federated access lets you use existing identities and permissions, and provide a uniform user experience with the same level of security as used by the rest of your company. By scoping AWS Identity and Access Management (IAM) policies according to the least-privilege principle, you can limit unauthorized access to resources.

EC2 Auto Scaling equally distributes Amazon EC2 instances in multiple availability zones (AZs) to increase fault tolerance and availability. It can detect when an instance is unhealthy, terminate it, and launch an instance to replace it. Additionally, if one AZ becomes unavailable, EC2 Auto Scaling can launch instances in another AZ to compensate. Amazon FSx, which supports the HPC application’s high input/output operations per second (IOPS) and large throughput, can also be deployed to multiple AZs, providing enhanced durability by synchronously replicating data across AZs. It also enhances availability during both planned system maintenance and unplanned service disruption by failing over automatically to the standby AZ. This protects data against instance failure and AZ disruption. Finally, Amazon S3 provides persistent and reliable storage for input and output data.

ParallelCluster spins up and down necessary instances to meet demand dynamically by using an EC2 Auto Scaling group, which makes sure that resources are the right size for the workload. Amazon FSx can process massive data sets with hundreds of gigabytes per second of throughput, millions of IOPS, and submillisecond latencies.

Step Functions and Lambda help minimize costs through their event-driven pattern: no costs are incurred when no jobs are submitted. Additionally, ParallelCluster uses an EC2 Auto Scaling group to spin out only the instances needed, avoiding resource idling and waste. ParallelCluster uses an EC2 Auto Scaling launch template to start instances for submitted jobs, and you can choose the most cost-effective instance type based on your performance benchmarking and resource utilization rate. CloudWatch monitors usage and delivers logs and insights that can help you right-size your fleet instances and operate cost-aware workloads.

Amazon S3 Intelligent-Tiering monitors access patterns and moves objects among tiers automatically, thus striking a balance between cost and energy reduction and access efficiency. EC2 Auto Scaling helps you dynamically scale the compute fleet of the HPC cluster to avoid resource idling, resulting in a more efficient and sustainable solution. Additionally, Step Functions and Lambda only operate in response to job submissions and don’t run during the HPC cluster’s idle time, thereby reducing the required resources and decreasing the environmental impact of your workloads.