Scale-Out Computing on AWS

View implementation guide

What does this AWS Solution do?

The Scale-Out Computing on AWS solution helps customers deploy and operate a multiuser environment for computationally intensive workflows, such as computer-aided engineering (CAE). The solution features a large selection of compute resources, a fast network backbone, unlimited storage, and budget and cost management directly integrated within AWS. 

Click here for related content 

Benefits

User interface

Scale-Out Computing on AWS deploys and sets up an example UI with a common set of APIs that the administrator and users can use to interact with their Amazon EC2 cluster.

Pre- and post-processing in the cloud

The solution leverages Desktop Cloud Visualization (DCV) graphical sessions to enable users to easily access the cluster to perform any pre- and post-processing visualization actions.

Real-time analytics

Schedulers and application logs are ingested in real-time and stored in the data lake for further processing.

Custom code and automation

The solution is deployed with a collection of scripts that are customizable and can be extended to help users collect data and run common cluster tasks.

AWS Solution overview

The diagram below presents the architecture you can automatically deploy using the solution's implementation guide and accompanying AWS CloudFormation template.

Scale-Out Computing on AWS architecture

The CloudFormation template deploys the following architecture consisting of eight components:

1. Amazon EC2 Auto Scaling to automatically provision the resources necessary to run cluster user tasks such as scale-out compute jobs.

2. This solution also deploys Amazon Elastic File System (Amazon EFS) for persistent storage, Amazon Simple Storage Service (Amazon S3) for persistent logs, and optional parallel file system Amazon FSx for Lustre.

3. At its core, the Amazon Elastic Compute Cloud (Amazon EC2) instance implements a scheduler, which dynamically provisions AWS resources required for jobs submitted by users. The scheduler instance also hosts web interface which allows users and administrators to interact with the environment.

4. Launch a 2D or 3D Workstation that uses NICE Desktop Cloud Visualization (DCV), that can be used to submit batch jobs and run GUI tools.

5. Security services and resources that are used include AWS Secrets Manager, AWS Certificate Manager, Security Groups , and AWS Identity and Access Management (IAM). 

6. AWS Lambda functions to verify the required prerequisites and create a default signed certificate for an Application Load Balancer (ALB) to manage access to DCV workstation sessions.

7. An Amazon OpenSearch Service cluster to store job and host information.

8. Elastic Load Balancing is used to ensure accessibility across Availability Zones, and Cost Allocation Tags are used with AWS Cost Explorer.

Read more 
Show less

Scale-Out Computing on AWS

Version 2.7.2
Release date: 06/2022
Author: AWS

Estimated deployment time: 35 min

Estimated cost  Source code 
CloudFormation template 
View implementation guide
Launch in the AWS Console
Deploy with an AWS IQ expert

Additional resources

Download implementation guide  

Resources and FAQ »
Contact us »

Use the button below to subscribe to solution updates.
Subscribe to RSS feed

Note: To subscribe to RSS updates, you must have an RSS plug-in enabled for the browser you are using.  

Did this Solutions Implementation help you?
Yes
No
Provide feedback 

Related content

Video
AWS re:Invent 2020: Rivian pushes the pace of automotive innovation with AWS
Video
Solving with AWS Solutions: Scale-Out Computing
Case Study
Rivian Executes Vision of Agile Engineering on AWS
In 2020, Rivian found that its on-premises research and development information technology infrastructure could not keep up with its performance needs.
 
Resource bottlenecks affected product lifecycle management, computer-aided design, and computer-aided engineering, so Rivian began using Amazon Web Services (AWS) to architect an agile engineering environment.
Read the full blog post 
AWS Industries Blog
Scaling EDA Workloads using Scale-Out Computing on AWS
Semiconductor and electronics companies using electronic design automation (EDA) applications can significantly accelerate their product development lifecycle and time to market by taking advantage of the near infinite compute, storage, and other resources available on AWS.
 
This blog post provides architectural and system-level guidance to build out an environment capable of scaling EDA applications to 30,000 cores or more.
Read the full blog post 
AWS Industries Blog
Deploying multi-physics simulations for biopharma process development on AWS
Amgen built and integrated a HPC platform based on the open-source solution Scale-Out Compute on AWS (SOCA) as an extension of their current AWS cloud infrastructure.
 
With the SOCA platform, Amgen was able to build a convenient and easy-to-use platform to run computation fluid dynamics simulations, expediting medical device development and biopharmaceutical process development.
Read the full blog post 
Back to top 
Build icon
Deploy an AWS Solution yourself

Browse our library of AWS Solutions to get answers to common architectural problems.

Learn more 
Find an APN partner
Find an AWS Partner Solution

Find AWS Partners to help you get started.

Learn more 
Explore icon
Explore Guidance

Find prescriptive architectural diagrams, sample code, and technical content for common use cases.

Learn more