Amazon EC2 UltraServers

UltraServers enable efficient training and inference of models with hundreds of billions to trillions of parameters by linking a larger set of accelerators with a high-bandwidth, low-latency interconnect to deliver more compute and memory than standalone EC2 instances.

UltraServers enable real-time inference for ultra-large models that demand substantial memory and memory bandwidth resources beyond what a single EC2 instance can offer.

UltraServers enable faster collective communication for model parallelism as compared to standalone instances, helping you reduce your time to train.

You can launch instances into an UltraServer and leverage a dedicated, high-bandwidth, and low-latency accelerator interconnect across these instances. UltraServers enable access to a larger number of accelerators connected with this dedicated interconnect, delivering significantly more compute and memory in a single node than standalone EC2 instances.

EC2 UltraServers deployed in EC2 UltraClusters are interconnected with petabit-scale EFA networking to improve performance for distributed training workloads.

You can use EC2 UltraServers together with high-performance storage solutions such as Amazon FSx for Lustre, fully managed shared storage built on the most popular high-performance parallel file system. You can also use virtually unlimited cost-effective storage with Amazon Simple Storage Service (Amazon S3).

EC2 UltraServers are built on the AWS Nitro System, a rich collection of building blocks that offloads many of the traditional virtualization functions to dedicated hardware and software. Nitro delivers high performance, high availability, and high security, reducing virtualization overhead.