Amazon EC2 P6e UltraServers and P6 instances

P6e UltraServers

With P6e-GB200 UltraServers, customers can access up to 72 Blackwell GPUs within one NVLink domain to use 360 petaflops of FP8 compute (without sparsity) and 13.4 TB of total high- bandwidth memory (HBM3e). P6e-GB200 UltraServers provide up to 130 terabytes per second of low-latency NVLink connectivity between GPUs and up to 28.8 terabits per second of total Elastic Fabric Adapter networking (EFAv4) for AI training and inference. This UltraServer architecture on P6e-GB200 enables customers to leverage a step change improvement in compute and memory, with up to 20x GPU TFLOPS, 11x GPU memory, and 15x aggregate GPU memory bandwidth under NVLink compared to P5en.

P6 instances

P6-B300 instances provide 8x NVIDIA Blackwell Ultra GPUs with 2.1 TB high bandwidth GPU memory, 6.4 Tbps EFA networking, 300 Gbps dedicated ENA throughput, and 4 TB of system memory. P6-B300 instances deliver 2x networking bandwidth, 1.5x GPU memory size, and 1.5x GPU TFLOPS (at FP4, without sparsity) compared to P6-B200 instances. These improvements make P6-B300 instances well suited for large-scale ML training and inference.

P6-B200 instances provide 8x NVIDIA Blackwell GPUs with 1440 GB of high- bandwidth GPU memory, 5th Generation Intel Xeon Scalable processors (Emerald Rapids), 2 TiB of system memory, up to 14.4 TBp/s of total bidirectional NVLink bandwidth, and 30 TB of local NVMe storage. These instances feature up to 2.25x GPU TFLOPs, 1.27x GPU memory size, and 1.6x GPU memory bandwidth compared to P5en instances.

P6e UltraServers and P6 instances are powered by the AWS Nitro System with specialized hardware and firmware designed to enforce restrictions so that no one, including anyone at AWS, can access your sensitive AI workloads and data. The Nitro System, which handles networking, storage, and other I/O functions, can deploy firmware updates, bug fixes, and optimizations while it remains operational. This increases stability and reduces downtime, which is critical to meeting training timelines and running AI applications in production.

To enable efficient distributed training, P6e UltraServers and P6 instances use fourth- generation Elastic Fabric Adapter networking (EFAv4). EFAv4 uses Scalable Reliable Datagram (SRD) protocol to intelligently route traffic across multiple network paths to maintain smooth operation even during congestion or failures.

P6e UltraServers and P6 instances are deployed in Amazon EC2 UltraClusters, which enable scaling up to tens of thousands of GPUs within a petabit-scale nonblocking network.

Each NVIDIA Blackwell GPU found in P6-B200 instances features a second-generation Transformer Engine and supports new precision formats such as FP4. It supports fifth- generation NVLink, a faster, wider interconnect delivering up to 1.8 TBp/s of bandwidth per GPU.

The Grace Blackwell Superchip, a key component of P6e-GB200, connects two high- performance NVIDIA Blackwell GPUs and an NVIDIA Grace CPU using the NVIDIA NVLink-C2C interconnect. Each Superchip delivers 10 petaflops of FP8 compute (without sparsity) and up to 372 GB of HBM3e. With the superchip architecture, 2 GPUs and 1 CPU are co-located within one compute module, increasing bandwidth between GPU and CPU by an order of magnitude compared to current generation P5en instances.

The NVIDIA Blackwell Ultra GPUs powering P6-B300 instances deliver a 2x increase in network bandwidth, 1.5x increase in GPU memory, and up to 1.5x FP4 compute improvements (without sparsity) in effective TFLOPs compared to P6-B200 instances.

P6e UltraServers and P6 instances provide 400 GB ps per GPU of EFAv4 networking for a total of 28.8 Tbps per P6e-GB200 UltraServer and 3.2 Tbps per P6-B200 instance.

P6-B300 instances offer 6.4 Tbps networking bandwidth, 2x compared to P6-B200 instances due to PCle Gen6, and are designed for large-scale distributed deep learning model training.

P6e UltraServers and P6 instances support Amazon FSx for Lustre file systems so you can access data at hundreds of GBp/s of throughput and millions of IOPS required for large-scale AI training and inference. P6e UltraServers support up to 405 TB of local NVMe SSD storage while P6 instances support up to 30 TB of local NVMe SSD storage for fast access to large datasets. You can also use virtually unlimited cost-effective storage with Amazon Simple Storage Service (Amazon S3).

Amazon SageMaker is a fully managed service for building, training, and deploying ML models. With Amazon SageMaker HyperPod, you can more easily scale to tens, hundreds, or thousands of GPUs to train a model quickly at any scale without worrying about setting up and managing resilient training clusters. (P6e-GB200 support coming soon)

AWS Deep Learning AMIs (DLAMI) provides ML practitioners and researchers with the infrastructure and tools to accelerate DL in the cloud, at any scale. AWS Deep Learning Containers are Docker images preinstalled with DL frameworks to streamline the deployment of custom ML environments by letting you skip the complicated process of building and optimizing your environments from scratch.

If you prefer to manage your own containerized workloads through container orchestration services, you can deploy P6e-GB200 UltraServers and P6-B200 instances with Amazon Elastic Kubernetes Service (Amazon EKS) or Amazon Elastic Container Service (Amazon ECS).

P6e UltraServers will also be available through NVIDIA NVIDA DGX Cloud, a fully managed environment with NVIDIA’s complete AI software stack. With NVIDIA DGX Cloud you get NVIDIA’s latest optimizations, benchmarking recipes, and technical expertise.

Learn more