Accelerated computing Amazon EC2 instance types

*Single instance specifications are provided for information only.
P6e-GB200 instances are only available in UltraServers ranging in size
from 36 to 72 GPUs.

Amazon EC2 P6e-GB200 UltraServers accelerated by NVIDIA GB200 NVL72 offer the highest GPU AI training and inference performance in Amazon Elastic Compute Cloud (Amazon EC2).

Features:

• Grace Blackwell Superchips are powered by ARM-based Grace CPUs and up to 72 Blackwell GPUs within one NVLink domain to deliver up to 360 petaflops of FP8 compute (without sparsity)
  
• Up 13.4 TB of high-bandwidth memory (HBM3e) GPU memory  
  
• Up to 28.8 terabits per second of network bandwidth with support for Elastic Fabric Adapter (EFAv4) and NVIDIA GPUDirect Remote Direct Memory Access (RDMA)
  
• 1800 GB/s peer-to-peer GPU communication with NVIDIA NVSwitch

Use Cases

• P6e-GB200 UltraServers accelerate both the training and inference of frontier models, including mixture of expert models and reasoning models, at the trillion-parameter scale.
• Agentic and generative AI applications, including question answering, code generation, video and image generation, speech recognition, and more.

P6e-GB200 instances have the following specs:

• Amazon EBS-optimized
  
• Enhanced Networking
• Elastic Fabric Adapter (EFAv4)

Amazon EC2 P6-B200 instances, accelerated by NVIDIA Blackwell GPUs, offer up to 2x performance compared to P5en instances for AI training and inference.

Features:

• 5th Generation Intel Xeon Scalable processors (Emerald Rapids)
  

• 8 NVIDIA Blackwell GPUs
  

• Up to 1440 GB of HBM3e GPU memory

• Up to 3.2 terabits per second network bandwidth with support for Elastic Fabric Adapter (EFAv4) and NVIDIA GPUDirect “Remote Direct Memory Access” (RDMA)

• 1800 GB/s peer-to-peer GPU communication with NVIDIA NVSwitch

P6-B200 instances have the following specs:

• Amazon EBS-optimized
• Enhanced Networking
• Elastic Fabric Adapter (EFAv4)

Use Cases

• P6-B200 instances are a cost-effective option to train and deploy medium-to-large frontier foundation models such as mixture of experts and reasoning models with high performance.
• Agentic and generative AI applications, including question answering, code generation, video and image generation, speech recognition, and more
• HPC applications at scale in pharmaceutical discovery, seismic analysis, weather forecasting, and financial modeling

*GPUDirect RDMA is not supported in P5.4xlarge

Amazon EC2 P5 instances are GPU-based instances and highest performance in Amazon EC2 for deep learning and high performance computing (HPC).

Features:

• Intel Sapphire Rapids CPU and PCIe Gen5 between the CPU and GPU in P5en instances; 3rd Gen AMD EPYC processors (AMD EPYC 7R13) and PCIe Gen4 between the CPU and GPU in P5 and P5e instances.
  

• Up to 8 NVIDIA H100 (in P5) or H200 (in P5e and P5en) Tensor Core GPUs  
  

• Up to 3,200 Gbps network bandwidth with support for Elastic Fabric Adapter (EFA) and NVIDIA GPUDirect RDMA (remote direct memory access)
  

• 900 GB/s peer-to-peer GPU communication with NVIDIA NVSwitch
  

P5 instances have the following specs:

• EBS Optimized
• Enhanced Networking
• Elastic Fabric Adapter (EFA)

Use Cases

Generative AI applications, including question answering, code generation, video and image generation, speech recognition, and more.

HPC applications at scale in pharmaceutical discovery, seismic analysis, weather forecasting, and financial modeling.

Amazon EC2 P4 instances provide high performance for machine learning training and high performance computing in the cloud.

• 3.0 GHz 2nd Generation Intel Xeon Scalable processors (Cascade Lake P-8275CL)

• Up to 8 NVIDIA A100 Tensor Core GPUs

• 400 Gbps instance networking with support for Elastic Fabric Adapter (EFA) and NVIDIA GPUDirect RDMA (remote direct memory access)

• 600 GB/s peer-to-peer GPU communication with NVIDIA NVSwitch

• Deployed in Amazon EC2 UltraClusters consisting of more than 4,000 NVIDIA A100 Tensor Core GPUs, petabit-scale networking, and scalable low-latency storage with Amazon FSx for Lustre

P4d instances have the following specs:

• 3.0 GHz 2nd Generation Intel Xeon Scalable processors
• Intel AVX, Intel AVX2, Intel AVX-512, and Intel Turbo
• EBS Optimized
• Enhanced Networking†
• Elastic Fabric Adapter (EFA)

Use Cases

Machine learning, high performance computing, computational fluid dynamics, computational finance, seismic analysis, speech recognition, autonomous vehicles, and drug discovery.

Amazon EC2 G6e instances are designed to accelerate deep learning inference and spatial computing workloads.

Features:

• 3rd generation AMD EPYC processors (AMD EPYC 7R13)
• Up to 8 NVIDIA L40S Tensor Core GPUs
• Up to 400 Gbps of network bandwidth
• Up to 7.6 TB of local NVMe local storage

Use Cases

Inference workloads for large language models and diffusion models for image, audio, and video, generation; single-node training of moderately complex generative AI models; 3D simulations, digital twins, and industrial digitization.

Amazon EC2 G6 instances are designed to accelerate graphics-intensive applications and machine learning inference.

Features:

• 3rd generation AMD EPYC processors (AMD EPYC 7R13)
• Up to 8 NVIDIA L4 Tensor Core GPUs
• Up to 100 Gbps of network bandwidth
• Up to 7.52 TB of local NVMe local storage

Use Cases

Deploying ML models for natural language processing, language translation, video and image analysis, speech recognition, and personalization as well as graphics workloads, such as creating and rendering real-time, cinematic-quality graphics and game streaming.

Amazon EC2 G6 instances are designed to accelerate graphics-intensive applications and machine learning inference.

Features:

• 3rd generation AMD EPYC processors (AMD EPYC 7R13)
• Up to 8 NVIDIA L4 Tensor Core GPUs
• Up to 100 Gbps of network bandwidth
• Up to 7.52 TB of local NVMe local storage

Use Cases

Deploying ML models for natural language processing, language translation, video and image analysis, speech recognition, and personalization as well as graphics workloads, such as creating and rendering real-time, cinematic-quality graphics and game streaming.

Amazon EC2 G5g instances are powered by AWS Graviton2 processors and feature NVIDIA T4G Tensor Core GPUs to provide the best price performance in Amazon EC2 for graphics workloads such as Android game streaming. They are the first Arm-based instances in a major cloud to feature GPU acceleration. Customers can also use G5g instances for cost-effective ML inference.

Features:

• Custom built AWS Graviton2 Processor with 64-bit Arm Neoverse cores
  
• Up to 2 NVIDIA T4G Tensor Core GPUs
  
• Up to 25 Gbps of networking bandwidth
  
• EBS-optimized by default
  
• Powered by the AWS Nitro System, a combination of dedicated hardware and lightweight hypervisor
  

Use Cases

Android game streaming, machine learning inference, graphics rendering, autonomous vehicle simulations

Amazon EC2 G5 instances are designed to accelerate graphics-intensive applications and machine learning inference. They can also be used to train simple to moderately complex machine learning models.

Features:

• 2nd generation AMD EPYC processors (AMD EPYC 7R32)
• Up to 8 NVIDIA A10G Tensor Core GPUs
• Up to 100 Gbps of network bandwidth
• Up to 7.6 TB of local NVMe local storage
  

G5 instances have the following specs:

• 2nd Generation AMD EPYC processors
• EBS Optimized
• Enhanced Networking†

Use Cases

Graphics-intensive applications such as remote workstations, video rendering, and cloud gaming to produce high fidelity graphics in real time. Training and inference deep learning models for machine learning use cases such as natural language processing, computer vision, and recommender engine use cases.

Amazon EC2 G4dn instances are designed to help accelerate machine learning inference and graphics-intensive workloads.

Features:

• 2nd Generation Intel Xeon Scalable Processors (Cascade Lake P-8259CL)
  
• Up to 8 NVIDIA T4 Tensor Core GPUs
  
• Up to 100 Gbps of networking throughput
  
• Up to 1.8 TB of local NVMe storage
  

All instances have the following specs:

• 2.5 GHz Cascade Lake 24C processors
• Intel AVX, Intel AVX2, Intel AVX-512, and Intel Turbo
• EBS Optimized
• Enhanced Networking†

Use Cases

Machine learning inference for applications like adding metadata to an image, object detection, recommender systems, automated speech recognition, and language translation. G4 instances also provide a very cost-effective platform for building and running graphics-intensive applications, such as remote graphics workstations, video transcoding, photo-realistic design, and game streaming in the cloud.  

Amazon EC2 G4ad instances provide the best price performance for graphics intensive applications in the cloud.

Features:

• 2nd Generation AMD EPYC Processors (AMD EPYC 7R32)
• AMD Radeon Pro V520 GPUs
• Up to 2.4 TB of local NVMe storage

All instances have the following specs:

• Second generation AMD EPYC processors
• EBS Optimized
• Enhanced Networking†

Use Cases

Graphics-intensive applications, such as remote graphics workstations, video transcoding, photo-realistic design, and game streaming in the cloud.

Amazon EC2 Trn2 instances, powered by AWS Trainium2 chips, are purpose built for high-performance generative AI training and inference of models with hundreds of billions to trillion+ parameters.

Features:

• 16 AWS Trainium2 chips
  
• Supported by AWS Neuron SDK
  
• 4th Generation Intel Xeon Scalable processor (Sapphire Rapids 8488C)
  
• Up to 12.8 Tbps third-generation Elastic Fabric Adapter (EFA) networking bandwidth
  
• Up to 8 TB local NVMe storage
  
• High-bandwidth, intra-instance, and inter-instance connectivity with NeuronLink
  
• Deployed in Amazon EC2 UltraClusters and available in EC2 UltraServers (available in preview)
  
• Amazon EBS-optimized
• Enhanced networking

Use Cases

Training and inference of the most demanding foundation models including large language models (LLMs), multi-modal models, diffusion transformers and more to build a broad set of next-generation generative AI applications.

Amazon EC2 Trn1 instances, powered by AWS Trainium chips, are purpose built for high-performance deep learning training while offering up to 50% cost-to-train savings over comparable Amazon EC2 instances.

Features:

• 16 AWS Trainium chips
  
• Supported by AWS Neuron SDK
  
• 3rd Generation Intel Xeon Scalable processor (Ice Lake SP)
  
• Up to 1600 Gbps second-generation Elastic Fabric Adapter (EFA) networking bandwidth
  
• Up to 8 TB local NVMe storage
  
• High-bandwidth, intra-instance connectivity with NeuronLink
  
• Deployed in EC2 UltraClusters that enable scaling up to 30,000 AWS Trainium accelerators, connected with a petabit-scale nonblocking network, and scalable low-latency storage with Amazon FSx for Lustre
  
• Amazon EBS-optimized
• Enhanced networking

Use Cases

Deep learning training for natural language processing (NLP), computer vision, search, recommendation, ranking, and more

Amazon EC2 Inf2 instances are purpose built for deep learning inference. They deliver high performance at the lowest cost in Amazon EC2 for generative artificial intelligence models, including large language models and vision transformers. Inf2 instances are powered by AWS Inferentia2. These new instances offer 3x higher compute performance, 4x higher accelerator memory, up to 4x higher throughput, and up to 10x lower latency compared to Inf1 instances

Features:

• Up to 12 AWS Inferentia2 chips

• Supported by AWS Neuron SDK

• Dual AMD EPYC processors (AMD EPYC 7R13)

• Up to 384 GB of shared accelerator memory (32 GB HBM per accelerator)

• Up to 100 Gbps networking
  

Use Cases

Natural language understanding (advanced text analytics, document analysis, conversational agents), translation, image and video generation, speech recognition, personalization, fraud detection, and more.

Amazon EC2 Inf1 instances are built from the ground up to support machine learning inference applications.

Features:

• Up to 16 AWS Inferentia Chips
• Supported by AWS Neuron SDK
• High frequency 2nd Generation Intel Xeon Scalable processors (Cascade Lake P-8259L)
• Up to 100 Gbps networking
  

Use Cases

Recommendation engines, forecasting, image and video analysis, advanced text analytics, document analysis, voice, conversational agents, translation, transcription, and fraud detection.

Amazon EC2 DL1 instances are powered by Gaudi accelerators from Habana Labs (an Intel company). They deliver up to 40% better price performance for training deep learning models compared to current generation GPU-based EC2 instances.

Features:

• 2nd Generation Intel Xeon Scalable Processor (Cascade Lake P-8275CL)
  
• Up to 8 Gaudi accelerators with 32 GB of high bandwidth memory (HBM) per accelerator
• 400 Gbps of networking throughput
• 4 TB of local NVMe storage
  

DL1 instances have the following specs:

• 2nd Generation Intel Xeon Scalable Processor
• Intel AVX†, Intel AVX2†, Intel AVX-512, Intel Turbo
• EBS Optimized
• Enhanced Networking†

Use Cases

Deep learning training, object detection, image recognition, natural language processing, and recommendation engines.

Amazon EC2 DL2q instances , powered by Qualcomm AI 100 accelerators, can be used to cost-efficiently deploy deep learning (DL) workloads in the cloud or validate performance and accuracy of DL workloads that will be deployed on Qualcomm devices.

Features:

• 8 Qualcomm AI 100 accelerators

• Supported by Qualcomm Cloud AI Platform and Apps SDK

• 2nd Generation Intel Xeon Scalable Processors (Cascade Lake P-8259CL)
  

• Up to 128 GB of shared accelerator memory  

• Up to 100 Gbps networking
  

Use Cases

Run popular DL and generative AI applications, such as content generation, image analysis, text summarization, and virtual assistants.; Validate AI workloads before deploying them across smartphones, automobiles, robotics, and extended reality headsets.

Amazon EC2 F2 instances offer customizable hardware acceleration with field programmable gate arrays (FPGAs).

Features:

• Up to 8 AMD Virtex UltraScale+ HBM VU47P FPGAs with 2.9 million logic cells and 9024 DSP slices

• 3rd generation AMD EPYC processor

• 64 GiB of DDR4 ECC-protected FPGA memory

• Dedicated FPGA PCI-Express x16 interface

• Up to 100 Gbps of networking bandwidth

• Supported by FPGA Developer AMI and FPGA Development Kit
  

Use Cases

Genomics research, financial analytics, real-time video processing, big data search and analysis, and security.

Amazon EC2 VT1 instances are designed to deliver low cost real-time video transcoding with support for up to 4K UHD resolution.

Features:

• 2nd Generation Intel Xeon Scalable Processors (Cascade Lake P-8259CL)
  
• Up to 8 Xilinx U30 media accelerator cards with accelerated H.264/AVC and H.265/HEVC codecs
• Up to 25 Gbps of enhanced networking throughput
• Up to 19 Gbps of EBS bandwidth

All instances have the following specs:

• 2nd Generation Intel Xeon Scalable Processors
• Intel AVX†, Intel AVX2†, Intel AVX-512, Intel Turbo
• EBS Optimized
• Enhanced Networking†

Use Cases

Live event broadcast, video conferencing, and just-in-time transcoding.