AWS EC2 Trn3 Instances

Trn3 UltraServers, powered by AWS Trainium3 chips, deliver up to 4.4x higher performance, 3.9x higher memory bandwidth, and 4x better performance per watt compared to our Trn2 UltraServers. On Amazon Bedrock, Trainium3 is the fastest accelerator, delivering up to 3x faster performance than Trainium2. This remarkable performance uplift also translates into significantly higher throughput for models like GPT-OSS serving at scale compared to Trainium2-based instances, while maintaining low latency per user.

Each Trn3 UltraServer scales up to 144 Trainium3 chips, and the new racks deliver over 2x the chip density compared to Trn2, increasing compute per rack and improving data center efficiency. Trn3 UltraServers are built on the AWS Nitro System and Elastic Fabric Adapter (EFA), and are deployed in non-blocking, multi-petabit scale EC2 UltraClusters 3.0, allowing you to scale to hundreds of thousands of Trainium chips for distributed training and serving.

Continuing Trainium’s legacy of performance leadership, Trn3 instances offer better price-performance than legacy AI accelerators, allowing you to drive down cost per token and cost per experiment. Higher throughput on workloads such as GPT-OSS and frontier-scale LLMs lower inference costs, and reduced training times for your most demanding models.

AWS Trainium3 chips, our first 3nm AI chips, are optimized to deliver the best token economics for next-generation agentic, reasoning, and video generation applications. Trn3 UltraServers deliver over 4× better energy efficiency than Trn2 UltraServers, and on Amazon Bedrock.  In real-world serving, Trn3 achieves over 5× higher output tokens per megawatt than Trn2 UltraServer while maintaining similar latency per user, helping you meet sustainability objectives without compromising performance.

Trn3 UltraServers are powered by AWS Neuron, the developer stack for AWS Trainium and AWS Inferentia, so you can run existing PyTorch and JAX code with without code changes.

TNeuron supports popular ML libraries such as vLLM, Hugging Face Optimum Neuron, PyTorch Lightning, TorchTitan, and integrates with services including Amazon SageMaker, Amazon SageMaker HyperPod, Amazon EKS, Amazon ECS, AWS Batch, and AWS ParallelCluster.

Each AWS Trainium3 chip delivers 2.52 FP8 PFLOPs of compute, and Trn3 UltraServers scale up to 144 Trainium3 chips, providing up to 362 FP8 PFLOPs of total FP8 compute in a single UltraServer. This high-density compute envelope is designed for training and serving frontier-scale transformers, Mixture-of-Experts models, and long-context architectures.

AWS Trainium3 delivers both memory capacity and bandwidth over the previous generation, with each chip offering 144 GB of HBM3e and 4.9 TB/s of memory bandwidth. Trn3 UltraServer delivers up to 20.7 TB of HBM3e and 706 TB/s of aggregate memory bandwidth, enabling larger batch sizes, extended context windows, and higher utilization for ultra-large multimodal, video, and reasoning models.

Trn3 UltraServers introduce NeuronSwitch-v1, an all-to-all fabric that doubles interchip interconnect bandwidth over Trn2 UltraServers, improving model-parallel efficiency and reducing communication overhead for MoE and tensor-parallel training. Trn3 UltraServers support up to 144 chips per UltraServer, over 2x more than Trn2 UltraServers.. For large-scale distributed training, we deploy Trn3 UltraServers in UltraCluster 3.0 with hundreds of thousands of Trainium3 chips in a single non-blocking, petabit scale network.

Trainium3 supports FP32, BF16, MXFP8, and MXFP4 precision modes, allowing you to balance accuracy and efficiency across dense and expert-parallel workloads. Built-in collective communication engines accelerate synchronization and reduce training overhead for large transformer, diffusion, and Mixture-of-Experts models, improving end-to-end training throughput at scale.

Trn3 UltraServers are programmed using the AWS Neuron SDK, which provides the compiler, runtime, training and inference libraries, and developer tools for AWS Trainium and AWS Inferentia. The Neuron Kernel Interface (NKI) offers low-level access to the Trainium instruction set, memory, and execution scheduling so performance engineers can build custom kernels and push performance beyond standard frameworks. Neuron Explorer delivers a unified profiling and debugging environment, tracing execution from PyTorch and JAX code down to hardware operations and providing actionable insights for sharding strategies, kernel optimizations, and large-scale distributed runs.