Announcing Amazon EC2 Trn3 UltraServers for faster, lower-cost generative AI training
AWS announces the general availability of Amazon Elastic Compute Cloud (Amazon EC2) Trn3 UltraServers powered by our fourth–generation AI chip Trainium3, our first 3nm AWS AI chip purpose-built to deliver the best token economics for next-generation agentic, reasoning, and video generation applications.
Each AWS Trainium3 chip provides 2.52 petaflops (PFLOPs) of FP8 compute, increases the memory capacity by 1.5x and bandwidth by 1.7x over Trainium2 to 144 GB of HBM3e memory, and 4.9 TB/s of memory bandwidth. Trainium3 is designed for both dense and expert-parallel workloads with advanced data types (MXFP8 and MXFP4) and improved memory-to-compute balance for real-time, multimodal, and reasoning tasks.
Trn3 UltraServers can scale up to 144 Trainium3 chips (362 FP8 PFLOPs total) and are available in EC2 UltraClusters 3.0 to scale to hundreds of thousands of chips. A fully configured Trn3 UltraServer delivers up to 20.7 TB of HBM3e and 706 TB/s of aggregate memory bandwidth. The next-generation Trn3 UltraServer, feature the NeuronSwitch-v1, an all-to-all fabric that doubles interchip interconnect bandwidth over Trn2 UltraServer.
Trn3 delivers up to 4.4x higher performance, 3.9x higher memory bandwidth and 4x better performance/watt compared to our Trn2 UltraServers, providing the best price-performance for training and serving frontier-scale models, including reinforcement learning, Mixture-of-Experts (MoE), reasoning, and long-context architectures. On Amazon Bedrock, Trainium3 is our fastest accelerator, delivering up to 3× faster performance than Trainium2 with over 5× higher output tokens per megawatt at similar latency per user.
New Trn3 UltraServers are built for AI researchers and powered by the AWS Neuron SDK, to unlock breakthrough performance. With native PyTorch integration, developers can train and deploy without changing a single line of model code. For AI performance engineers, we’ve enabled deeper access to Trainium3 so they can fine-tune performance, customize kernels, and push models even further. Because innovation thrives on openness, we are committed to engaging with our developers through open-source tools and resources.