Amazon EC2 G6e Instances

G6e instances offer up to 1.2x more GPU memory than P4d instances enabling customers to deploy LLMs and diffusion models while saving up to 20% in cost. Powered by L40S GPUs that feature fourth-generation tensor cores, they are a highly performant and cost-efficient solution for customers who want to use NVIDIA libraries such as TensorRT, CUDA, and cuDNN to run their ML applications.

G6e instances offer the same networking bandwidth (400 Gbps) and up to 1.2x more GPU memory than P4d instances. This makes them well suited for cost-efficient single-node fine-tuning or training of smaller models.

With 384 GB of total GPU memory paired with NVIDIA’s third generation ray tracing cores G6e instances deliver the highest performance for spatial computing workloads. The 400 Gbps of networking bandwidth improves real-time performance for spatial computing applications that are fed with real world inputs. These instances unlock new opportunities for customers to create more expansive 3D digital twins for workloads like factory planning, robot simulation, and network optimization.

G6e instances are built on the AWS Nitro System, a combination of dedicated hardware and lightweight hypervisor which delivers practically all of the compute and memory resources of the host hardware to your instances for better overall performance and security. With G6e instances, the Nitro system provisions the GPUs in a pass-through mode, providing performance comparable to bare-metal.

G6e instances feature NVIDIA L40S Tensor Core GPUs that combine powerful AI compute with best-in-class graphics and media acceleration. Each instance features up to 8 L40S Tensor Core GPUs that come with 48 GB of memory per GPU, fourth-generation NVIDIA Tensor Cores, third-generation NVIDIA RT cores, and DLSS 3.0 technology.

G6e instances offer NVIDIA RTX Enterprise and gaming drivers to customers at no additional cost. NVIDIA RTX Enterprise drivers can be used to provide high quality virtual workstations for a wide range of graphics-intensive workloads. NVIDIA gaming drivers provide unparalleled graphics and compute support for game development. G6e instances also support CUDA, cuDNN, NVENC, TensorRT, cuBLAS, OpenCL, DirectX 11/12, Vulkan 1.3, and OpenGL 4.6 libraries.

G6e instances come with up to 400 Gbps of networking throughput enabling them to support the low latency needs of machine learning inference and graphics-intensive applications. 48 GB of memory per GPU, up to 1.536 TB of system memory, and up to 7.6 TB of local NVMe SSD storage enable local storage of large models and datasets for high performance machine learning training and inference. G6e instances can also store large video files locally resulting in increased graphics performance and the ability to render larger and more complex video files.

G6e instances are built on the AWS Nitro System, which is a rich collection of building blocks that offloads many of the traditional virtualization functions to dedicated hardware and software to deliver high performance, high availability, and high security while also reducing virtualization overhead.