Amazon EC2 G5 Instances

G5 instances deliver up to 3x higher graphics performance and up to 40% better price performance than G4dn instances. They have more ray tracing cores than any other GPU-based EC2 instance, feature 24 GB of memory per GPU, and support NVIDIA RTX technology. This makes them ideal for rendering realistic scenes faster, running powerful virtual workstations, and supporting graphics heavy applications at higher fidelity.

G5 instances deliver up to 3x higher performance and up to 40% better price performance for machine learning inference compared to G4dn instances. They are a highly performant and cost-efficient solution for customers who want to use NVIDIA libraries such as TensorRT, CUDA, and cuDNN to run their ML applications.

G5 instances offer up to 15% lower cost-to-train than Amazon EC2 P3 instances. They also deliver up to 3.3x higher performance for ML training compared to G4dn instances. This makes them a cost-efficient solution for training moderately complex and single node machine learning models for natural language processing, computer vision, and recommender engine use cases.

G5 instances are built on the AWS Nitro System, a combination of dedicated hardware and lightweight hypervisor which delivers practically all of the compute and memory resources of the host hardware to your instances for better overall performance and security. With G5 instances, the Nitro system provisions the GPUs in a pass-through mode, providing performance comparable to bare-metal.

G5 instances are the first in the cloud to feature NVIDIA A10G Tensor Core GPUs that deliver high performance for graphics-intensive and machine learning applications. Each instance features up to 8 A10G Tensor Core GPUs that come with 80 ray tracing cores and 24 GB of memory per GPU. They also offer 320 third-generation NVIDIA Tensor Cores delivering up to 250 TOPS resulting in high performance for ML workloads.

G5 instances offer NVIDIA RTX Enterprise and gaming drivers to customers at no additional cost. NVIDIA RTX Enterprise drivers can be used to provide high quality virtual workstations for a wide range of graphics-intensive workloads. NVIDIA gaming drivers provide unparalleled graphics and compute support for game development. G5 instances also support CUDA, cuDNN, NVENC, TensorRT, cuBLAS, OpenCL, DirectX 11/12, Vulkan 1.1, and OpenGL 4.5 libraries.

G5 instances come with up to 100 Gbps of networking throughput enabling them to support the low latency needs of machine learning inference and graphics-intensive applications. 24 GB of memory per GPU along with support for up to 7.6 TB of local NVMe SSD storage enable local storage of large models and datasets for high performance machine learning training and inference. G5 instances can also store large video files locally resulting in increased graphics performance and the ability to render larger and more complex video files.

G5 instances are built on the AWS Nitro System, which is a rich collection of building blocks that offloads many of the traditional virtualization functions to dedicated hardware and software to deliver high performance, high availability, and high security while also reducing virtualization overhead.

DLAMI provides ML practitioners and researchers with the infrastructure and tools to accelerate DL in the cloud, at any scale. Deep Learning Containers are Docker images preinstalled with DL frameworks to streamline the deployment of custom ML environments by letting you skip the complicated process of building and optimizing your environments from scratch.

If you prefer to manage your own containerized workloads through container orchestration services, you can deploy G5 instances with Amazon EKS or Amazon ECS.

You can use various Amazon Machine Images (AMIs) offered by AWS and NVIDIA offer that come with the NVIDIA drivers installed.

You can deploy ML models on G5 instances using Amazon SageMaker  a fully managed service for building, training, and deploying ML models.