AWS Deep Learning AMI GPU PyTorch 2.5 (Ubuntu 22.04)

For help getting started, please see the AWS Deep Learning AMI Developer Guide.
 

AMI Name format:

• Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.5.${PATCH_VERSION} (Ubuntu 22.04) ${YYYY-MM-DD}

Supported EC2 Instances:

• Please refer to Important changes to DLAMI
• Deep Learning with OSS Nvidia Driver supports G4dn, G5, G6, Gr6, P4, P4de, P5, P5e, P5en.

The AMI includes the following:

• Supported AWS Service: EC2
• Operating System: Ubuntu 22.04
• Compute Architecture: x86
• Python: /opt/conda/envs/pytorch/bin/python
• NVIDIA Driver:
  • OSS Nvidia driver: 550.144.03
• NVIDIA CUDA12.4 stack:
  • CUDA, NCCL and cuDDN installation path: /usr/local/cuda-12.4/
  • Default CUDA: 12.4
    • PATH /usr/local/cuda points to /usr/local/cuda-12.4/
    • Updated below env vars:
      • LD_LIBRARY_PATH to have /usr/local/cuda/lib:/usr/local/cuda/lib64:/usr/local/cuda:/usr/local/cuda/targets/x86_64-linux/lib
      • PATH to have /usr/local/cuda/bin/:/usr/local/cuda/include/
  • Compiled system NCCL Version present at /usr/local/cuda/: 2.21.5
  • PyTorch Compiled NCCL Version from PyTorch conda environment: 2.21.5
• NCCL Tests Location:
  • all_reduce, all_gather and reduce_scatter: /usr/local/cuda-xx.x/efa/test-cuda-xx.x/
  • To run NCCL tests, LD_LIBRARY_PATH is already with updated with needed paths.
    • Common PATHs are already added to LD_LIBRARY_PATH:
      • /opt/amazon/efa/lib:/opt/amazon/openmpi/lib:/opt/aws-ofi-nccl/lib:/usr/local/lib:/usr/lib
    • LD_LIBRARY_PATH is updated with CUDA version paths
      • /usr/local/cuda/lib:/usr/local/cuda/lib64:/usr/local/cuda:/usr/local/cud/targets/x86_64-linux/lib
• EFA Installer: 1.34.0
• Nvidia GDRCopy: 2.4.1
• Nvidia Transformer Engine: v1.11.0
• AWS OFI NCCL: 1.11.0-aws
  • Installation path: /opt/aws-ofi-nccl/ . Path /opt/aws-ofi-nccl/lib is added to LD_LIBRARY_PATH.
  • Tests path for ring, message_transfer: /opt/aws-ofi-nccl/tests
  • Note: PyTorch package comes with dynamically linked AWS OFI NCCL plugin as a conda package aws-ofi-nccl-dlc package as well and PyTorch will use that package instead of system AWS OFI NCCL.
• AWS CLI v2 as aws2 and AWS CLI v1 as aws
• EBS volume type: gp3
• Python version: 3.11
• Query AMI-ID with SSM Parameter (example region is us-east-1):
  • OSS Nvidia Driver:
    • aws ssm get-parameter --name /aws/service/deeplearning/ami/x86_64/oss-nvidia-driver-gpu-pytorch-2.5-ubuntu-22.04/latest/ami-id --region us-east-1 --query "Parameter.Value" --output text
• Query AMI-ID with AWSCLI (example region is us-east-1):
  • OSS Nvidia Driver:
    • aws ec2 describe-images --region us-east-1 --owners amazon --filters 'Name=name,Values=Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.5.? (Ubuntu 22.04) ????????' 'Name=state,Values=available' --query 'reverse(sort_by(Images, &CreationDate))[:1].ImageId' --output text

Notice

P5/P5e Instances:

• DeviceIndex is unique to each NetworkCard, and must be a non-negative integer less than the limit of ENIs per NetworkCard. On P5, the number of ENIs per NetworkCard is 2, meaning that the only valid values for DeviceIndex is 0 or 1. Below is the example of EC2 P5 instance launch command using awscli showing NetworkCardIndex from number 0-31 and DeviceIndex as 0 for first interface and DeviceIndex as 1 for rest 31 interrfaces.

aws ec2 run-instances --region $REGION \
--instance-type $INSTANCETYPE \
--image-id $AMI --key-name $KEYNAME \
--iam-instance-profile "Name=dlami-builder" \
--tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=$TAG}]" \
--network-interfaces "NetworkCardIndex=0,DeviceIndex=0,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa" \
 "NetworkCardIndex=1,DeviceIndex=1,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa" \
 "NetworkCardIndex=2,DeviceIndex=1,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa" \
 "NetworkCardIndex=3,DeviceIndex=1,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa" \
 "NetworkCardIndex=4,DeviceIndex=1,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa" \
 ....
 ....
 ....
 "NetworkCardIndex=31,DeviceIndex=1,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa"

Kernel:

• Kernel version is pinned using command:
  • • echo linux-aws hold | sudo dpkg —set-selections
      echo linux-headers-aws hold | sudo dpkg —set-selections
      echo linux-image-aws hold | sudo dpkg —set-selections
• We recommend users to avoid updating their kernel version (unless due to a security patch) to ensure compatibility with installed drivers and package versions. If users still wish to update they can run the following commands to unpin their kernel versions:
  • • echo linux-aws install | sudo dpkg —set-selections
      echo linux-headers-aws install | sudo dpkg —set-selections
      echo linux-image-aws install | sudo dpkg —set-selections
    • apt-get upgrade -y
• For each new version of DLAMI, latest available compatible kernel is used.

Release Date: 2025-02-17

AMI Names:

• Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.5.1 (Ubuntu 22.04) 20250216

Updated

• Updated NVIDIA Container Toolkit from version 1.17.3 to version 1.17.4
  • Please see the release notes page here for more information: https://github.com/NVIDIA/nvidia-container-toolkit/releases/tag/v1.17.4
  • In Container Toolkit version 1.17.4, the mounting of CUDA compat libraries is now disabled. In order to ensure compatibility with multiple CUDA versions on container workflows, please ensure you update your LD_LIBRARY_PATH to include your CUDA compatibility libraries as shown in under the “If you use a CUDA compatibility layer” tutorial here - https://docs.aws.amazon.com/sagemaker/latest/dg/inference-gpu-drivers.html#collapsible-cuda-compat

Removed

• Removed user space libraries cuobj and nvdisasm provided by NVIDIA CUDA toolkit to address CVE’s present in the NVIDIA CUDA Toolkit Security Bulletin for February 18, 2025

Release Date: 2025-01-21

AMI Names:

• Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.5.1 (Ubuntu 22.04) 20250119

Updated

• Upgraded Nvidia driver from version 550.127.05 to 550.144.03 to address CVE’s present in the NVIDIA GPU Display Driver Security Bulletin for January 2025.

Release Date: 2024-11-21

AMI Names:

• Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.5.1 (Ubuntu 22.04) 20241121

Added

• Initial release of Deep Learning AMI GPU PyTorch 2.4.1 (Ubuntu 22.04) series. Including a conda environment pytorch complimented with NVIDIA Driver R550, CUDA=12.4.1, cuDNN=8.9.7, PyTorch NCCL=2.21.5, and EFA=1.37.0.

Fixed

• Due to a change in the Ubuntu kernel to address a defects in the Kernel Address Space Layout Randomization (KASLR) functionality, G4Dn/G5 instances are unable to properly initialize CUDA on the OSS Nvidia driver. In order to mitigate this issue, this DLAMI includes functionality that dynamically loads the proprietary driver for G4Dn and G5 instances. Please allow a brief initialization period for this loading in order to ensure that your instances are able to work properly.
  • To check the status and health of this service, you can use the following commands:

$ sudo systemctl is-active dynamic_driver_load.service
active