AWS Deep Learning AMI (Amazon Linux 2)

 

For help getting started, please see the AWS Deep Learning AMI Developer Guide.

AMI Name format:

• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version ${XX.X}
• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version ${XX.X}

Supported EC2 Instances:

• Please refer to Important changes to DLAMI
• Deep Learning with OSS Nvidia Driver supports G4dn, G5, G6, Gr6, P4d, P4de, P5.
• Deep Learning with Proprietary Nvidia Driver supports G3 (G3.16x not supported), P3, P3dn

The AMI includes the following:

• Supported AWS Service: EC2
• Operating System: Amazon Linux 2
• Compute Architecture: x86
• Conda environments framework and python versions:
  • Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2):
    • python3: Python 3.10
    • tensorflow2_p310: TensorFlow 2.16, Python 3.10
    • pytorch_p310: PyTorch 2.2, Python 3.10
  • Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2):
    • python3: Python 3.10
    • tensorflow2_p310: TensorFlow 2.16, Python 3.10
    • pytorch_p310: PyTorch 2.2, Python 3.10
• NVIDIA Driver:
  • OSS Nvidia driver: 550.144.03
  • Proprietary Nvidia driver: 550.144.03
• NVIDIA CUDA12.1-12.4 stack:
  • CUDA, NCCL and cuDDN installation path: /usr/local/cuda-xx.x/
  • Default CUDA: 12.1
    • PATH /usr/local/cuda points to CUDA12.1
    • Updated below env vars:
      • LD_LIBRARY_PATH to have /usr/local/cuda-12.1/lib:/usr/local/cuda-12.1/lib64:/usr/local/cuda-12.1:/usr/local/cuda-12.1/targets/x86_64-linux/lib
      • PATH to have /usr/local/cuda-12.1/bin/:/usr/local/cuda-11.8/include/
    • For any different CUDA version, please update LD_LIBRARY_PATH accordingly.
  • Compiled NCCL Version for CUDA 12.1-12.4: 2.22.3
  • NCCL Tests Location:
    • all_reduce, all_gather and reduce_scatter: /usr/local/cuda-xx.x/efa/test-cuda-xx.x/
    • To run NCCL tests, LD_LIBRARY_PATH needs to passed having below updates.
      • Common PATHs are already added to LD_LIBRARY_PATH: 
        • /opt/amazon/efa/lib:/opt/amazon/openmpi/lib:/opt/aws-ofi-nccl/lib:/usr/local/lib:/usr/lib
      • For any different CUDA version, please update LD_LIBRARY_PATH accordingly.
• EFA Installer: 1.38.0
• GDRCopy: 2.4
• AWS OFI NCCL: 1.13.0
  • System location: /usr/local/cuda-xx.x/efa
  • This is added to run NCCL tests located at /usr/local/cuda-xx.x/efa/test-cuda-xx.x/
  • Also, PyTorch package comes with dynamically linked AWS OFI NCCL plugin as a conda package aws-ofi-nccl-dlc package as well and PyTorch will use that package instead of system AWS OFI NCCL.
• NCCL Tests Location: /usr/local/cuda-xx.x/efa/test-cuda-xx.x/
• AWS CLI v2 at /usr/local/bin/aws2 and AWS CLI v1 at /usr/local/bin/aws
• EBS volume type: gp3
• Query AMI-ID with SSM Parameter (example region is us-east-1):
  • OSS Nvidia Driver:
    • aws ssm get-parameter --name /aws/service/deeplearning/ami/x86_64/multi-framework-oss-nvidia-driver-amazon-linux-2/latest/ami-id --region us-east-1 --query "Parameter.Value" --output text
  • Proprietary Nvidia Driver:
    • aws ssm get-parameter --name /aws/service/deeplearning/ami/x86_64/multi-framework-proprietary-nvidia-driver-amazon-linux-2/latest/ami-id --region us-east-1 --query "Parameter.Value" --output text
• Query AMI-ID with AWSCLI (example region is us-east-1):
  • OSS Nvidia Driver:
    • aws ec2 describe-images --region us-east-1 --owners amazon --filters 'Name=name,Values=Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version ??.?' 'Name=state,Values=available' --query 'reverse(sort_by(Images, &CreationDate))[:1].ImageId' --output text
  • Proprietary Nvidia Driver:
    • aws ec2 describe-images --region us-east-1 --owners amazon --filters 'Name=name,Values=Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version ??.?' 'Name=state,Values=available' --query 'reverse(sort_by(Images, &CreationDate))[:1].ImageId' --output text

Notice

EFA Updates from 1.37 to 1.38 (Release on 2025-02-05)

• EFA now bundles the AWS OFI NCCL plugin, which can now be found in /opt/amazon/ofi-nccl rather than the original /opt/aws-ofi-nccl/. If updating your LD_LIBRARY_PATH variable, please ensure that you modify your OFI NCCL location properly.

Neuron Conda Environment Removal

• Deep Learning Proprietary Nvidia Driver AMIs released after July 18, 2024 will be shipped without neuron conda environments for PyTorch and TensorFlow. Please use the Neuron DLAMIs on the DLAMI Release Notes instead, to utilize neuron environments.

Audit Package Removal

• DLAMI’s released between March 26,2024 (2024-03-26) and April 12, 2024 (2024-04-12) were shipped without the audit package. If you require this specific package for your logging and monitoring needs, please migrate your workflows to the latest DLAMI in order to consume those with the audit package installed.

Horovod

• Horovod is removed from the current pytorch_p310 and tensorflow2_p310 conda environments on the DLAMI. Customers will be able install the horovod libraries by following the horovod guidelines and install them on their DLAMIs for their distributed training jobs.

Release Date: 2025-02-05

AMI Names:

• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 80.2
• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 80.4

Updated

• Upgraded EFA version from 1.37.0 to 1.38.0
  • EFA now bundles the AWS OFI NCCL plugin, which can now be found in /opt/amazon/ofi-nccl rather than the original /opt/aws-ofi-nccl/. If updating your LD_LIBRARY_PATH variable, please ensure that you modify your OFI NCCL location properly.
• Upgraded Nvidia Container Toolkit from 1.17.3 to 1.17.4

Release Date: 2025-01-17

AMI Names:

• Deep Learning Base OSS Nvidia Driver AMI (Amazon Linux 2) Version 68.3
• Deep Learning Base Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 66.0

Updated

• Upgraded Nvidia driver from version 550.127.05 to 550.144.03 to address CVE’s present in the NVIDIA GPU Display Driver Security Bulletin for January 2025

Release Date: 2024-12-09

AMI Names:

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 80.1
• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 79.9

Updated

• Upgraded Nvidia Container Toolkit from version 1.17.0 to 1.17.3

Release Date: 2024-11-11

AMI Names:

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 79.9
• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 79.7

Updated

• Upgraded Nvidia Container Toolkit from version 1.16.2 to 1.17.0, addressing the security vulnerability CVE-2024-0134.

Release Date: 2024-10-22

AMI Names:

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 79.6
• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 79.6

Updated

• Upgraded Nvidia driver from version 550.90.07 to 550.127.05 to address CVE’s present in the NVIDIA GPU Display Security Bulletin for October 2024

Release Date: 2024-10-03

AMI Names:

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 79.3
• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 79.3

Updated

• Upgraded Nvidia Container Toolkit from version 1.16.1 to 1.16.2, addressing the security vulnerability CVE-2024-0133.

Release Date: 2024-07-18

AMI Names:

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 78.6
• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 78.7

Updated

• Removed aws_neuron_pytorch_p38 and aws_neuron_tensorflow_p38 conda environments from the Deep Learning Proprietary Nvidia Driver AMI.
• Removed Inf1 instance family support from the Deep Learning Proprietary Nvidia Driver AMI.

Release Date: 2024-06-06

AMI Names:

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 78.5
• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 78.5

Updated

• Updated Nvidia driver version to 535.183.01 from 535.161.08

Release Date: 2024-05-17

AMI Names:

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 78.1
• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 78.1

Updated

• Updated torchserve from v0.8.2 to v0.11.0 in the pytorch_p310 environment.

Release Date: 2024-05-07

AMI Names:

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 78.0
• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 78.0

Updated

• TensorFlow version updated from 2.15 to 2.16 in the tensorflow2_p310 environment.
• Updated EFA version from version 1.30 to version 1.32
• Updated AWS OFI NCCL plugin from version 1.7.4 to version 1.9.1
• Updated Nvidia container toolkit from version 1.13.5 to version 1.15.0
  • NOTE: Version 1.15.0 does NOT include the nvidia-container-runtime and nvidia-docker2packages. It is recommended to use nvidia-container-toolkit packages directly by following Nvidia container toolkit docs.

Added

• Added CUDA12.3 stack with CUDA12.3, NCCL 2.21.5, CuDNN 8.9.7

Removed

• Removed CUDA11.7, CUDA12.0 stacks present at /usr/local/cuda-11.7 and /usr/local/cuda-12.0
• Removed nvidia-docker2 package and its command nvidia-docker as part of Nvidia container toolkit update from 1.13.5 to 1.15.0 which does NOT include the nvidia-container-runtime and nvidia-docker2 packages.

Release Date: 2024-04-04

AMI Names:

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 77.0
• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 77.0

Updated

• PyTorch version updated from 2.1 to 2.2 in the pytorch_p310 environment.
• For OSS Nvidia driver DLAMIs, added G6 and Gr6 EC2 instances support. Please refer EC2 instance selection page for more information.

Release Date: 2024-03-29

AMI Names:

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 76.8
• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 76.9

Updated

• Updated Nvidia driver from 535.104.12 to 535.161.08 in both Proprietary and OSS Nvidia driver DLAMIs.
• The new supported instances for each DLAMI are as follows:
  • Deep Learning with Proprietary Nvidia Driver supports G3 (G3.16x not supported), P3, P3dn, Inf1
  • Deep Learning with OSS Nvidia Driver supports G4dn, G5, P4d, P4de.

Removed

• Removed G4dn, G5, G3.16x EC2 instances support from Proprietary Nvidia driver DLAMI.

Version 76.8

Release Date: 2024-03-20

AMI Names:

• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 76.8

Added

• Added awscliv2 in the AMI as /usr/local/bin/aws2, alongside awscliv1 as /usr/local/bin/aws on Proprietary Nvidia Driver AMI

Version 76.7

Release Date: 2024-03-20

AMI Names:

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 76.7

Added

• Added awscliv2 in the AMI as /usr/local/bin/aws2, alongside awscliv1 as /usr/local/bin/aws on OSS Nvidia Driver AMI
• Updated OSS Nvidia driver DLAMI with G4dn and G5 support, based on it current support looks like below:
  • Deep Learning Base Proprietary Nvidia Driver AMI (Amazon Linux 2) supports P3, P3dn, G3, G5, G4dn.
  • Deep Learning Base OSS Nvidia Driver AMI (Amazon Linux 2) supports G4dn, G5, P4, P5.
• OSS Nvidia driver DLAMIs are recommended to be used for G4dn, G5, P4, P5.

Version 76.3

Release Date: 2024-02-14

Updated

• Updated TensorFlow from 2.13.0 to 2.15.0
• Updated EFA from 1.29.0 to 1.30.0
• Updated AWS-OFI-NCCL from 1.7.3-aws to 1.7.4-aws
• Updated Nvidia Driver to 535.104.12 on Deep Learning Proprietary Nvidia Driver AMI
• Updated Nvidia Driver to 535.154.05 on Deep Learning OSS Nvidia Driver AMI

Version 76.2

Release Date: 2024-02-02

AMI Names:

• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 76.2
• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 76.4

Security

• Updated runc package version to consume patch for CVE-2024-21626.

Version 76.1

Release Date: 2023-12-27

Updated

• Updated PyTorch from 2.0.1 to 2.1.0

Version 75.1

Release Date: 2023-11-17

AMI Names:

Please refer to Important changes to DLAMI

• Deep Learning OSS Nvidia Driver AMI (Amazon Linux 2) Version 75.1
• Deep Learning Proprietary Nvidia Driver AMI (Amazon Linux 2) Version 75.1

Added

• AWS Deep Learning AMI (DLAMI) is split into two separate groups:
  • DLAMI that uses Nvidia Proprietary Driver (to support P3, P3dn, G3, G5, G4dn).
  • DLAMI that uses Nvidia OSS Driver to enable EFA (to support P4, P5).
• Please refer to public annoucement for more information on DLAMI split.
• AWS cli queries for above are in the release notes under bullet point Query AMI-ID with AWSCLI (example region is us-east-1)

Updated

• EFA updated from 1.26.1 to 1.29.0
• GDRCopy updated from 2.3 to 2.4

Version 74.4

Release Date: 2023-10-27

Updated

• AWS OFI NCCL Plugin updated from version 1.7.2 to version 1.7.3
• Updated CUDA 12.0-12.1 directories with NCCL version 2.18.5
• CUDA12.1 updated as the default CUDA Version
  • Updated LD_LIBRARY_PATH to have /usr/local/cuda-12.1/targets/x86_64-linux/lib/:/usr/local/cuda-12.1/lib:/usr/local/cuda-12.1/lib64:/usr/local/cuda-12.1 and PATH to have /usr/local/cuda-12.1/bin/
  • For customers looking to change to any different CUDA version, please define the LD_LIBRARY_PATH and PATH variables accordingly.
• Updated Pillow from version 9.4.0 to 10.1.0 to fix SNYK-PYTHON-PILLOW-5918878 in all conda environments
• Updated opencv-python from 4.8.0.74 to 4.8.1.78 to fix SNYK-PYTHON-OPENCVPYTHON-5926695 in all conda environments

Added

• Kernel Live Patching is now enabled. Live patching enables customers to apply security vulnerability and critical bug patches to a running Linux kernel, without reboots or disruptions to running applications.
  • Please note that live patching support for kernel 5.10.192 will end on 11/30/23.
  • For more information please reference the official AWS documents here - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/al2-live-patching.html

Version 74.0

Release Date: 2023-07-19

Updated

• Updated TensorFlow from 2.12 to 2.13
  • Horovod has been removed from the conda environment in this release. See Notice for details on installing horovod.

Version 73.1

Release Date: 2023-06-12

Updated

• Updated PyTorch from 2.0.0 to 2.0.1

Version 73.0

Release Date: 2023-05-30

Removed

• Removed out of support frameworks as below:
  • All MXNet conda environments such as mxnet_p38 and aws_neuron_mxnet_p37
  • All Elastic Inference conda environments such as amazonei_mxnet_p36, amazonei_pytorch_latest_p36 and amazonei_tensorflow2_p36

Version 72.0

Release Date: 2023-05-10

Update

• Updated conda environment name from aws_neuron_pytorch_p37 to aws_neuron_pytorch_p38 to update PyTorch version to 1.13 with python3.8 support
• Updated conda environment name from aws_neuron_tensorflow2_p37 to aws_neuron_tensorflow2_p38 to update TensorFlow version to 2.10 with python3.8 support

Version 71.0

Release Date: 2023-03-30

Update

• Updated pytorch_p39 to pytorch_p310
• PyTorch version updated from 1.13.1 to 2.0 in the pytorch_p310 environment.
• PyTorch package comes with statically linked custom NCCL 2.16.2 supporting the dynamic buffer depth patch and it won’t use custom NCCL. Custom NCCL source code available at: https://github.com/NVIDIA/nccl/tree/inc_nsteps
• PyTorch package comes with dynamically linked AWS OFI NCCL plugin as a conda package aws-ofi-nccl-dlc package as well and PyTorch will use that package instead of system AWS OFI NCCL.
• Torch.Compile Support
  • PT 2.0 includes the use of torch.compile() for training. Please refer to PyTorch Documentation here on usage.
  • Torch.compile is a beta feature of PyTorch 2.0 and we are continuing to test it extensively on AWS. As of this release:
    • Torch.compile has been tested on P4, P3 and G5 instances.
    • Torch.compile has been tested with its default setting using TorchInductor as backend.
    • Torch.compile has been tested at float32 precision for training
  • NOTE: The “Triton” based Torch Inductor is not supported on G3 instance. In this case, “eager mode” will need to be used. Please see OpenAI Triton GPU compatibility here.
• Horovod is supported in the current pytorch_p310 conda environment on the DLAMI. However, Horovod will be removed from the conda environment for upcoming version of PyTorch v2.1. Customers will be able install the horovod libraries by following the guidelines and install them on their DLAMIs for their distributed training jobs.

Removed

• We have temporarily not added fastai package due to their pending inductor backend support on their end. Once fastai adds support, we will add it back in upcoming releases.

Version 70.0

Release Date: 2023-03-24

Update

• Updated Nvidia Driver version from 515.65.01 to 525.85.12
• Added support for G5 instance type.
• TensorFlow version updated from 2.11.0 to 2.12.0 in the tensorflow2_p310 environment.
• Removed `tensorflow-serving-api` package since there is no v2.12.0 available at this time.
• Horovod is supported in the current `tensorflow2_p310` conda environment on the DLAMI. However, Horovod will be removed from the conda environment for upcoming version of TensorFlow v2.13. Customers will be able install the horovod libraries by following the guidelines and install them on their DLAMIs for their distributed training jobs.

 

Version 69.1

Release Date: 2022-12-28

Update

• PyTorch version updated from 1.13.0 to 1.13.1 in the pytorch_p39 environment.

Version 69.0

Release Date: 2022-12-05

Update

• TensorFlow version updated from 2.10.0 to 2.11.0 in the tensorflow2_p310 environment.

Version 68.0

Release Date: 2022-11-09

Added

• Added cuda-11.7 at /usr/local/cuda-11.7/

Update

• PyTorch from 1.12 to 1.13 in pytorch_p39 environment.
• Updated NVIDIA Driver from 510.47.03 to 515.65.01

Version 67.0

Release Date: 2022-11-01

Update

• Updated base environment from Python 3.9 to Python 3.10
• Updated python3 environment from Python 3.9 to Python 3.10
• Updated tensorflow_p39 environment to tensorflow_p310, from Python 3.9 to Python 3.10

Version 66.4

Release Date: 2022-09-26

Update

• PyTorch version updated from 1.12.0 to 1.12.1 in the pytorch_p39 environment.

Version 66.2

Release Date: 2022-09-19

Update

• TensorFlow version updated from 2.9.2 to 2.10.0 in the tensorflow2_p39 environment.

Version 66.0

Release Date: 2022-09-15

Update

• TensorFlow patch version updated from 2.9.1 to 2.9.2 in the tensorflow2_p39 environment.

Version 65.0

Release Date: 2022-07-20

Update

• Updated TensorFlow to 2.9 in the tensorflow2_p39 environment and PyTorch to 1.12 in the pytorch_p39 environment
• Updated TensorFlow-Neuron 1.15 to TensorFlow-Neuron 2.8 in the aws_neuron_tensorflow2_p37 environment
• Removed TFS from the tensorflow2_p39 environment

Version 64.0

Release Date: 2022-07-15

Update

• Updated TensorFlow to 2.8
  • Updated conda env tensorflow2_p38 to tensorflow2_p39 to update TensorFlow version from 2.7 with python 3.8 to 2.8 with python 3.9
• TFS will be deprecated in upcoming release as it is recommend by TFS to use docker https://github.com/tensorflow/serving#set-up
• EFA version updated to 1.16.0

Version 63.0

Release Date: 2022-07-01

Update

• Update Tensorflow Neuron Conda framework
  • Updated conda env aws_neuron_tensorflow_p36 (Python 3.6) to aws_neuron_tensorflow_p37 (Python 3.7).
• Update Mxnet Neuron Conda framework
  • Updated conda env aws_neuron_mxnet_p36 (Python 3.6) to aws_neuron_mxnet_p37 (Python 3.7).
• Update PyTorch Neuron Conda framework
  • Updated conda env aws_neuron_pytorch_p36 (Python 3.6) to aws_neuron_pytorch_p37 (Python 3.7).

Version 62.1

Release Date: 2022-06-22

Update

• Updated opencv-python to >=4.6.0 in all conda environments

Version 62.0

Release Date: 2022-06-09

Update

• Updated Apache MXNet to 1.9
  • Updated conda env mxnet_p37 to mxnet_p38 to update Apache MXNet version from 1.8 with python 3.7 to 1.9 with python 3.8.
• Updated PyTorch to 1.11
  • Updated conda env pytorch_p38 to pytorch_p39 to update PyTorch version from 1.10 with python 3.8 to 1.11 with python 3.9
• Updated conda env python3 from python version 3.8 to 3.9

Version 61.2

Release Date: 2022-05-20

Fixed

• For conda environment tensorflow2_p38, fixed 'GLIBC_2.27' not found error in tensorflow2_model_server

Version 61.1

Release Date: 2022-04-28

Added

• Added Amazon CloudWatch Agent, for more details please refer https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html
• Added three systemd services which uses predefined json files available at path /opt/aws/amazon-cloudwatch-agent/etc/ to configure GPU metrics using linux user cwagent
  • dlami-cloudwatch-agent@minimal
    • Commands to enable GPU metrics:
      • sudo systemctl enable dlami-cloudwatch-agent@minimal
      • sudo systemctl start dlami-cloudwatch-agent@minimal
    • It creates below metrics:
      • “utilization_gpu”,
      • “utilization_memory”
  • dlami-cloudwatch-agent@partial
    • Commands to enable GPU metrics:
      • sudo systemctl enable dlami-cloudwatch-agent@partial
      • sudo systemctl start dlami-cloudwatch-agent@partial
    • It creates below metrics:
      • "utilization_gpu",
      • "utilization_memory",
      • "memory_total",
      • "memory_used",
      • "memory_free"
  • dlami-cloudwatch-agent@all
    • Commands to enable GPU metrics:
      • sudo systemctl enable dlami-cloudwatch-agent@all
      • sudo systemctl start dlami-cloudwatch-agent@all
    • It creates all available GPU metrics

Version 60.0

Release Date: 2022-03-18

Added

• Updated kernel version from 4.14 to 5.10. Current version is 5.10.102-99.473.amzn2.x86_64

Version 59.0

Release Date: 2022-03-04

Added

• Updated Nvidia Driver to 510.47.03