AWS Deep Learning Containers v6.2 for Tensorflow

The AWS Deep Learning Containers for TensorFlow include containers for Training and Inference for CPU and GPU, optimized for performance and scale on AWS

Release Date: March 20, 2020
Created On: March 20, 2020
Last Updated: March 21, 2020

The AWS Deep Learning Containers for TensorFlow include containers for Training and Inference for CPU and GPU, optimized for performance and scale on AWS. These Docker images have been tested with Amazon SageMaker, EC2, ECS, and EKS and provide stable versions of NVIDIA CUDA, cuDNN, Intel MKL, Horovod, and other required software components to provide a seamless user experience for deep learning workloads. All software components in these images are scanned for security vulnerabilities and updated or patched in accordance with AWS Security best practices.

Detailed Release Note Changes

Security Advisory

  1. AWS recommends that customers monitor critical security updates in the AWS Security Bulletin
  2. Issue: pycrypto security vulnerability - pycrypto cannot be upgraded beyond v2.6.1 in Python 2 containers

Highlights of the Release

  • Introduced SageMaker Python SDK version 1.50.17 for Python3 Containers
  • Added debugging support for TensorFlow 2.1.0 machine learning models in smdebug 0.7.1 for Python3 Containers
  • Updated smexperiments version to 0.1.7 for Python3 Containers

Prepackaged Deep Learning Frameworks Included

  • TensorFlow: TensorFlow is an open source software library for numerical computation using data flow graphs.
    • branch/tag used : v2.1.0
      • Justification : Stable and well tested
    • Supported with CUDA 10.1 and Intel MKL-DNN v0.21
  • Keras: Deep Learning Library for Python
    • Tensorflow integration with Keras v2.3.1
      • Justification : Stable release
  • Horovod: Horovod is a distributed training framework. The goal of Horovod is to easily take single-GPU deep learning program and train it on multiple GPUs. Horovod nodes communicate directly with each other instead of going through a centralized node and average gradients using the ring-allreduce algorithm.
    • branch/tag used : v0.18.2
      • Justification : Stable and well tested
  • SageMaker Python SDK: SDK is an open source library for training and deploying machine learning models on Amazon SageMaker. With the SDK, you can train and deploy models using popular deep learning frameworks Apache MXNet and TensorFlow. You can also train and deploy models with Amazon algorithms, which are scalable implementations of core machine learning algorithms that are optimized for SageMaker and GPU training. If you have your own algorithms built into SageMaker compatible Docker containers, you can train and host models using these as well.
    • branch/tag used: v1.50.17
      • Justification : Stable and well tested

Bill of Materials: List of all components

  • CPU: Training container
    • sagemaker==1.50.17
    • sagemaker-tensorflow-training==3.1.7.dev0
    • sagemaker-tensorflow==2.1.0.1.0.0
    • sagemaker-containers==2.8.1
    • sagemaker-experiments==0.1.7 (py3)
    • smdebug 0.7.1 (py3)
    • numpy==1.16.6 (py2)
    • numpy==1.18.1 (py3)
    • OpenMPI=4.0.1
    • Horovod=0.18.2
    • scipy==1.2.2 (py2)
    • scipy==1.4.1 (py3)
    • scikit-learn==0.20.4 (py2)
    • scikit-learn==0.22
    • pandas==0.24.2 (py2)
    • pandas==1.0.1 (py3)
    • Pillow==6.2.2 (py2)
    • Pillow==7.0.0 (py3)
    • h5py==2.10.0
    • keras_applications==1.0.8
    • keras_preprocessing==1.1.0
    • keras==2.3.1
    • requests==2.22.0
    • awscli==1.18.22
  • CPU: Inference container
    • tensorflow-model-server==2.1.0
    • tensorflow-serving-api==2.1.0
    • awscli==1.18.22
  • GPU: Training Container
    • sagemaker==1.50.17
    • sagemaker-tensorflow-training==3.1.2.dev0
    • sagemaker-containers==2.8.1
    • sagemaker-tensorflow==2.1.0.1.0.0
    • sagemaker-experiments==0.1.7 (py3)
    • smdebug 0.7.1 (py3)
    • cuda-command-line-tools-10-1
    • cuda-cufft-10-1
    • cuda-curand-10-1
    • cuda-cusolver-10-1
    • cuda-cusparse-10-1
    • libcudnn7=7.6.2.24-1+cuda10.1
    • libnccl2=2.4.7-1+cuda10.1
    • libnccl-dev=2.4.7-1+cuda10.1
    • llibcublas10=10.1.0.105-1
    • OpenMPI=4.0.1
    • Horovod=0.18.2
    • numpy==1.16.6 (py2)
    • numpy==1.18.1 (py3)
    • scipy==1.2.2 (py2)
    • scipy==1.4.1 (py3)
    • scikit-learn==0.20.4 (py2)
    • scikit-learn==0.22 (py3)
    • pandas==0.24.2 (py2)
    • pandas==1.0.1 (py3)
    • Pillow==6.2.2 (py2)
    • Pillow==7.0.0 (py3)
    • h5py==2.10.0
    • keras_applications==1.0.8
    • keras_preprocessing==1.1.0
    • keras==2.3.1
    • requests==2.22.0
    • awscli==1.18.22
  • GPU: Inference Container
    • tensorflow-model-server==2.1.0
    • tensorflow-serving-api-gpu==2.1.0
    • cuda-command-line-tools-10-1
    • cuda-cufft-10-1
    • cuda-curand-10-1
    • cuda-cusolver-10-1
    • cuda-cusparse-10-1
    • libcudnn7: 7.6.2.24-1+cuda10.1
    • libnccl2: 2.4.7-1+cuda10.1
    • llibcublas10=10.1.0.105-1
    • libnvinfer6=6.0.1-1+cuda10.1
    • awscli==1.18.22

Python Support

Python 2.7 and Python 3.6 are supported in the containers for all of the installed deep learning frameworks.

End of Life Notices

The Python open source community has officially ended support for Python 2 on January 1, 2020. The TensorFlow community has also announced that the TensorFlow 1.15 and TensorFlow 2.1 releases will be the last ones supporting Python 2. DLC releases with the next versions of the TensorFlow frameworks will not contain the Python 2 containers. Updates to the Python 2 DLC will be provided on previously published DLC versions only if there are security fixes published by the open source community for those versions. Previous releases of the TensorFlow DLC that contain Python 2 will continue to be available.

CPU Instance Type Support

The containers supports CPU instance types. TensorFlow is built with support for Intel MKL2019 DNN library support.

GPU Instance Type support

The containers supports GPU instance types and contain the following software components for GPU support.

  • CUDA 10.1 / cuDNN 7.6.2.24-1+cuda10.1 / NCCL 2.4.7-1+cuda10.1

AWS Regions support


Region
Code

US East (Ohio)
us-east-2

US East (N. Virginia)
us-east-1

US West (Oregon)
us-west-2

US West (N. California)
us-west-1

Asia Pacific (Mumbai)
ap-south-1

Asia Pacific (Seoul)
ap-northeast-2

Asia Pacific (Singapore)
ap-southeast-1

Asia Pacific (Sydney)
ap-southeast-2

Asia Pacific (Tokyo)
ap-northeast-1

Central (Canada)
ca-central-1

EU (Frankfurt)
eu-central-1

EU (Ireland)
eu-west-1

EU (London)
eu-west-2

EU(Paris)
eu-west-3

SA (Sau Paulo)
sa-east-1

EU (Stockholm)
eu-north-1
AP East (Hong Kong) ap-east-1
ME South (Bahrain) me-south-1

Build and Test

  • Built on: c5.18xlarge
  • Tested on: c4.8xlarge, c5.18xlarge, g3.16xlarge, m4.16xlarge, p2.16xlarge, p3.16xlarge, p3dn.24xlarge
  • Tested with MNIST and Resnet50/ImageNet datasets on EC2, ECS AMI (Amazon Linux AMI 2.0.20190614) and EKS AMI (1.11-v20190614) and Amazon Sagemaker.

Known Issues

  1. Issue: pycrypto security vulnerability - pycrypto cannot be upgraded beyond v2.6.1 in Python 2 containers
  2. There are unresolved performance issues with TensorFlow 2.1: https://github.com/tensorflow/tensorflow/issues?utf8=%E2%9C%93&q=is%3Aopen+label%3A%22TF+2.1%22+label%3Atype%3Aperformance+