Getting Started with the AWS Deep Learning Conda and Base AMIs
Today AWS announced the availability of two new versions of the AWS Deep Learning AMI: a Conda-based AMI and a Base AMI. This post will walk you through the instructions and additional resources for making the most of your new AMIs.
New Deep Learning AMI with Conda-managed environments
The new Deep Learning AMIs for Amazon Linux and Ubuntu come pre-installed with Python environments for deep learning created using Conda – a popular open source package and environment management tool. The Conda-managed Python environments are pre-configured for popular deep learning frameworks including Apache MXNet, TensorFlow, Caffe2, PyTorch, Keras, CNTK, and Theano. In addition, each Python environment comes in two flavors – Python 2 and Python 3. After you log in to your AWS EC2 instance using the AWS Management Console, you will be greeted with a console message that lists all the Conda environments.
You can also get this list by running the following command:
Next, to activate a Python environment for a deep learning framework of your choice, say MXNet, run the following:
For Python 2
For Python 3
After you are inside the Python environment, you can view the list of installed packages by running the following command:
Running your deep learning code inside the Python environment is simple. First start the python shell:
Then import a deep learning framework, or run your deep learning Python code like you normally would:
Now let’s switch to another deep learning framework, say TensorFlow. First exit the python shell
Then deactivate the current environment for MXNet
Then switch to a Python environment like you did before, but this times you’ll activate TensorFlow:
For Python 2
For Python 3
To read more about managing Conda environments, you can get the Conda commands cheat sheet and additional learning resources in the getting started guide for Conda. You can also go to the new AWS Deep Learning AMI doc site and explore the introductory tutorials that you can run directly from the command line.
Manage Conda from your Jupyter notebook Interface
You can also manage the Conda environments straight from your Jupyter notebook browser interface. You can launch a Jupyter notebook server on the Conda-based AMI using instructions on our doc site. Conda supports close integrations to the Jupyter notebook with following features:
Choosing your deep learning environment
First access the Jupyter server on your internet browser. On the main Files page you can choose a Conda environment with the deep learning framework of your choice from the drop-down list, as shown in the following screen shot. You can then continue starting a new notebook. It will automatically be linked to the Python environment that you selected.
You can also use the drop-down list on this page to switch to another environment with a different deep learning framework. To help you get started with your first notebook, the Conda-based AMI comes bundled with several Jupyter notebooks and ready to launch tutorials.
Open the Conda tab, and you’ll see a dedicated page for managing the Conda environments on the AMI:
On this page, you can browse the list of all the pre-installed Conda environments, the software packages installed inside an environment, and even reconfigure an environment by upgrading packages or uninstalling them.
Configuring the new Deep Learning Base AMI
The Base AMI for Amazon Linux and Ubuntu comes with a foundational platform of GPU drivers and acceleration libraries you can use to deploy your own customized deep learning environment. By default, the AMI is configured with an NVidia CUDA 9 environment. However, you can also switch to a CUDA 8 environment by reconfiguring the environment variable LD_LIBRARY_PATH. You simply need to replace the CUDA 9 portion of the environment variable string with its CUDA 8 equivalent.
CUDA 9 portion of the LD_LIBRARY_PATH string (installed by default)
Replace with CUDA 8
About the Author
Sumit Thakur is a Senior Product Manager for AWS Deep Learning. He works on products that make it easy for customers to get started with deep learning on cloud, with a specific focus on making it easy to use engines on Deep Learning AMI. In his spare time, he likes connecting with nature and watching sci-fi TV series.