Making neural nets uncool again – AWS style

This is a guest post by Jeremy Howard, co-founder of fast.ai. 

Just as the goal of Amazon AI is to democratize machine learning with the development of platforms such as Amazon SageMaker,  the goal of fast.ai is to level the educational playing field so that anyone can pick up machine learning and be productive. The fast.ai tagline is “Making neural nets uncool again.” This is not a play to decrease the popularity of deep neural networks, but instead to broaden their appeal and accessibility beyond the academic elites who have dominated the research in this area.

With the explosion of use cases for deep learning such as computer vision, natural language processing, and machine translation, we’ve also seen an explosion in the developer community’s interest in learning about machine learning and applying it to multiple problems. To put this into perspective, Udacity, developers of the Deep Learning Nanodegree, have over 8 million users globally. More than 50,000 of those users are pursuing Nanodegrees, and a significant portion of those degrees are focused on deep learning. We are starting to see machine learning hit the mainstream, but it is still being taught in a way that often starts with the research first and follows with the applications. Enter fast.ai with its incredibly popular massive open online courses (MOOCs), where over 100,000 students study deep learning online while leveraging the AWS Cloud’s global footprint.

fast.ai’s 2017 course

fast.ai just completed recording their latest deep learning course in December 2017. The lessons form the basis of a recently released MOOC that went online at the end of 2017. The lessons were recorded live at the USF Data Institute, with 120 students (including more than 40 Diversity Fellows) attending in person, and 400 International Fellows participating through YouTube Live from all over the world.

Location of some fast.ai 2017 International Fellows

The students produced some fantastic work, including writing a number of articles that explain many of the concepts introduced in the course. Here’s a small sample of what they produced:

• Improving the way we work with learning rates explains recent advances in handling learning rate schedules, one of the most important parts of practical deep learning.
• Fun with small image data-sets shows how to get perfect accuracy on image classifiers training with just 14 images.
• Decoding the ResNet architecture is an excellent introduction to this ImageNet-winning architecture.
• Structured Deep Learning shows how to use deep learning to get state-of-the-art results with structured data, such as database tables and spreadsheets.
• How do we ‘train’ neural networks introduces the basic idea of gradient descent, the foundation of all deep learning.

A special gift 

There are “many small and seemingly trivial details that create barriers for students from developing countries who want to learn” says 2016 fast.ai International Fellow Tahsin Mayeesha from Bangladesh. fast.ai was concerned that it was hard for many of the International Fellows to get a credit card to allow them to sign up for AWS, and that the $0.90/hour cost could be prohibitive for many to access an AWS deep learning instance. Students need a GPU to train models in any reasonable time. It can cost $1000 or more to build a PC with a GPU for deep learning – as well as taking many hours and significant expertise to set up. AWS is therefore an ideal platform for students, since they can get started right away without any upfront investment.

We had previously received help from some passionate and thoughtful AWS folks in getting our students connected, so we reached out and asked if there was anything they could do to help. We were absolutely overwhelmed by the response: AWS offered around $250,000 in credits to our students to fund all of their fast.ai studies!

When we told our students that their AWS costs would be fully paid for, they were floored. On forums.fast.ai, where our deep learning community comes together, hundreds of students expressed their gratitude and excitement. Many of our students’ successes would not have happened if it weren’t for this wonderful gift.

fast.ai and AWS: making deep learning accessible 

The 2016 edition of fast.ai’s free online course, Deep Learning for Coders, has provided 25 hours of lessons to over 100,000 students with nothing but high school math and basic programming experience. They learn everything from how to use pre-trained networks for basic image recognition, all the way through to learning to implement the latest deep learning papers from scratch. Students are shown how to create and connect to a deep learning server hosted on AWS, which they use to complete the assignments during the course.

As Forbes explains in Artificial Intelligence Education Transforms The Developing World, these students have gone on to launch many important projects around the world. Thanks to AWS, now even more students have opportunities to become world-class deep learning practitioners.

Looking to teach a class or just the fastest way to do ML at scale? 

Amazon SageMaker is a fully-managed service that enables developers and data scientists to quickly and easily build, train, and deploy machine learning models at any scale. Its modular architecture can be used together or independently to fit your workflow. Amazon SageMaker provides the most common machine learning algorithms, optimized for performance, and reduces model tuning time through automatic hyperparameter optimization. Amazon SageMaker also allows you to bring your own container with whatever deep learning framework you prefer.  

What’s next? 

• Check out the latest fast.ai MOOC here to learn more from practical to cutting edge coding in deep learning.
• Give Amazon SageMaker a try with any one of several sample Jupyter notebooks including a blog post from AWS Evangelist Randall Hunt on how to get started.
• Leverage the Deep Learning AMI to get started quickly with frameworks such as PyTorch, Apache MXNet, and TensorFlow.

About the Authors

Jeremy Howard is a founding researcher at fast.ai, a research institute dedicated to making deep learning more accessible. He is also a Distinguished Research Scientist at the University of San Francisco, a faculty member at Singularity University, and a Young Global Leader with the World Economic Forum.

Joseph Spisak leads AWS’ partner ecosystem focused on Artificial Intelligence and Machine Learning. He has more than 17 years in deep tech working for companies such as Amazon, Intel and Motorola focused mainly on Video, Machine Learning and AI. In his spare time, he plays ice hockey and reads sci-fi.