Posted On: Jul 12, 2021
Today, we are excited to announce AWS Deep Learning Containers (DLCs) with integrated SDKs for inference that enable customers to easily deploy Hugging Face models in Amazon SageMaker at scale. This was the number one requirement from customers that have been leveraging the Hugging Face AWS DLCs for training (Hugging Face training DLCs) that released in March 2021.
Starting today, Amazon SageMaker supports deploying Hugging Face models using the Hugging Face AWS Deep Learning Containers for inference (Hugging Face inference DLCs) in addition to the Hugging Face training DLCs. The Hugging Face inference DLC supports both TensorFlow and PyTorch frameworks giving our customers a choice. The Hugging Face inference DLCs simplify model hosting, enabling customers to deploy at scale in just a few minutes. This also enables customers to streamline their operations with SageMaker Pipelines. Finally, this release enables customers to deploy Hugging Face models directly from their Hugging Face Model Hub, or deploy a model that they previously refined with our Hugging Face training DLCs.
Since 2016, Hugging Face has been a leader in the NLP community with their transformers library and Model Hub which features over 10,000 pre-trained models that make it easier for developers to get started. With over 41,000 GitHub stars and over 25 million downloads, Hugging Face's transformers library has become the de facto for developers building NLP models. The Hugging Face inference DLC in the Amazon SageMaker Python SDK makes it easy for developers to deploy these models on AWS. The Hugging Face inference DLC contains the Hugging Face transformers library, Deep Learning (DL) framework, and a DL model server optimized for SageMaker. Developers can deploy their pre-trained Hugging Face models to AWS with minimal additional code compared to hosting a custom container. Developers working with Hugging Face models can now more easily develop on Amazon SageMaker as well as benefit from the cost-efficiency, scalability, production-readiness and high security bar that SageMaker provides for model hosting.
The Hugging Face inference DLC is available in all regions where Amazon SageMaker is available and comes at no additional cost. Read the blog and documentation to learn more, or access the sample notebooks to try out the new integrations.