Generative AI on Amazon Elastic Kubernetes Service (EKS)

The generative AI/ML space is rapidly evolving, with many innovations coming from the open-source community, which often includes native Kubernetes integration. Amazon EKS offers unparalleled customization, allowing organizations to control their infrastructure down to the instance level and configure environments to meet specific requirements, including support for all Amazon EC2 instance types and specialized GPUs. Amazon EKS provides versatile deployment options across on-premises, edge, and cloud environments, along with seamless scalability and continuous cost optimization through tools like Karpenter and GPU sharing mechanisms.

What You’ll Gain:

In this session, we will explore the reasons why customers are choosing to run generative AI on Amazon EKS, along with their common use cases. We will then delve into the technical details of how generative AI operates on Amazon EKS. Finally, we will demonstrate these concepts in a live demo and discuss best practices for deploying and performing inference with large language models (LLMs) on Amazon EKS.

Agenda

• AI on Amazon EKS Introduction
• • Benefits of using Amazon EKS for AI
• Infrastructure Options for AI/ML Workloads on Amazon EKS
• • Compute Resources
  • Storage
  • Amazon EKS - optimized accelerated AMI
• Inference and Training on Amazon EKS
• • Building Blocks
  • Essential Frameworks for Deploying Generative AI on Amazon EKS
  • Efficiently Scaling LLMs using vLLM, Ray, and AWS Neuron
  • Observability for Generative AI workloads on Amazon EKS
  • Deep Learning Containers
• Customer Use Cases
• • GPU sharing using Time Slicing and Multi-Instance GPU (MIG)
  • Optimize Model loading and container startup time on Amazon EKS
  • Running RAG workloads on Amazon EKS
  • Building Model Context Protocol on Amazon EKS

By registering, you agree to the AWS Event Terms & Conditions and AWS Code of Conduct.