Join us for an immersive hands-on workshop exploring how to build and scale production-ready Generative AI deployments on Amazon EKS using NVIDIA GPUs. As organizations move beyond experimentation to production deployment of Gen AI applications, Kubernetes has emerged as a preferred platform for managing inference workloads at scale, offering robust orchestration, cost optimization, and enterprise-grade reliability.

Whether you're looking to deploy your first language model or scale existing Gen AI workloads, this workshop will provide you with best practices and hands-on experience using industry-leading tools and frameworks. Learn directly from AWS experts who have helped organizations successfully deploy and manage large-scale Gen AI infrastructure.

Agenda

Part 1: Overview of Inference on Amazon EKS:

• AI on Amazon EKS Introduction
• Infrastructure Options for AI/ML Workloads on Amazon EKS
• Technical Implementation of Inference on Amazon EKS

Part 2: Through hands-on labs and real-world use cases, you'll learn:

• Amazon EKS cluster setup optimized for NVIDIA GPU workloads
• Efficient model serving and scaling using vLLM
• Distributed inference architecture implementation with Ray
• Comprehensive vLLM and Ray observability using Prometheus and Grafana
• Best practices for production Gen AI deployments on Kubernetes

Who should attend?

• Machine Learning Engineers
• System Architects
• DevOps Professionals
• Solutions Architect
• Innovators

By registering, you agree to the AWS Event Terms & Conditions and AWS Code of Conduct.