Join us for a comprehensive session on running AI/ML & GenAI inference workloads on Amazon EKS. This technical deep dive will explore how to effectively deploy and manage inference workloads while optimizing for both cost and latency.

We'll cover essential production best practices including scalability, performance, governance, and security, using DoEKS blueprints as our foundation.

We'll demonstrate advanced compute orchestration using Karpenter for efficient node provisioning and Ray for distributed AI/ML workloads. Learn how to leverage Spot instances effectively to significantly reduce costs while maintaining reliability for your inference services.

Agenda:

• Machine Learning on EKS
• Generative AI Overview
• Amazon EKS Infrastructure for Generative AI Workloads
• Running Generative AI Workloads on EKS
• Takeaways