Detalles del evento
-
-
EN LÍNEA
- Events
- Cost-efficient GenAI/ML Inference with Spot on Kubernetes and Karpenter
Cost-efficient GenAI/ML Inference with Spot on Kubernetes and Karpenter
IA
Contenedores
IA generativa
Machine learning
Día:
-
Horario:
-
Tipo:
EN LÍNEA
Ponentes:
Eng-Hwa Tan | Principal Containers Specialist Solutions Architect, AWS, Utkarsh Pundir | Associate Solutions Architect (specializing in containers), AWS, Hoa Thieu Trinh | SA, Chief Engineer of Apero AI Lab
Idioma:
English
Niveles:
300: avanzado
Ponentes
Join us for a comprehensive session on running AI/ML & GenAI inference workloads on Amazon EKS. This technical deep dive will explore how to effectively deploy and manage inference workloads while optimizing for both cost and latency.
We'll cover essential production best practices including scalability, performance, governance, and security, using DoEKS blueprints as our foundation.
We'll demonstrate advanced compute orchestration using Karpenter for efficient node provisioning and Ray for distributed AI/ML workloads. Learn how to leverage Spot instances effectively to significantly reduce costs while maintaining reliability for your inference services.
Agenda:
- Machine Learning on EKS
- Generative AI Overview
- Amazon EKS Infrastructure for Generative AI Workloads
- Running Generative AI Workloads on EKS
- Takeaways