Author: Elamaran Shanmugam

In this post, we focus on observing and scaling ML operations (MLOps) infrastructure on Kubernetes. MLOps platforms running on Amazon EKS provide powerful built-in capabilities for logging, monitoring, and alerting that are essential for maintaining healthy ML systems at scale.

This post looks at various options for container image caching, model training, and inferencing workloads. This post also discusses various storage options such as Amazon Simple Storage Service (Amazon S3), FSx for lustre, S3 Express One Zone, and Amazon S3 Connector for PyTorch.

In this post, we demonstrate how to monitor network performance for Amazon EKS workloads using new advanced network observability features powered by Network Flow Monitor. We explore how to capture Kubernetes-enriched network metrics, export them to AWS Managed Open-Source services like Amazon Managed Service for Prometheus and Amazon Managed Grafana, and visualize critical performance indicators including throughput, packet drops, latency, and connection states across your containerized services.

Kubernetes has become the de-facto standard for container orchestration, providing powerful capabilities for deploying and managing stateless workloads. However, users running stateful applications on Kubernetes face unique challenges, especially in VMware environments. A key issue is that the virtual disks used by stateful apps can’t be attached to pods as easily as ephemeral storage. The […]

Analyze the traffic patterns on any public-facing website or web app, and you’ll notice connection requests from all over the world. Apart from the intended traffic, a typical web application responds to requests from bots, health checks, and various attempts to circumvent security and gain unauthorized access. In addition to impacting your customer’s experience, these […]

Introduction AWS App Runner is a fully managed container application service that lets you build, deploy, and run containerized web applications and API services without prior infrastructure or container experience. Starting today, AWS App Runner supports deploying services from source code repositories that follow a monorepo structure. Now you can tell AWS App Runner the […]

Introduction When Amazon VPC Container Network Interface (CNI) plugin assigns IPv4 addresses to Pods, it allocates them from the VPC CIDR range assigned to the cluster. While it makes Pods first-class citizens within the VPC network, it often leads to exhaustion of the limited number of IPv4 addresses available in the VPCs. The long term […]

Introduction Kubecost provides real-time cost visibility and insights for teams using Kubernetes. It has an intuitive dashboard to help you understand and analyze the costs of running your workloads in a Kubernetes cluster. Kubecost is built on OpenCost, which was recently accepted as a Cloud Native Computing Foundation (CNCF) Sandbox project, and is actively supported […]

Introduction With the proliferation of open-source and commercial container software products, customers want to know upfront which products are compatible with the container orchestrator of their choice. Customers expect compatibility testing to be continuous and include all the available dimensions, such as Amazon Elastic Kubernetes Service (Amazon EKS) / Amazon EKS Anywhere (EKS-A) versions, OS, […]

Introduction Many AWS customers use Amazon Elastic Kubernetes Service (Amazon EKS) to run machine learning workloads. Containerization allows machine learning engineers to package and distribute models easily, while Kubernetes helps in deploying, scaling, and improving. When working with customers that run machine learning training jobs in Kubernetes, we ‘ve seen that as the data set […]