Containers

Tag: Karpenter

Scaling a Large Language Model with NVIDIA NIM on Amazon EKS with Karpenter

Many organizations are building artificial intelligence (AI) applications using Large Language Models (LLMs) to deliver new experiences to their customers, from content creation to customer service and data analysis. However, the substantial size and intensive computational requirements of these models may have challenges in configuring, deploying, and scaling them effectively on graphic processing units (GPUs). […]

Announcing Karpenter 1.0

Introduction In November 2021, AWS announced the launch of v0.5 of Karpenter, “a new open source Kubernetes cluster auto scaling project.” Originally conceived as a flexible, dynamic, and high-performance alternative to the Kubernetes Cluster Autoscaler, in the nearly three years since then Karpenter has evolved substantially into a fully featured, Kubernetes native node lifecycle manager. […]

How Getir optimized their Amazon EKS compute using Karpenter

Introduction Getir is the pioneer of ultrafast grocery delivery. Getir was founded in 2015 and revolutionized last-mile delivery with its grocery in-minutes delivery proposition. Today, Getir is a conglomerate incorporating nine verticals under the same brand. Challenge Getir uses Amazon Elastic Kubernetes Service (Amazon EKS) to host applications on AWS. One of the foremost challenges […]

Cordial’s journey implementing Bottlerocket and Karpenter in Amazon EKS

Overview Cordial is a cross-channel marketing platform that offers tools to fully automate marketing strategies. By automating marketing execution, Cordial liberates technology teams to focus on their core strengths: building and creativity. It empowers technology teams to delegate data access and management to marketers, using Cordial’s robust platform to migrate, transform, and deliver complex data […]

How Thomson Reuters achieved 5X operational efficiency and 30% cost optimization with Plexus on Amazon EKS

Introduction In today’s dynamic business landscape, operational efficiency and cost optimization are two critical ingredients for a successful business outcome. This is especially true for companies navigating through digital transformation. In 2020, Thomson Reuters (TR), a leader at the intersection of content and technology with trusted data, committed to a cloud-first strategy with Amazon Web […]

Host the Whisper Model with Streaming Mode on Amazon EKS and Ray Serve

OpenAI Whisper is a pre-trained model for automatic speech recognition (ASR) and speech translation. It has demonstrated strong ASR performance across various languages, including the ability to transcribe speech in multiple languages and translate them into English. The Whisper model is open-sourced under the Apache 2.0 license, making it accessible for developers to build useful […]

Deploying Karpenter Nodes with Multus on Amazon EKS

Container based Telco workloads use Multus CNI primarily for traffic or network segmentation. Amazon Elastic Kubernetes Service (Amazon EKS) supports Multus CNI enabling users to attach multiple network interfaces, apply advanced network configuration and segmentation to Kubernetes-based applications running on AWS. One of the many benefits of running applications on AWS is resource elasticity (scaling out and scaling […]

Mobileye: Revolutionizing HD map creation for autonomous vehicles with Spark on Amazon EKS

Mobileye (Nasdaq: MBLY), a global leader in advanced driver-assistance systems (ADAS), is at the forefront of the autonomous driving revolution. Founded in 1999, they have pioneered groundbreaking technologies such as REM™ crowdsourced mapping, True-Redundancy™ sensing, and Responsibility-Sensitive Safety (RSS). These innovations are paving the way for a future filled with self-driving vehicles and advanced mobility […]

How Slack adopted Karpenter to increase Operational and Cost Efficiency

Bedrock – Slack’s internal Kubernetes platform Slack is the AI-powered platform for work that connects people, conversations, apps, and systems together in one place. Slack adopted Amazon Elastic Kubernetes Service (Amazon ) to build “Bedrock,” the codename for an internal compute orchestration platform that simplifies container deployment and management. Bedrock handles build, deploy, and runtime […]

How Upstox built Next-Generation trading platform using Amazon EKS, Karpenter, and Spot Instances

This is a guest post by Pranav Kapoor, Head of DevOps at Upstox co-authored with Jayesh Vartak, Solutions Architect at AWS and Jitendra Shihani, Technical Account Manager (TAM) at AWS. Upstox is India’s largest investech, a multi-unicorn valued at $3.5 billion. It allows you to buy and sell stocks, mutual funds, and derivatives, and is […]