Containers

Category: Analytics

Spark on Amazon EKS networking – Part 2

This post was co-authored by James Fogel, Staff Software Engineer on the Cloud Architecture Team at Pinterest Part 2: Spark on EKS network design at scale Introduction In this two-part series, my counterpart, James Fogel (Staff Cloud Architect at Pinterest), and I share Pinterest’s journey designing and implementing their networking topology for running large-scale Spark […]

Spark on Amazon EKS networking – Part 1

This post was co-authored by James Fogel, Staff Software Engineer on the Cloud Architecture Team at Pinterest Part 1: Design process for Amazon EKS networking at scale Introduction Pinterest is a platform that helps inspire people to live a life they love. Big data and machine learning (ML) are core to Pinterest’s platform and product, […]

Analyze EKS Fargate costs using Amazon Quicksight

Introduction AWS Fargate is a serverless compute engine for running Amazon Elastic Kubernetes Service (Amazon EKS) and Amazon Elastic Container Service (Amazon ECS) workloads without managing the underlying infrastructure. AWS Fargate makes it easy to provision and scale secure, isolated, and right-sized compute capacity for containerized applications. As a result, teams are increasingly choosing AWS […]

Run Spark-RAPIDS ML workloads with GPUs on Amazon EMR on EKS

Introduction Apache Spark revolutionized big data processing with its distributed computing capabilities, which enabled efficient data processing at scale. It offers the flexibility to run on traditional Central Processing Unit (CPUs) as well as specialized Graphic Processing Units (GPUs), which provides distinct advantages for various workloads. As the demand for faster and more efficient machine […]

Using SBOM to find vulnerable container images running on Amazon EKS clusters

Introduction When you purchase a packaged food item in your local grocery store, you probably check the list of ingredients written to understand what’s inside and make sure you aren’t consuming ingredients inadvertently that you don’t want to or are known to have adverse health effects. Do you think in a similar way when you […]

Introducing Data on EKS – Modernize Data Workloads on Amazon EKS

Introduction We are thrilled to introduce Data on EKS (DoEKS), a new open-source project aimed at streamlining and accelerating the process of building, deploying, and scaling data workloads on Amazon Elastic Kubernetes Service (Amazon EKS). With DoEKS, customers get access to a comprehensive range of resources including Infrastructure as Code (IaC) templates, performance benchmark reports, […]

Getting visibility into your Amazon EKS Cross-AZ pod to pod network bytes

Getting visibility into your Amazon EKS Cross-AZ pod to pod network bytes

Introduction Many customers use Amazon Elastic Kubernetes Service (Amazon EKS) to host their mission-critical applications. As a best practice, we ask our customers to spread their applications across multiple distinct availability zones (AZ). Because “everything fails all the time,” Werner Vogel, CTO, Amazon To achieve high availability, customers deploy Amazon EKS worker nodes (Amazon EC2 […]

Using Amazon EMR on Amazon EKS for transient EMR clusters

Using Amazon EMR on Amazon EKS for transient EMR clusters

Introduction Many organizations as part of their cloud journey into Amazon Web Services migrate and modernize their ETL (extract-transform-load) batch processing workloads running on on-premises Hadoop clusters to AWS. They often start their journey with the lift and shift approach, by hosting their Hadoop environment on Amazon Elastic Compute Cloud (Amazon EC2) or migrate to […]

featured image with blog title

Shipping logs to third-parties with Kinesis Data Firehose and Fluent Bit for Amazon EKS on AWS Fargate

AWS Fargate is a technology that provides on-demand capacity for running pods on EKS clusters. Fargate provides a more hands-off experience, helping you run container applications without needing to manage the EC2 instances underneath. AWS Fargate runs each Kubernetes pod in its own isolated security boundary. This means it has a slightly different operating model […]

Fluent Bit for Amazon EKS on AWS Fargate is here

Akshay Ram, Prithvi Ramesh, Michael Hausenblas In issue 701 of our containers roadmap we discussed supporting our CNCF Fluent Bit-based log router in the context of EKS on Fargate. In this blog post we provide you context on this new feature and walk you through the usage of it, shipping logs directly to CloudWatch with […]