Tag: Data on EKS

Introduction Quora is a leading Q&A platform with a mission to share and grow the world’s knowledge, serving hundreds of millions of users worldwide every month. Quora uses machine learning (ML) to generate a custom feed of questions, answers, and content recommendations based on each user’s activity, interests, and preferences. ML drives targeted advertising on […]

Introduction Apache Kafka, a distributed streaming platform, has become a popular choice for building real-time data pipelines, streaming applications, and event-driven architectures. It is horizontally scalable, fault-tolerant, and performant. However, managing and scaling Kafka clusters can be challenging and often time-consuming. This is where Kubernetes, an open-source platform for automating deployment, scaling, and management of […]

Introduction In recent years, there’s been a remarkable surge in the adoption of Kubernetes for data analytics and machine learning (ML) workloads in the tech industry. This increase is underpinned by a growing recognition that Kubernetes offers a reliable and scalable infrastructure to handle these demanding computational workloads. Furthermore, a recent wave of Generative AI […]

This post was co-written with Derek Brown, Infrastructure Engineer, Lacework Introduction Lacework is a security platform that automatically monitors and detects misconfigurations and security vulnerabilities across our customers’ cloud environments. Lacework takes a data-driven approach to alerting and detection, consuming trillions of events from a huge list of sources including eBPF, Kubernetes Audit Logs, cloud […]

Introduction We are thrilled to introduce Data on EKS (DoEKS), a new open-source project aimed at streamlining and accelerating the process of building, deploying, and scaling data workloads on Amazon Elastic Kubernetes Service (Amazon EKS). With DoEKS, customers get access to a comprehensive range of resources including Infrastructure as Code (IaC) templates, performance benchmark reports, […]

Introduction Kubernetes has gained widespread adoption in the field of data processing because of its ability to package and deploy applications as containers with all required dependencies, as well as its support for running data frameworks. This makes it easy for developers to run their Data Analytics/Machine Learning (ML) applications within a Kubernetes cluster and […]