Posted On: Sep 15, 2023
Today, we are excited to announce that Amazon EMR on EKS now supports Apache Flink, available in public preview. With this launch, customers who already use EMR can run their Apache Flink application along with other types of applications on the same Amazon EKS cluster, helping improve resource utilization and simplify infrastructure management. For customers who already run big data frameworks on Amazon EKS, they can now let Amazon EMR automate provisioning and management.
Apache Flink is an open-source framework that customers use to run real-time stream processing on high throughput data sources. Prior to today’s launch, customers could run self-managed Apache Flink applications on EKS, but they would be required to manually install and configure Apache Flink which requires in-depth understanding. With today’s launch, customers are only required to install the Flink Kubernetes operator and can use the operator to submit jobs directly to their EKS cluster while letting EMR on EKS manage the lifecycle of the Apache Flink application. This allows customers to share resources across their applications and use a single set of Kubernetes tools to centrally monitor and manage their infrastructure. Additionally, with this launch, customers also get access to key accessibility and monitoring features such as launching Flink application using jars in Amazon S3, monitoring integration with Amazon S3 and Amazon CloudWatch and container log rotation.
To learn more and get started, please visit our Apache Flink section documentation. Amazon EMR on EKS with Apache Flink is now available in preview in regions where Amazon EMR on EKS is currently available.
Oct 5, 2023 - The previous version of this post incorrectly referenced Apache Flink on Amazon EMR on EKS as a fully managed offering. We have now corrected this.