Posted On: Mar 10, 2023
We are excited to announce support for customer-level metrics when running interactive Spark workloads via managed endpoints. Amazon EMR on EKS enables customers to run open-source big data frameworks such as Apache Spark on Amazon EKS. Amazon EMR on EKS customers can setup and use a managed endpoint (available in preview) to run interactive workloads using integrated development environments (IDEs) such as EMR Studio.
Until now, customers running managed endpoints did not have a mechanism to monitor or visualize kernel-based execution behavior for them. Without visibility into metrics such as failures, latency or successful launches, customers might experience difficulty troubleshooting and understanding what is happening with a managed endpoint. With this release, customers will be able to monitor, create alarms and better troubleshoot issues in their managed endpoints, by leveraging metrics via Amazon CloudWatch for kernel lifecycle operations such as request counts, request latency, request errors and kernel launch failures.
To learn more about this feature and get a list of the supported metrics, please visit our documentation. Managed endpoints customer metrics are supported for Amazon EMR on EKS 6.10 release and above, and are available in all regions where Amazon EMR on EKS is currently available.