AWS Big Data Blog
Author: Matthew Tan
I am a Senior Analytics Solutions Architect in the US Northeast team working across the Analytics domain providing guidance to customers developing solutions with AWS Analytics services on their analytics workloads.
Stream Amazon EMR on EKS logs to third-party providers like Splunk, Amazon OpenSearch Service, or other log aggregators
Spark jobs running on Amazon EMR on EKS generate logs that are very useful in identifying issues with Spark processes and also as a way to see Spark outputs. You can access these logs from a variety of sources. On the Amazon EMR virtual cluster console, you can access logs from the Spark History UI. […]