Posted On: Jun 15, 2020
Amazon EMR now persists the YARN timeline server and Tez user interface (UI) along with the event and container logs outside the cluster. You can now debug and monitor your YARN applications independent of the cluster’s life cycle by logging directly into the web interfaces using the EMR console. The YARN timeline server and Tez UI are open-source applications that provide metrics and visual tools for active and terminated clusters.
Debugging distributed applications requires parsing through logs from multiple sources. The native YARN (resource scheduler), Tez (big data framework), and Spark (analytics engine) improve debugging experience by overlaying the logs to the execution context and presenting an intuitive UI. However, you need to go through multiple steps by connecting to the master node to view the web interfaces on your EMR cluster for security reasons. Moreover, if you are running a transient cluster, the UI will not be available when the cluster is terminated. Amazon EMR simplifies the debugging and monitoring by introducing the persistent Spark History Server, YARN Timeline Server, and Tez UI accessible directly from the EMR console.
To get started with YARN Timeline Server and Tez UI, please visit AWS Management Console.
This feature is available on EMR version 6.0.0 and later, in the US East (N. Virginia and Ohio), US West (N. California and Oregon), Canada (Central), EU (Frankfurt, Ireland, and London), and Asia Pacific (Mumbai, Seoul, Singapore, Sydney, and Tokyo) Regions.