Posted On: Jul 31, 2023
Amazon EMR Serverless is a serverless option that makes it simple for data analysts and engineers to run open-source big data analytics frameworks like Apache Spark and Apache Hive without configuring, managing, and scaling clusters or servers. Starting today, you can call the EMR Serverless APIs to view the Application UIs e.g. the live Spark UI or Tez UI for running jobs and the Spark History Server or the persistent Tez UI for completed jobs.
Spark or Hive workloads are often submitted using orchestration tools like Apache Airflow, AWS Step Functions etc. To monitor such pipelines it is useful to be able to navigate to the live application UIs for running jobs and the persistent application UIs for completed jobs. The EMR Serverless application UIs are available from the EMR Studio UI but for users who prefer an API, they can now use the get-dashboard-for-job-run API to get a secure URL to view the application UIs.
This feature is available in all regions where Amazon EMR Serverless is available.
To learn more about this feature, view the EMR Serverless documentation and API docs.