Posted On: Jul 14, 2023
We are excited to announce that Amazon EMR on EKS now supports programmatic execution of Jupyter notebooks when running interactive workloads via managed endpoints. Amazon EMR on EKS enables customers to run open-source big data frameworks such as Apache Spark on Amazon EKS. Amazon EMR on EKS customers can setup and use a managed endpoint (available in preview) to run interactive workloads using integrated development environments (IDEs) such as EMR Studio.
Today, customers use Jupyter notebooks on Amazon EMR on EKS using managed endpoints with the convenience of EMR Studio Workspaces using a web-based interface. With programmatic execution for EMR on EKS managed endpoints, data engineers now have the added flexibility to run jupyter notebooks using scripts, chain multiple notebooks, or use orchestration services such as AWS Step Functions or Apache Airflow to build pipelines and run interactive workloads..
To learn more about this feature, please visit our programmatic execution documentation. Programmatic execution for managed endpoints is supported on Amazon EMR on EKS 6.10 release and later, and available in all regions where Amazon EMR on EKS is currently available.