Posted On: Jan 6, 2022
Amazon EMR on EKS enables customers to easily run open-source big data frameworks such as Apache Spark on Amazon EKS. Amazon EMR on EKS customers setup and use a managed endpoint (available in preview) to run interactive workloads using integrated development environments (IDE) such as EMR Studio.
Data scientists and data analysts may need to use libraries in their code that are not available in the public distribution of Amazon EMR on EKS Spark runtime. Customers use custom images functionality to create Spark driver and executor images that contain their application dependencies. Until now, customers who used Amazon EMR on EKS’ managed endpoints to run interactive workloads were unable to use custom images. With this release, you can now build a custom image with your application dependencies and use the image in your interactive jobs run using managed endpoints.
To learn more about how to build and use a custom image, please visit our documentation. Custom image support for managed endpoints is supported for all Amazon EMR on EKS releases and is available in all regions where Amazon EMR on EKS is currently available.