Posted On: Nov 4, 2022
We are excited to announce the general availability of job templates in Amazon EMR on EKS. Job templates allow you to create and store templates to configure Spark applications parameters. This helps you ensure consistent settings across applications by reusing and enforcing configuration overrides in data pipelines.
EMR on EKS applies default Spark settings to optimize an application’s performance when customers use the StartJobRun API to run Spark-based applications and data pipelines. With job templates, data engineers can now define reusable templates to apply additional customizations, configuring executor and driver compute capacity, setting security and governance properties such as IAM roles, and customized docker image to use across multiple applications and data pipelines. Additionally, customers can also enforce specific configuration values, preventing overrides when calling the StartJobRun API using the template. Job templates can be optionally encrypted with customer-managed master keys stored in AWS Key Management Service.
Job templates are now generally available in all AWS regions where the EMR on EKS feature is released. To learn more about how to use job templates, please visit our documentation.