Posted On: Oct 12, 2023
AWS Step Functions announces an Optimized Integration for Amazon EMR Serverless , adding support for the Run a Job (.sync) integration pattern with 6 EMR Serverless API Actions (CreateApplication, StartApplication, StopApplication, DeleteApplication, StartJobRun, and CancelJobRun).
EMR Serverless is a serverless option in Amazon EMR that makes it easy for data analysts and engineers to run open-source big data analytics frameworks like Apache Spark and Apache Hive without configuring, managing, and scaling clusters or servers. AWS Step Functions is a visual workflow service that makes it easy to compose AWS services into scalable, reliable, and resilient application components. Customers use the visual authoring and operator experience of Step Functions to create resilient and manageable multi-step EMR data processing pipelines. With this new Optimized Integration, customers can simplify these pipelines by removing steps used to monitor completion of asynchronous jobs and replace with a single Step Functions state.
This Optimized Integration for EMR Serverless is available in all AWS Regions where Amazon EMR Serverless is available.
To get started, you can use the new “Run an EMR Serverelss Spark Job” Sample Project for Step Functions in the AWS Console or build a workflow using Step Functions Worfklow Studio. To learn more, please see the AWS Step Functions Developer Guide and the Launch Blog to get started.