StreamSets Transformer is a data pipeline engine designed for any developer or data engineer to build ETL and ML pipelines that execute on Spark-- without coding. Transformer pipelines also provide unparalleled visibility into the execution of Spark applications with data previews and easy trouble-shooting, reducing the time to design and operate pipelines on Spark for developers of all skill levels.

Design and operate ETL pipelines that harness the power of EMR to batch load data into S3, RDS or Redshift, and to perform ETL and ML operations to conform and curate data for Redshift, Delta Lake or other business-ready analytic and ML platforms.

StreamSets Transformer (Medium) allows a maximum of ten spark executors per pipeline.