StreamSets Transformer is an execution engine within the StreamSets DataOps platform that allows users to create data processing pipelines that execute on Spark. Using a simple to use drag and drop UI users can create pipelines for performing ETL, stream processing and machine learning operations. It allows everyone, not just the savvy Spark developer, but also the Data Analyst, Data Scientist or legacy ETL developer to fully utilize the power of Apache Spark without requiring a deep technical understanding of the platform.

Transformer pipelines are heavily instrumented and provide deep visibility into the execution of Spark applications. Users can see exactly how long every operation takes, how much data gets transferred at every stage, and view proactive and contextual error messages if and when problems occur. These features further abstract the user away from the internals of the Spark cluster and allow them to solve the core business problem.

Pipelines can read from and write to Batch or Streaming sources and destinations and mix and match as they choose; users never have to make batch, streaming, lamda, kappa architectural decisions when designing pipelines instead they focus on working with Continuous Data (data as, when and where they need).