Visual Flow, powered by Apache Spark is a cloud-native open-source ETL tool based on Apache Spark unified analytics engine with a Graphical User Interface. It combines the best attributes of Kubernetes, Spark, and Argo into a single system with the following features:

Portability, flexibility and multi-cloud compatibility
Increased developer productivity
High availability, performance and fault tolerance
Cost-effective
Open-source

Visual Flow provides a Graphical User Interface to create ETL jobs and combine them in Data Processing Pipelines, run/schedule them, and monitor execution. It is a multi-tenant system. You can create multiple projects in a single cluster. Each project is represented as a namespace, which allows allocating resources and providing access to the users in different roles.

Visual Flow does not require a database. It is possible to create all objects as native Kubernetes resources. Our tool leverages Kubernetes authorization to manage users and their roles within a project.

Visual Flow provides the ability to create parameters (e.g., connection info) and reuse them in Jobs.

Why Visual Flow?

Graphical User Interface speeds up learning and jobs development
Good monitoring and alerting capabilities out of the box
Error Handling out of the box
Organizing jobs into Data pipelines
Unlimited parallelism
Unlimited scalability
Easy to customize and extend
Easy to test automatically
Can be installed on-premise as well as on cloud provider of your choice
Can be packaged and provided as SaaS solution on your own Cloud platform