Sold by: IBA Group
Visual Flow, powered by Apache Spark is a cloud-native ETL tool based on Apache Spark with a Graphical User Interface. It combines the best features of Kubernetes, Spark, and Argo into a single system.
Overview
Video
Video 1
Video
Video 2
Video
Product video
Visual Flow, powered by Apache Spark is a cloud-native open-source ETL tool based on Apache Spark unified analytics engine with a Graphical User Interface. It combines the best attributes of Kubernetes, Spark, and Argo into a single system with the following features:
Portability, flexibility and multi-cloud compatibility Increased developer productivity High availability, performance and fault tolerance Cost-effective Open-source
Visual Flow provides a Graphical User Interface to create ETL jobs and combine them in Data Processing Pipelines, run/schedule them, and monitor execution. It is a multi-tenant system. You can create multiple projects in a single cluster. Each project is represented as a namespace, which allows allocating resources and providing access to the users in different roles.
Visual Flow does not require a database. It is possible to create all objects as native Kubernetes resources. Our tool leverages Kubernetes authorization to manage users and their roles within a project.
Visual Flow provides the ability to create parameters (e.g., connection info) and reuse them in Jobs.
Why Visual Flow?
Graphical User Interface speeds up learning and jobs development Good monitoring and alerting capabilities out of the box Error Handling out of the box Organizing jobs into Data pipelines Unlimited parallelism Unlimited scalability Easy to customize and extend Easy to test automatically Can be installed on-premise as well as on cloud provider of your choice Can be packaged and provided as SaaS solution on your own Cloud platform
Highlights
- Unique open-source ETL tool with a Graphical User Interface Leverages the best features of Apache Spark, Kubernetes and Argo. Unlimited parallelism and scalability
- Visual Flow landing page https://visual-flow.com/
- Visual Flow on YouTube https://tinyurl.com/VisualFlow-playlistst
Details
Sold by
Delivery method
Delivery option
Deployment into EKS
Latest version
Operating system
Linux
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
This product is available free of charge. Free subscriptions have no end date and may be canceled any time.
Vendor refund policy
Open source software is freely available to use and made by the people, for the people.
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
Deployment into EKS
Supported services: Learn more
- Amazon EKS
Container image
Containers are lightweight, portable execution environments that wrap server application software in a filesystem that includes everything it needs to run. Container applications run on supported container runtimes and orchestration services, such as Amazon Elastic Container Service (Amazon ECS) or Amazon Elastic Kubernetes Service (Amazon EKS). Both eliminate the need for you to install and operate your own container orchestration software by managing and scheduling containers on a scalable cluster of virtual machines.
Version release notes
1.5.0 (2025-03-11)
Improvements and new features:
- Added GenAI feature (new stage).
- Added Incremental load feature (Read stage).
- Added Interactive mode feature (new 'Debugging' status) with the ability to preview metadata and data exemplar.
- Added SQL editor with syntax markup.
- Added new connectors (Databricks, Azure Blob Storage, Google Cloud Storage, Kafka).
- Added new types, formats and parcers (Delta, Binary, PDF, DOC/DOCX etc).
- Added Sandbox mode.
- Updated Spark version to 3.4.1.
- Added hints for stage helper.
- Changed search crieria for Users/Roles to search by the name instead of id.
- Improved CDC stage: processing null values.
- Added ability to copy/paste stages between jobs within the same project (namespace).
- Changed boolean drop down lists to a new toggle element.
- Improved security to manage connection passwords.
- Improved search placeholders.
- Improved Avro schema.
- Improved UI: prevention to run empty jobs and pipelines.
- Improved import/export functionality.
- Added new sort option 'By last edit' for Jobs and Pipelines.
- Improved sorting method.
- Added possibility to sort jobs/pipelines in Pipeline Designer.
- Cosmetic updates in JD for drop down lists, scroll bars and multiline fields.
Fixed:
- Date/time stage issue.
- Role-based issues.
- Jobs/Pipelines dependency issue for 'Copy' action.
- Notification parameters issue.
- Fields duplication issue.
- Drop-down list moves while scrolling the page issue.
- Logs level list moves while choosing a level.
- Incorrect rows number in resulting STDOUT for write stage issue.
- Incorrect data in Dataframe modal window.
- Typo issues in different modes.
- Jobs vulnerabilities (CVE-2022-33891, CVE-2021-25642, CVE-2021-33036, CVE-2021-37404, CVE-2020-36632, CVE-2020-10650, CVE-2020-9480, CVE-2008-1997, CVE-2021-32626, CVE-2021-40531, CVE-2018-7489, CVE-2020-35491, CVE-2020-35490, CVE-2020-10673, CVE-2017-7525, CVE-2022-25168, CVE-2023-25194, CVE-2019-14887, CVE-2023-22946, CVE-2020-8840).
- DBService vulnerabilities (CVE-2024-45772, CVE-2024-1597, CVE-2024-7254, CVE-2024-32888, CVE-2024-22262, CVE-2024-38809, CVE-2024-22243, CVE-2024-38816).
Additional details
Usage instructions
Use provided Helm chart to deploy Visual Flow containers into EKS
Resources
Support
Vendor support
Support options:
Essential support - EU Business hours Advanced support - 24x5 Premium support - 24x7 + Technical Account Manager
Additionally on demand:
Professional Services Architect's consult Initial installation Feature requests
Contact details for pre-purchase support: info@visual-flow.com Contact details for post-purchase support: info@visual-flow.com
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
Data Streaming Architecture Service Managed by IOanyT Innovations
By IOanyT Innovations, Inc.
Data Streaming Architecture refers to the framework and processes designed to efficiently collect, process, and deliver real-time data from various sources to enable seamless data integration, analysis, and decision-making. It involves the continuous and uninterrupted flow of data from its origin to its destination, ensuring minimal latency and maximum reliability. Data streaming architectures leverage technologies such as Apache Kafka, Apache Flink, or Apache Spark Streaming to handle high volumes of data in motion.
Visual Studio Professional 2022
By Amazon Web Services
Amazon EC2 running Microsoft Visual Studio on Microsoft Windows Server is a fast and dependable environment that provides the latest platform capabilities with enhanced development tools and quality enablement. Amazon EC2 enables you to run compatible Windows-based solutions on AWS' high-performance, reliable, cost-effective, cloud computing platform. Supported Version is Visual Studio Professional 2022. IT administrators or license administrators can easily manage Visual Studio licenses and access for their end users via AWS License Manager. The product is currently supported on Windows Server 2016, Windows Server 2019 and Windows Server 2022 as base operating systems.
Visual Studio Enterprise 2022
By Amazon Web Services
Amazon EC2 running Microsoft Visual Studio on Microsoft Windows Server is a fast and dependable environment that provides the latest platform capabilities with enhanced development tools and quality enablement. Amazon EC2 enables you to run compatible Windows-based solutions on AWS' high-performance, reliable, cost-effective, cloud computing platform. The product is currently supported on Windows Server 2016, Windows Server 2019 and Windows Server 2022 as base operating systems.
Visual Studio Code Server packaged by Code Creator
By Code Creator
This product has a fee associated with the provision and deployment of the application and AMI support. Code-server is a Visual Studio Code instance running on a remote server accessible through any web browser.
Visual Flow for Databricks®
By IBA Group
Visual Flow is an ETL/ELT tool designed for effective data management via convenient and user-friendly interface developed for Databricks environment.
Customer reviews
No customer reviews yet
Be the first to write a review for this product.