Listing Thumbnail

    Visual Flow for Databricks®

     Info
    Sold by: IBA Group 
    Visual Flow is an ETL/ELT tool designed for effective data management via convenient and user-friendly interface developed for Databricks environment.
    Listing Thumbnail

    Visual Flow for Databricks®

     Info
    Sold by: IBA Group 

    Overview

    Visual Flow for Databricks is a cloud-native open-source ETL tool developed for Databricks environment with a handy Graphical User Interface. It uses best Databricks practices and combines them into a single system with the following features: Portability, flexibility and multi-cloud compatibility Increased developer productivity High availability, performance and fault tolerance Cost-effective Open-source Visual Flow for Databricks provides a Graphical User Interface to create ETL jobs and combine them in Data Processing Pipelines, run/schedule them, and monitor execution. Individual cluster setup is available for every Job. It is a multi-tenant system. You can create multiple projects in a single cluster. Each project is represented as Databricks workspace. Visual Flow for Databricks supports all existing Databricks Cloud systems and auto-detects them from the project settings. Our solution provides the ability to securely manage parameters and connections and reuse them in Jobs. Import/Export functionality allows to migrate data between projects with ease. Why Visual Flow for Databricks? Supports all existing Databricks Cloud environments Graphical User Interface speeds up learning and jobs development Good monitoring and alerting capabilities out of the box Error Handling out of the box Organizing jobs into Data pipelines with the ability to run them on schedule Unlimited parallelism Unlimited scalability Easy to customize and extend Easy to test automatically Can be installed on-premise as well as on cloud provider of your choice Can be packaged and provided as SaaS solution on your own Cloud platform

    REQUIRED SERVICES & APPLICATIONS: Before deployment: GitHub  account for OAuth. After deployment: Databricks account. Guide to connect to Databricks 

    Highlights

    • Unique open-source ETL tool with a Graphical User Interface Developed for Databricks environment Unlimited parallelism and scalability
    • Visual Flow landing page https://visual-flow.com/
    • Visual Flow on YouTube https://tinyurl.com/VisualFlow-playlistst

    Details

    Delivery method

    Delivery option
    Helm chart to deploy Visual Flow for Databricks containers into EKS

    Latest version

    Operating system
    Linux

    Pricing

    Visual Flow for Databricks®

     Info
    This product is free. Subscriptions have no end date and can be canceled anytime.

    Vendor refund policy

    Open source software is freely available to use and made by the people, for the people.

    Legal

    Vendor terms and conditions

    Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .

    Content disclaimer

    Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.

    Usage information

     Info

    Delivery details

    Helm chart to deploy Visual Flow for Databricks containers into EKS

    Supported services: Learn more 
    • Amazon EKS
    Container image

    Containers are lightweight, portable execution environments that wrap server application software in a filesystem that includes everything it needs to run. Container applications run on supported container runtimes and orchestration services, such as Amazon Elastic Container Service (Amazon ECS) or Amazon Elastic Kubernetes Service (Amazon EKS). Both eliminate the need for you to install and operate your own container orchestration software by managing and scheduling containers on a scalable cluster of virtual machines.

    Version release notes

    0.2 (2024-08-19)

    Improvements and new features:

    • Added support of OAuth M2M as an authentication mechanism for Databricks service principles.
    • Added ability to configure OAuth while creating a project.

    Additional details

    Usage instructions

    1. Create EKS cluster 
    2. Install AWS Load Balancer Controller  and AWS EBS CSI driver  on top of your EKS cluser
    3. Create GitHub OAuth . Homepage URL = https://visual-flow-dummy-url.com/vf/ui/  and Authorization callback URL = https://visual-flow-dummy-url.com/vf/ui/callback 
    4. Install Helm chart
    # Admin users for this app. You can have more than 1 GitHub user, see example below
    export GITHUB_USER_LIST=GitHubUser1,GitHubUser2
    
    # use GITHUB_APP_ID and GITHUB_APP_SECRET from step #3
    export GITHUB_APP_ID=YOUR_GITHUB_APP_ID_HERE
    export GITHUB_APP_SECRET=YOUR_GITHUB_APP_SECRET_HERE
    
    # set the latest available version
    export VF_HELM_VERSION=0.2
    
    # helm install command
    helm upgrade -i vfdbricks-aws-app oci://709825985650.dkr.ecr.us-east-1.amazonaws.com/iba-group/visual-flow-databricks -n default \
      --set vfdbricks-aws-services.databricks.configFile.superusers={${GITHUB_USER_LIST}} \
      --set vfdbricks-aws-services.frontend.deployment.secretVariables.GITHUB_APP_ID=${GITHUB_APP_ID} \
      --set vfdbricks-aws-services.frontend.deployment.secretVariables.GITHUB_APP_SECRET=${GITHUB_APP_SECRET} \
      --debug \
      --version ${VF_HELM_VERSION}
    

    Find your autogenerated URL and replace visual-flow-dummy-url.com from step #2 with your autogenerated URL: kubectl get svc vfdbricks-aws-app-frontend -o yaml | grep hostname | cut -c 17-

    Resources

    Vendor resources

    Support

    Vendor support

    Support options:

    Essential support - EU Business hours Advanced support - 24x5 Premium support - 24x7 + Technical Account Manager

    Additionally on demand:

    Professional Services Architect's consult Initial installation Feature requests

    Contact details for pre-purchase support: info@visual-flow.com  Contact details for post-purchase support: info@visual-flow.com 

    AWS infrastructure support

    AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.

    Similar products

    Customer reviews

    Ratings and reviews

     Info
    0 ratings
    5 star
    4 star
    3 star
    2 star
    1 star
    0%
    0%
    0%
    0%
    0%
    0 AWS reviews
    No customer reviews yet
    Be the first to write a review for this product.