Sign in
Categories
Migration Mapping Assistant Your Saved List Partners Sell in AWS Marketplace Amazon Web Services Home Help

LightningFlow - Integrated orchestration tool based on Apache Airflow

LightningFlow - Integrated orchestration tool based on Apache Airflow

By: Lightning-Analytics Inc Latest Version: V1.1
Linux/Unix
Linux/Unix

Product Overview

LightningFlow comes pre-integrated with all the required Airflow libraries namely, webserver, scheduler and worker configurations, local Spark cluster, Apache Livy, and a postgres database. When you start the EC2 instance, all the required services are instantiated with standard configurations, thus enabling to start running and testing the DAGs instantaneously. The DAGs run on a local Spark cluster, thus eliminating the need to create an EMR cluster. However, appropriate permissions (e.g. S3 and RedShift access) need to be added to the LightningFlow EC2 through an IAM role. Alternatively, jobs can be submitted to an EMR cluster communicating via Livy REST API, which would require additional configurations. In case you need any assistance for configuring, deploying and running scalable jobs in the Production environment, please reach out to us at info@lightning-analytics.com for more details.

Version

V1.1

Operating System

Linux/Unix, CentOS 7

Delivery Methods

  • Amazon Machine Image

Pricing Information

Usage Information

Support Information

Customer Reviews