Olive - Data Ingestion Framework
Linux/Unix
Product Overview
Olive Data Ingestion Framework (ODIF), is a data ingestion tool which can connect to any source and sink to make data ingestion/transfer faster and easier. ODIF built with a cloud agnostic approach with no pre-installation of cluster and can be deployed with minimal resource footprint. It provides a user friendly web interface which helps user in, data source registration, job config, job runs and monitoring.
ODIF guiding principles are Cloud Native Design, Platform Agnostic, Dynamic Compute, API Driven, IaaC (Infrastructure as a Code)
ODIF Features
Reusable connectors - Once created connectors can be used as source as well as sink.
RDBMS Source - Provide feature to select multiple databases, tables, as well as, feature to select complete dataset or particular dataset with where clause.
Split Job - Job gets split based on dataset size when input source is large, which accelerate ingestion.
File Format - Support csv, txt, parquet and json file format at sink.
Load Type - Feature support incremental load for regular ingestion and full load for historical or one time load.
UI and REST API - Support web interface as well as REST APIs.
Job Scheduling - Job can be schedule and can run on given time interval.
Livy Support - Support Livy on static cluster.
Cluster Type - ODIF use cloud and platform agnostic approach, hence can be run on static as well as on-demand cluster (AWS,Azure,GCP)
Version
1.0.0
By
Abzooba IncCategories
Operating System
Linux/Unix, Amazon Linux Amazon Linux 2
Delivery Methods