This cloudimg packaged AMI will allow you to quickly deploy and integrate open source Apache Airflow, enabling orchestration capabilities for AWS based Big Data and Machine Learning workloads, allowing Data Scientists to focus on SageMaker and EMR related developments rather than infrastructure preparations. This would be ideal for data scientists getting familiar with enterprise orchestration concepts and experimenting Apache Airflow in development environments using an EC2 instance. This image includes all up-to-date modules and prerequisites of Apache Airflow v2.x versions.

This AMI has been configured to be used as a Development Apache Airflow environment. This offering makes use of the bundled sqlite database component as opposed to MySQL/PostGreSQL advised for Production environments.

More comprehensive Apache Air Flow deployment guidance and relevant AWS cloud formation templates for production deployments can be found under the open source project Turbine; https://github.com/villasv/aws-airflow-stack.

More information regarding the Apache AirFlow can be found at https://airflow.apache.org/start.html