AWS Open Source Blog

Category: Amazon Managed Workflows for Apache Airflow (Amazon MWAA)

solution architecture described in blog post

Amazon MWAA with AWS CodeArtifact for Python dependencies

This post was written by Dzenan Softic and Sam Dengler. Many organizations rely on Apache Airflow, an open source project, to orchestrate their data pipelines. In 2020, Amazon Web Services (AWS) released Amazon Managed Workflows for Apache Airflow (Amazon MWAA), which lets engineers focus on business solutions rather than on running and maintaining infrastructure for […]

Read More

Automating a DAG deployment with Amazon Managed Workflows for Apache Airflow

Many developers and data engineers use Managed Workflows for Apache Airflow (Amazon MWAA), a managed service for open source Apache Airflow, to programmatically author, schedule, and monitor workflows. With Amazon MWAA, you can focus on business logic and build workflows without worrying about the management responsibilities such as setup, patching, upgrades, security, scaling, and so […]

Read More
Tierney - stock.adobe.com Cloud computing with abstract high speed technology POV motion blur

Migrating from self-managed Apache Airflow to Amazon Managed Workflows for Apache Airflow (MWAA)

This post was written by Tomas Christ, Solution Architect at eprimo GmbH. eprimo GmbH is a wholly owned subsidiary of E.ON SE, situated near Frankfurt, Germany. It represents the largest purely green-energy supplier in Germany with some 1.7 million customers. Currently, eprimo has a staff of approximately 160 people. We have been using Amazon Web […]

Read More
phonlamaiphoto - stock.adobe.com cloud computing technology

Deploying to Amazon Managed Workflows for Apache Airflow with CI/CD tools

Apache Airflow‘s active open source community, familiar Python development as directed acyclic graph (DAG) workflows, and extensive library of pre-built integrations have helped it become a leading tool for data scientists and engineers for creating data pipelines. Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is a fully managed service that makes running open source […]

Read More
Chepko Danil – stock.adobe.com

Move your Apache Airflow connections and variables to AWS Secrets Manager

Data scientists and engineers have made Apache Airflow a leading open source tool to create data pipelines due to its active open source community, familiar Python development as directed acyclic graph (DAG) workflows, and extensive library of prebuilt integrations. However, managing the connections and variables that these pipelines depend on can be a challenge, especially […]

Read More
peshkov – stock.adobe.com AdobeStock_303323028_news_1000x500

Amazon Managed Workflows for Apache Airflow unaffected by Airflow 1.10.12 vulnerability

Amazon Managed Workflows for Apache Airflow (MWAA) is not affected by the recently announced vulnerability in Apache Airflow 1.10.12. The default airflow.cfg file uses a temporary key that is the same for all installations. In Airflow 1.10.12 and earlier, there was no restriction in using that temporary key on the Airflow web server, meaning that […]

Read More