How do I access the Apache Airflow UI using the private network access mode in my Amazon MWAA environment?

Short description

The Amazon MWAA environment provides public and private access modes for Apache Airflow UI. With the public access mode, the Apache Airflow Web server is open to the internet and can be accessed without any additional setup. However, you can't control the network access of the Apache Airflow Web server using a security group. Therefore, this option is considered less secure compared to the private access mode. With the private access mode, the Apache Airflow Web server is exposed only within the Amazon Virtual Private cloud (Amazon VPC) in the environment. Amazon MWAA creates a VPC endpoint in your VPC for each environment. Therefore, access to the VPC is necessary to access the Apache Airflow UI. The private network access mode requires additional networking setup.

Resolution

You can access the Apache Airflow Web server in private network access mode using one of the following approaches.

AWS Direct Connect or VPN

If you have an AWS Direct Connect or VPN connection to connect your on-premises network to Amazon VPC, consult with your networking team to configure access to the Amazon MWAA Web server address. After the network path is established, review the security groups associated with the environment to be sure that the Amazon MWAA Web server allows HTTPS (TCP 443) traffic from the source IP or subnet. You can also use an AWS Client VPN to configure the private network access.

If you are still facing issues, try the following:

• Verify the DNS resolution with a tool, such as nslookup or dig (dig <airflow-web-server-address>).
• Verify port level connectivity using a tool, such as telnet (telnet <airflow-web-server-address-443>).

If the issue still persists, check whether the browser has a web proxy configured. In that case, try disabling the proxy for testing or fix the issue in the proxy configuration.

Linux bastion host

A bastion host can be used as an intermediary between the internet and private subnet. The host is usually a Linux machine with SSH server running. You can establish an SSH tunnel to the bastion host to access resources within your Amazon VPC. Then, you can use a browser proxy, such as FoxyProxy, to open the Apache Airflow UI through the SSH Tunnel. To set up a Linux bastion host, see Tutorial: Configuring private network access using a Linux bastion host.

If you have an Amazon Elastic Compute Cloud (Amazon EC2) instance with a graphical user interface (GUI), such as a Windows machine, you can use this instance to access the Apache Airflow UI.

Load balancer

You can also use a load balancer as an intermediary between the internet and private subnet. Unlike with the bastion host, you can directly access the load balancer's address without any additional setup.

To configure the load balancer, do the following:

1. Identify the private IP addresses of the MWAA Web server. To get these IP addresses, do a DNS lookup on the Apache Airflow UI address (dig +short <airflow-web-server-address>).
2. Create a target group for the private IP addresses of the Amazon MWAA Web server.
3. Configure the health check settings for the target group to include 200 and 302 for Matcher.
   Note: Without this setting, the targets might be flagged as unhealthy when the Apache Airflow Web server responds with a 302 Redirect.
4. Create an Application Load Balancer with an HTTPS listener and the target group created.
   Note: You must have an SSL certificate before creating an HTTPS listener. You can create an SSL certificate with AWS Certificate Manager (ACM) by signing in to the domain or subdomain of your choice.
5. Test access to the Apache Airflow UI using the address of the Application Load Balancer that you created.

Note: Amazon MWAA requires a web login token to access the Apache Airflow UI. Therefore, you need to create the web login token and pass this token as a query string when accessing the Apache Airflow UI through the Application Load Balancer. To automate the creation of this token, see Accessing a private Amazon MWAA environment using federated identities.

---

Related information

Apache Airflow access modes

Accessing the Apache Airflow UI