Announcing Amazon Virtual Private Gateway Ingress Routing support for Gateway Load Balancer

Today, on 30th August 2023, AWS launched a new enhancement to the Amazon Virtual Private Cloud (Amazon VPC) Ingress Routing feature. With this enhancement, customers can now specify a Gateway Load Balancer Endpoint (GWLBE) as the next-hop in the virtual private gateway (VGW) route table. This allows customers to inspect their traffic coming into AWS via AWS Site-to-Site VPN or AWS Direct Connect using AWS Network Firewall (which is powered by Gateway Load Balancer (GWLB)) or third-party firewalls running in AWS. This enhancement simplifies deployment and helps ensure consistent traffic inspection at entry points into customers’ Amazon VPCs.

In this post, we explain how the newly launched feature works, how to configure it, describe common deployment scenarios, and discuss considerations while using this feature.

What’s new

There are now five ways customers can connect their environments to a VPC via VGW, as shown in Figure 1.

• Figure 1a shows an on-premises corporate data center connecting to VPC using Direct Connect (DX) Private Virtual Interface (VIF) to VGW.
• Figure 1b shows an on-premises corporate data center connecting to VPCs using Direct Connect (DX) Private Virtual Interface (VIF) via Direct Connect Gateway (DXGW) to VGW.
• Figure 1c shows an on-premises corporate data center connecting to VPC using IPSec Site-to-site VPN over Direct Connect (DX) Public Virtual Interface (VIF) to VGW.
• Figure 1d shows an on-premises corporate data center connecting to VPC using IPSec Site-to-site VPN over the Internet.
• Figure 1e shows an IPSec Site-to-site VPN connectivity between two AWS environments.

Figure 1: VGW connectivity options

Prior to this feature, it was not possible to send the incoming traffic arriving at VGW directly to a GWLB endpoint or GWLB for inspection. As a workaround, customers used an AWS Transit Gateway and an inspection VPC. This incurred additional costs, management overhead, and latency. Now, with the VGW route table supporting the ingress routing feature for GWLBE, customers can set GWLBE as a next-hop in the VGW route table. This allows customers to steer traffic directly to GWLBE and GWLB, thereby reducing costs, management overhead, and latency. This new feature works across all scenarios in Figure 1.

Feature deep dive

Let’s describe how this feature works and how to configure it in a step-by-step manner. We will do this for the three most common deployment scenarios:

• Scenario 1: Using VGW Ingress routing with GWLB and third-party virtual appliances.
• Scenario 2: Using VGW Ingress routing with AWS Network Firewall.
• Scenario 3: Using Ingress routing with an Internet Gateway (IGW) and VGW.

Scenario 1: Using VGW Ingress routing with GWLB and third-party virtual appliances

In this scenario (Figure 2), on-premises connectivity to the Amazon VPC is via VPN or Direct Connect (DX). The intent is to inspect all the traffic using third-party virtual appliances in the Security VPC before it reaches the protected application subnet (10.2.2.0/24).

Figure 2: VGW Ingress routing with GWLB

How does the feature work?

The following steps describe a packet walkthrough when the traffic originates from the remote site either via a VPN or DX connection toward the application hosted in the protected VPC:

(1) The remote network device conducts a route lookup for the destination in the VPC and selects the VPN or DX.

(2) When the packet arrives at the VGW a route lookup occurs in the VPC Ingress route table that is associated with the VGW, a more specific route for the application subnet is pointing toward the Gateway Load Balancer Endpoint (10.2.2.0/24 -> vpce-id).

(3) Traffic, routed through the GWLBE, is delivered securely and privately to the GWLB using AWS PrivateLink.

(4) GWLB sends the traffic to the third-party virtual appliances, such as next generation firewalls (NGFW), or intrusion detection and prevention systems (IDS/IPS). The third-party appliances inspect the traffic, and then send it back to the GWLBE.

(5) GWLBE sends the traffic to the protected application subnet (10.2.2.0/24).

Reverse traffic (not shown) follows the same path in the reverse direction.

How to configure this feature

Before configuring this feature, make sure you complete the following prerequisite actions:

• Set-up your VPN/DX, Security VPCs, and Protected VPC.
• Configure your Security VPC (i.e. the one containing GWLB and targets) with:
  • A GWLB with a target group that has third-party virtual appliances.
  • An endpoint service created for the GWLB:
    • Create VPC Endpoint Service and associate it with the GWLB. Note the endpoint service name which starts with “com.amazonaws.vpce.”.
    • If the Protected VPC is in a different AWS account, then it should be listed in the allowed principal list for the endpoint service (see manage permissions).
• Configure your Protected VPC (i.e. the one containing GWLBEs) with:
  • A VPC Endpoint (GWLBE) pointing to the same service name as identified on the endpoint service above. Note that when creating the endpoint, only the Availability Zones (AZs) that are in common are available for selection.

Now that the prerequisites are ready, let’s take a look at how to configure this feature using either AWS Command Line Interface (AWS CLI) or the AWS Management Console. You can use either method, as per your preference.

Configure the feature using AWS CLI

Forward Traffic: First, let’s set up routing tables for forward traffic (traffic going from corporate data center to the protected subnet).

• Create a new route table to be the ingress route table, and associate it with the VGW:

aws ec2 create-route-table \
     --vpc-id vpc-038d18783f44f9148
{
    "RouteTable": {
        "Associations": [],
        "PropagatingVgws": [],
        "RouteTableId": "rtb-0cc22db998e104134",
        "Routes": [
            {
                "DestinationCidrBlock": "10.2.0.0/16",
                "GatewayId": "local",
                "Origin": "CreateRouteTable",
                "State": "active"
            }
        ],
        "Tags": [],
        "VpcId": "vpc-038d18783f44f9148",
        "OwnerId": "795662187403"
    }
}


$ aws ec2 associate-route-table \
--route-table-id rtb-0cc22db998e104134 \
--gateway-id vgw-0763bd41338524306
{
    "AssociationId": "rtbassoc-092b8c045de3f5812",
    "AssociationState": {
        "State": "associated"
    }
}

• For each application subnet you want to protect, create a route in the VGW Ingress route table, with the subnet CIDR as the route destination, and the GWLBE as the route target. This steers all incoming traffic destined to that subnet to the VPCe (GWLBE).

$ aws ec2 create-route \
--route-table-id rtb-0cc22db998e104134 \
--destination-cidr-block 10.2.2.0/24 \
--vpc-endpoint-id vpce-0c3ef88210b4cea51
{
    "Return": true
}

Reverse Traffic: Now, let’s set-up routing tables for reverse traffic, the traffic going from the Protected Subnet to your corporate data center.

• First, get the route table associated with your application’s private subnet.

$ aws ec2 describe-route-tables \
        --filters Name=association.subnet-id,Values=subnet-0e23061abca5265eb
{
    "RouteTables": [
        {
            "Associations": [
                {
                    "Main": false,
                    "RouteTableAssociationId": "rtbassoc-0fe664c250d902312",
                    "RouteTableId": "rtb-0767064621a86a46e",
                    "SubnetId": "subnet-0e23061abca5265eb",
                    "AssociationState": {
                        "State": "associated"
                    }
                }
            ],

# output truncated

• For each application’s private subnet, create the route for on-premises CIDR and point it toward the VPCe (GWLBE) in the corresponding route table.

$ aws ec2 create-route \
--route-table-id rtb-0767064621a86a46e \
--destination-cidr-block 172.16.0.0/16 \
--vpc-endpoint-id vpce-0c3ef88210b4cea51

{
    "Return": true
}

• In the route table associated with the GWLBE subnet, point the route for remote CIDR toward the VGW:

$ aws ec2 create-route \
--route-table-id rtb-095fd6a71689a0b4f \
--destination-cidr-block 172.16.0.0/16 \
--gateway-id vgw-0763bd41338524306
{
    "Return": true
}

Configure the feature using the Console

Forward Traffic: First, let’s set-up routing tables for forward traffic, the traffic going from your corporate data center to the Protected Subnet.

Create a new route table, as shown in Figure 3a.

Figure 3a: Create route table

Associate the newly created route table with the VGW, as shown in Figure 3b. This is the ingress route table.

Figure 3b: Associate route table with VGW

Select the gateway to be associated with the route table, as shown in Figure 3c.

Figure 3c: Select gateway to associate with the route table

Verify that the gateway has been associated with the route table, as shown in Figure 3d.

Figure 3d: Verify gateway association

In the VGW ingress route table, point the route for the application subnet toward the GWLBE (note that in the route table the next-hop is shown as vpce-id), as shown in Figure 3e.

Figure 3e: Configure route for the protected subnet

Add the next-hop for the protected subnet as GWLBE in the route table, as shown in Figure 3f.

Figure 3f: Target in the VGW Ingress route table

Verify that the route has been added successfully, as shown in Figure 3g.

Figure 3g: Route entry in the VGW Ingress route table

Reverse Traffic: Now, let’s set-up route tables for reverse traffic, the traffic going from the Protected Subnet to Corporate Data Center.

Make sure the application subnet has the reverse route pointing toward the GWLB Endpoint as shown in Figure 3h.

Figure 3h: Application subnet route table

The subnet hosting the VPCe (GWLBE) should have a route for the remote site pointing toward the VGW, as shown in Figure 3i.

Figure 3i: Firewall subnet route table

Scenario 2: Using VGW Ingress routing with AWS Network Firewall

In this scenario (Figure 4), on-premises connectivity to the Amazon VPC is via VPN. The intent is to inspect all traffic using AWS Network Firewall before it reaches the protected application subnet (10.2.2.0/24). In this scenario, we are inserting Network Firewall (which is powered by GWLB) in the traffic flow. When a new network firewall is created, a GWLB Endpoint gets created in that subnet, in the routing table you will see vpce-id as the target. For details on how to configure Network Firewall, refer to this documentation. Note that customers do not need to manage the underlying GWLB used by our AWS Network Firewall service.

Figure 4: VGW Ingress Routing with Network Firewall architecture

How does the feature work?

The following steps describe a packet walkthrough when the traffic originates from the remote site via a VPN toward an application hosted in the protected VPC:

(1) The remote network device conducts a route lookup for the destination in VPC and selects the VPN.

(2) When the packet arrives at the VGW a route lookup occurs in the VPC Ingress route table that is associated with the VGW, a more specific route for the application subnet is pointing toward the Gateway Load Balancer Endpoint (10.2.2.0/24 -> vpce-id).

(3) Traffic, routed through a firewall endpoint, is delivered securely and privately to Network Firewall using AWS PrivateLink.

(4) Network Firewall performs the traffic inspection, and it sends the traffic back to the firewall endpoint.

(5) Firewall Endpoint sends the traffic to the protected application subnet (10.2.2.0/24).

Reverse traffic (not shown) follows the same path in the reverse direction.

Scenario 3: Using Ingress routing with IGW and VGW

There may be a scenario where you want to use ingress routing capability to protect an application subnet for both traffic coming via VGW and IGW. The VPC Ingress Routing feature, which prioritizes the longest prefix match, makes this possible. Figure 5 shows an example scenario where on-premises Corporate Data Center is connected to the AWS VPC via VPN or DX, and the VPC is also connected to the Internet using IGW.

Figure 5: Ingress routing IGW, VGW, and GWLB

How does the feature work with VGW?

The following steps describe a packet walkthrough when the traffic originates from the remote site either via a VPN or DX toward the application hosted in the protected VPC:

(1) The remote network device conducts a route lookup for the destination in a VPC and selects the VPN or DX.

(2) When the packet arrives at the VGW a route lookup occurs in the VPC Ingress route table that is associated with the VGW, a more specific route for the application subnet is pointing toward the Gateway Load Balancer Endpoint (10.2.2.0/24 -> vpce-id).

(3) Traffic, routed through GWLBE, is delivered securely and privately to GWLB using AWS PrivateLink.

(4) The third-party appliances inspect the traffic, and then send it back to the GWLBE.

(5) GWLBE sends the traffic to the protected application subnet (10.2.2.0/24).

How does the feature work with IGW?

The following steps describe a packet walkthrough when the traffic originates from a client on the Internet and is sent to an application hosted in a VPC via an IGW. Note that this capability (IGW Ingress Routing support for GWLB) has been available since 2020.

A. The remote client conducts a route lookup for the destination in a VPC and reaches IGW.

B. When the packet arrives at the IGW a route lookup occurs in the VPC Ingress route table that is associated with the IGW, a more specific route for the application subnet is pointing toward the Gateway Load Balancer Endpoint (10.2.2.0/24 -> vpce-id).

C. Traffic, routed through GWLBE, is delivered securely and privately to GWLB using AWS PrivateLink.

D. The third-party appliances inspect the traffic, and then send it back to the GWLBE.

E. GWLBE sends the traffic to the protected application subnet (10.2.2.0/24).

Considerations

Consider the following requirements when preparing to deploy VGW Ingress Routing using GWLBE as the next-hop:

• The GWLBE used as the next-hop target must be deployed in the same VPC that is associated with the VGW.
• The destination subnets must belong to a VPC associated with the VGW.
• You cannot enable VGW route propagation for the same route table you use as the VGW Ingress route table.
• Using VGW Ingress routing functionality you can also add IPv6 dual-stack subnet as well as IPv6 only subnets as targets
• Site-to-Site VPN connections on a VGW do not support IPv6. However, Site-to-Site VPN supports IPv6 traffic for VPN connections attached to Transit Gateway.
• Connectivity to VPC via VGW is 1:1, which means one VGW connects to only one VPC. This feature applies only in these scenarios. If you have multiple VPCs and are looking for centralized inspection models using hub and spoke topology, then see this post.

Conclusion:

In this post, we saw how to steer traffic coming from on-premises using VPN or DX via a VGW to a GWLBE or a Network Firewall Endpoint using the new VGW Ingress routing enhancement. This enhancement allows customers to insert third-party appliances or AWS Network Firewall seamlessly before the traffic reaches the application workloads.

There are no additional costs to use Amazon VPC ingress routing. It is available in all AWS Regions (including AWS GovCloud) and you can start using it today.

You can learn more about gateway routing tables in the updated VPC documentation. For more details on GWLB, refer to GWLB documentation and GWLB posts.