This blog post is written by Jared Thompson, Specialist Solutions Architect, Hybrid Edge.

Today, we announced AWS Outposts rack support for intra-VPC communication across multiple Outposts. You can now add routes in your Outposts rack subnet route table to forward traffic between subnets within the same VPC spanning across multiple Outposts using the Outpost local gateways (LGW). The LGW enables connectivity between your Outpost subnets and your on-premises network. With this enhancement, you can establish intra-VPC instance-to-instance IP communication across Outposts through your on-premise network, via direct VPC routing.

You can take advantage of this enhancement to architect for high availability for your on-premises applications and, at the same time, improve application performance by reducing the latency between application components that are in the same VPC but running on different Outposts.

This post shows you how you can use intra-VPC communication across multiple Outposts to build a Multi-AZ like architecture for your on-premises applications and services by leveraging direct VPC routing.

To clarify a few concepts before we go into the details: Outposts rack is the 42U form factor of the AWS Outposts Family of services. An Outpost is a pool of AWS compute and storage capacity deployed at a customer’s site. An Outpost may comprise of one or more racks connected together at the site.

Overview

Prior to today’s announcement, applications and services running on multiple Outposts were not able to communicate with each other if they were in the same VPC and if the Outpost was configured to use direct VPC routing. To overcome this limitation it was necessary to separate workloads into multiple VPCs and align each VPC with a separate Outpost, or to configure the Outpost local gateway route table to use Customer-owned IP (CoIP) mode. This limitation was because the traffic between two subnets that are in the same VPC but in disparate Outposts was not able to communicate each other through the service link, as it was blocked in the Region. (See the following diagram in Figure 1)

To show how this worked previously, as an example, let’s assume we have a VPC CIDR range of 10.77.0.0/16, and we want to route 10.77.11.0/24 using the local gateway:

When we attempted to apply this change, we would get the following error message:

The destination CIDR block 10.77.11.0/24 is equal or more specific than one of this VPC’s CIDR blocks. This route can target only an interface or an instance.

Because we were not able to specify a more specific route, we were not able to route between these subnets.

Figure 1 – Prior to this feature, you could not send traffic to the local gateway, as you could not set a route that was more specific than the VPC’s CIDR Range

Using intra-VPC communication across multiple Outposts with direct VPC routing you can now define routes that are more specific than the local VPC CIDR range and has local gateway as target. This enables you to direct traffic from one subnet to another within the same VPC, using the Outpost’s local gateways (LGW). (See Figure 2)

Figure 2 – Two Outpost racks in the same VPC can be configured to communicate over the Outpost local gateways

With this feature, you can design highly available architectures on the edge with multiple Outpost racks, eliminating the need to use multiple VPCs.

Let’s see it in action!

For this example, we will assume that we have a VPC CIDR of 10.77.0.0/16, Outpost A has a subnet CIDR of 10.77.7.0/24, and Outpost B has a subnet CIDR of 10.77.11.0/24.  By default, resources on these racks will not be able to communicate with each other since the default local route of each route table within the VPC is set to 10.77.0.0/16. If the traffic is on another Outpost, the traffic would be blocked because service link traffic cannot hairpin through the region. We are going to route this traffic across our on-premises infrastructure. (See Figure 3)

Figure 3 –This is what our example environment looks like. Note, we have one VPC with two Outpost subnets

For the purposes of this example, we are going to assume that the Customer WAN (See Figure 3) is already set up to route traffic between Outpost A and Outpost B subnets.  For more information, see Local gateway BGP connectivity in the AWS Outpost documentation.  Additionally, we will want to ensure that our local gateway routing tables are in direct VPC routing mode.

Let’s suppose that we want Instance A (10.77.7.88/24) to reach Instance B (10.77.11.119). We will try this with a ping:We can see that none of our pings worked. Since both of these subnets are on two different Outposts, we will need to configure our subnets to route traffic to each other by using intra-VPC communication across multiple Outposts with direct VPC routing.

To enable traffic between these two private subnets, we will configure the routing table to direct traffic towards the neighboring Outpost Subnet to use the Outpost local gateway, allowing traffic to flow between your on-premises network infrastructure. We do this by specifying a more specific route than the default VPC CIDR range.

1.To accomplish this, we will need to associate our VPC with the Outpost’s local gateway route table on each Outpost. From the console, navigate to AWS Outposts / Local gateway route tables. Find the local gateway route table that is associated with each Outpost, go to the VPC associations tab, and select Associate VPC.

Now that these VPC are associated to the local gateway routing table, we will be able to configure the route tables for these subnets to target the Outpost local gateway.

2. For our 10.77.7.0/24 subnet on Outpost Rack A, we will add a route to our other subnet, 10.77.11.0/24 in the subnet’s routing table. One of the target options is Outpost Local Gateway:

Selecting this option will bring up two options, for each of our local gateways. Be sure to select the correct local gateway ID for Outpost A’s local gateway, which is lgw-008e7656cf09c9c21 for my Outpost Rack A.

3. Do the same for our 10.77.11.0/24 subnet, this time setting a destination of 10.77.7.0/24 via the local gateway ID of Outpost Rack B:

Now that we have our routes updated, let’s try our ping again.

Success! We are now able to reach the other instance over the local gateways. This is because our route tables in the Outposts subnets are forwarding traffic over the local gateway, utilizing our on-premises network infrastructure for the communication backbone.

Availability

Intra-VPC communication across multiple Outposts with direct VPC routing is available in all AWS Regions where Outposts rack is available. Your existing Outposts racks may require an update to enable support for Intra-VPC communication across multiple Outposts. If this feature does not work for you, please contact AWS Support.

Conclusion

Utilizing intra-VPC communication across Outposts with direct VPC routing allows you to route traffic between subnets within the same VPC. This feature will allow traffic to route across different Outposts by utilizing Outposts local gateway and your on-premises network, without needing to divide your infrastructure into multiple VPCs. You can take advantage of this enhancement for your on-premises applications, while improving application performance by reducing latency between application components running on multiple Outposts.