Connecting a Single Customer Router to Multiple VPCs

Amazon Virtual Private Cloud (Amazon VPC) provides customers with tremendous flexibility in how corporate networks can be connected to one or more VPCs. Customers can connect multiple customer gateways (CGW) to a single VPC virtual private gateway (VGW) or can connect a single router to multiple VPC VGWs. This document describes two approaches when connecting a single customer router to multiple VPCs.


Submitted By: Steve Morad
AWS Products Used: Amazon VPC, Amazon EC2
Created On: October 11, 2012


Connecting a Single Customer Router to Multiple VPCs

Topics

Amazon Virtual Private Cloud (Amazon VPC) provides customers with tremendous flexibility in how corporate networks can be connected to one or more VPCs. Customers can connect multiple customer gateways (CGW) to a single VPC virtual private gateway (VGW) or can connect a single router to multiple VPC VGWs. This document describes two approaches when connecting a single customer router to multiple VPCs, as shown in the diagram to the right.

Please reference the VPC Network Administrator Guide for complete VPC networking documentation; however the following diagram and definitions will be helpful for understanding the content of this paper:

Customer Gateway (CGW)

A CGW is the anchor on the customer's side of the VPN connection. It can be a physical or software appliance.

Virtual Private Gateway (VGW)

A VGW is the anchor on the AWS side of the VPN connection.

VPN Connection

A VPN connection is used to describe the network connectivity that is established between a single CGW and a single VGW. You must establish additional VPN connections for each unique CGW and VGW combination (e.g. a single CGW connecting to multiple VPCs, multiple CGWs connecting to a single VPC, or multiple CGWs connecting to multiple VPCs).

VPN Tunnels

The diagram to the right depicts two lines between the customer gateway and virtual private gateway because the VPN connection consists of two tunnels. AWS chose this design to provide increased availability for the Amazon VPC service by automatically failing over from one tunnel to another in the event of an AWS device failure. When you configure your customer gateway, it's important that you configure both tunnels.

  1. IPSec requires that each VPN connection have a unique, publicly routable IP address. Therefore, a single customer gateway connecting to multiple VPCs requires a public IP address for each connection.
  2. VPN connection creation logic is designed to ensure unique tunnel IP addresses for each connection within a single VPC, but not necessarily across multiple VPCs. This is a potential issue only when a customer is connecting a single customer gateway to multiple VPCs and chooses not to follow the Virtual Routing and Forwarding (VRF)approach recommended below. This specific scenario, along with a workaround, is covered in the Alternative Approach section.
  3. AWS will be updating its configuration templates to ensure IPSec keyrings and profiles are explicitly bound to each VPC connection; however, you must ensure IPSec keys are appropriately bound to each VPC connection, or at least one connection will fail (see the IPSec Key Pair and Profile Bindingsection of this document for additional details and examples).

Virtual Routing and Forwarding (VRF) is a technology that allows multiple instances of a routing table to co-exist within the same router at the same time. Because the routing instances are independent, the same or overlapping IP addresses can be used without conflicting with each other.[1] AWS recommends using VRFs when connecting a single customer gateway to multiple Amazon VPCs because the VPN connection creation logic is designed to ensure unique tunnel IP addresses for each connection within a single VPC, but not necessarily across multiple VPCs.

When using VRFs, the customer may follow the standard Amazon VPC documentation for creating VPCs, customer gateways, virtual private gateways, and VPN connections. Each VPN connection provides configuration details that can be included in the customer's network and VRF configuration. Figures 1 and 2 highlight example customer gateway configurations:


Figure 1 - VRF configuration with multiple IPs bound to external interface



Figure 2 - VRF configuration with a second IP address bound to a loop-back adapter

Alternative Approach

When implementing multiple VPC connections from a single customer gateway without VRFs, customers must be aware that VPC does not guarantee unique tunnel and Border Gateway Protocol (BGP) peer IP addresses. As a result, it is possible that these addresses automatically generated for one VPC may be duplicated when creating connections to another VPC. Although AWS does not currently support manual VPC connection address assignments, it is possible to implement a workaround to ensure unique addresses are eventually created for each connection. Figure 3 shows an example customer gateway configuration without VRFs.


Figure 3 - Alternative configuration without using VRFs and using loopback addresses for each required customer gateway public IP address


Customers who do not choose to use VRFs must be aware of this behavior and use the following workaround in the event they receive duplicate tunnel IP addresses when attempting to connect a single customer gateway to multiple VPCs:

  1. Create two VPCs and use the wizard or manual steps to create a virtual private gateway, customer gateway, and VPN connection in each VPC.
  2. Compare the resulting VPN connection configuration files to determine whether the tunnel and BGP neighbor IP addresses are unique (sections 3 & 4 of the configuration files). If the IP addresses are unique, no workaround is required. Otherwise, proceed to step 3 (choose one of the VPCs and perform all steps with this VPC).
  3. Delete the VPC connection with the duplicate tunnel IP addresses and wait for the state to change to deleted.
  4. Create a new "dummy" customer gateway using a "dummy" IP address (e.g. use an AWS Elastic IP that you have been assigned).
  5. Create a new "dummy" connection between the virtual private gateway and this new, "dummy" customer gateway and wait for the connection state to change to available.
  6. Recreate the VPC connection between the virtual private gateway and the original customer gateway.
  7. Wait for the connection state to change to available and verify that the new configuration is, indeed, unique.
  8. Delete the "dummy" connection and "dummy" customer gateway.

Step 1 assumes a customer is connecting a single customer gateway to two VPCs. If a connection to more than two VPCs is required, the customer can recurse through steps 3-7, creating as many "dummy" connections as needed until each VPC connection is associated with unique tunnel and BGP neighbor IP addresses.

When using these new tunnel IP addresses, make sure you also follow the instructions in the IPSec Key Pair and Profile Bindingsection.

This section walks a user through the steps required to work around duplicate VPN connection tunnel and BGP peer IP addresses. To illustrate the work around, the following customer and virtual private gateway IDs will be used in the accompanying screenshots:


Figure 4 - Use of the customer and virtual private gateway IDs

  1. Create two VPCs and use the wizard or manual steps to create a customer gateway, virtual private gateway, and VPN connection in each VPC.

  2. Compare the resulting VPN connection configuration files to determine whether or not the tunnel and BGP neighbor IP addresses are unique (sections 3 & 4 of the configuration files). If the IP addresses are unique, no workaround is required. Otherwise, proceed to step #3 (choose one of the VPCs and perform all steps with this VPC). In this example, the second tunnel (169.254.255.38) and BGP neighbor (169.254.255.37) IPs overlap.

  3. Delete the VPC connection (vpn-f1da3d98) with the duplicate tunnel IP addresses, and wait for the state to change to deleted.

  4. Create a temporary "dummy" Customer Gateway using a "dummy" IP address (e.g. use an AWS EIP that you have been assigned).

  5. Create a new "dummy" connection between the Virtual Private Gateway and this new, "dummy" Customer Gateway (123.123.123.123), and wait for the connection state to change to available.

  6. Recreate the VPC connection between the virtual private gateway and the original customer gateway (184.72.122.165), and wait for the connection state to change to available.

  7. Verify that the new configuration is unique.

  8. Delete the "dummy" connection, wait for the state to change to deleted, and then delete the "dummy" Customer Gateway.

The VPC configuration template currently does not explicitly bind each IPSec keyring and profile to a specific VPC connection. When implementing multiple VPC connections, an explicit mapping is required to ensure the appropriate IPSec keys are used and the VPC tunnels function properly. The following example demonstrates how to create these explicit mappings by binding each keyring and profile to the appropriate VPC client (customer gateway) IP address, where the client IP for VPC #1 is 184.72.122.164 and the client IP for VPC #2 is 184.72.122.165: