Introducing AWS Site-to-Site VPN Private IP VPNs

Update 10/13/22: Added walkthrough with the AWS Management console and link to code in CDK and Terraform.

One of the most common ways that customers connect securely to AWS from on premises is by using the AWS Site-to-Site VPN managed IPSec VPN solution. One key benefit our customers look for when using the service is not having to manage 3rd-party or custom VPN solutions built using EC2 instances. The native service is already built to be highly-available with two tunnels in two different Availability Zones and has native integration with AWS Transit Gateway (via a VPN attachment) which allows customers to scale the connectivity to multiple VPCs with a single Transit Gateway-based VPN connection.

The typical way to create native IPSec VPN connections is to use the Internet as a transport, however some customers create the IPSec VPNs using AWS Direct Connect. Why? Remember that Direct Connect is a private, dedicated connection from on-premises environments to AWS (unlike the Internet), but it is not encrypted. For some customers, security and compliance regulations require encryption at layer 2 or 3 for any communication using this dedicated line. Since April 2021 Direct Connect supports IEEE 802.1AE MAC Security Standard (MACsec) encryption for 10Gbps and 100Gbps Dedicated connections. This makes Site-to-Site VPNs the best way to achieve encryption for 1Gbps and sub-1Gbps Direct Connect connections. Previously, the only way to create Site-to-Site VPN connections over Direct Connect was by using Public VIFs and connecting to the service’s public endpoint. However, for many of our customers, the use of public IPs introduces several challenges:

• The increase of the attack surface area.
• The need to filter routes advertised from/to AWS (with BGP communities).
• They are not able to use public IPs.

That’s why we are announcing Private IP VPN, a new feature that provides customers the ability to deploy AWS Site-to-Site VPN connections over Direct Connect using private IP addresses (RFC1918). With this feature, customers can encrypt traffic between their on-premises networks and AWS via Direct Connect connections without the need for public IP addresses, thus enabling enhanced security and network privacy at the same time. Private IP VPN is deployed on top of Transit VIFs, so it allows you to use AWS Transit Gateway for centralized management of customers’ Virtual Private Clouds (VPCs) and connections to the on-premises networks in a more secured, private and scalable manner.

Use Cases

Following are some of the most typical use-cases for this feature:

• Security and compliance requirements. Some customers (for example Financial services, Healthcare, and Federal) have requirements for robust encryption of network traffic, this includes between on premises and AWS over Direct Connect connections. Furthermore, they have a requirement to only use private IP addresses for this communication. Private IP VPN ensures that traffic between AWS and on-premises networks now can be encrypted over Direct Connect using only private IP addresses, allowing customers to comply with their regulatory and security mandates.
• Traffic Segmentation over AWS Direct Connect. Customers that want to connect their on-premises network to a Transit Gateway over Direct Connect can do that using a Transit VIF. However, you might have reached the quota of Transit virtual interfaces per AWS Direct Connect dedicated or hosted connections. For customers that want to extend their VRFs to AWS now they can create multiple AWS Site-to-Site VPN connections with multiple Transit Gateway attachments – on top of the Transit VIF. That way, each segment is encrypted using only private IPs. The end to end segmentation is achieved mapping each VRF to a different Site-to-Site VPN attachment, segmenting the traffic accordingly with multiple route tables in the Transit Gateway.
• Higher Route Scale over AWS Direct Connect using only private IPs. One common limitation that customers have by using Direct Connect is the limit of route announcements: currently this limit is 20 outbound and 100 inbound routes. With Private IP VPN, customers can now advertise 5,000 routes outbound and 1,000 inbound while they achieve traffic encryption at the same time.

How does this feature work?

A Private IP VPN connection requires a Direct Connect gateway and a Transit VIF as the underlying transport. The Private IP VPN attachment is then associated with the appropriate Direct Connect gateway attachment on the Transit Gateway – as can be seen in Figure 1. Each Private IP VPN connection comprises of two IPSec tunnels that encrypt all network traffic over those tunnels. You can assign private IP addresses to both ends of the IPSec tunnels (Transit Gateway and on-premises router).

Figure 1 shows the use case where you want to encrypt all the traffic over the Direct Connect link. In this example, only the Site-to-Site VPN attachment is associated and propagated to the Transit Gateway Route Table. You need to ensure that the traffic from the on-premises router is forwarded to AWS via the VPN tunnels. With this configuration, both VPCs will be able to communicate between each other, and with the on-premises network via the Site-to-Site VPN connection.

Figure 1. AWS Site-to-Site Private IP VPN configuration

As mentioned before, another possible scenario is to use Private IP VPNs for traffic segmentation. In this particular use case, you can extend different on-premises VRFs to the Transit Gateway by creating one Site-to-Site Private IP VPN connection per VRF. In the Transit Gateway, one Route Table per environment is used to segment, and associate each VPN connection to a different Route Table. In Figure 2 you can see the Production and Non-Production Route Tables. Production VPC and Production VPN attachments are associated and propagated to the Production Route Table (same with the Non-Production attachments). The on-premises router simply needs to be configured to forward each VRF to the corresponding Site-to-Site VPN connection.

Figure 2. Traffic Segmentation – extending on-premises VRFs using AWS Site-to-Site Private IP VPN connections

For now we have seen only use cases where all the traffic between AWS and the on-premises environment is exchanged via the VPN tunnels. However, you can use both the VPN tunnels to exchange encrypted traffic, and the underlying Direct Connect connection for unencrypted traffic. Figure 3 shows you an example: the Transit Gateway Encrypted Route Table enables access from the on-premises network to the VPC A using the Site-to-Site Private IP VPN connection (encrypted traffic). On the other hand, the Transit Gateway Non-Encrypted Route Table is associated to the Direct Connect gateway attachment, so the traffic from the on-premises router to VPC B is forwarded to the Transit Gateway using directly the Transit VIF BGP peer (not using the overlay created with the VPN connection).

Figure 3. Encrypted and unencrypted traffic over a Direct Connect connection

Walkthrough

You can create an AWS Site-to-Site Private IP VPN using the AWS Management Console or Command Line Interface (CLI). In this blog post, we’ll show both options. We’ll start with our Direct Connect connection and Transit VIF already created. The values used when creating the different resources will follow the ones shown in Figure 1. For the CLI steps, we’ll be using the AWS CLI v2 and jq to create the relevant AWS components.

You can also use AWS CloudFormation, AWS CDK, and Terraform to provision Private IP VPN resources. Check this GitHub AWS Samples link with CDK (Typescript) and Terraform deployment examples.

Step 1: Create the Direct Connect Gateway

First, we create the AWS Direct Connect Gateway:

Figure 4. Creation of the Direct Connect gateway using the AWS Management Console

aws directconnect create-direct-connect-gateway --direct-connect-gateway-name "DxGatewayPrivateIP"

To get the ID of the newly created Direct Connect gateway, we will filter the results using the Direct Connect gateway name:

DXGWID=$(aws directconnect describe-direct-connect-gateways | jq -r '.directConnectGateways[] | select(.directConnectGatewayName == "DxGatewayPrivateIP") | .directConnectGatewayId')

Step 2: Create the Transit Gateway

Next, we create the AWS Transit Gateway. To support the creation of the Private IP VPN, we also need to configure the Transit Gateway VPC CIDR block – in this example we are using 10.24.10.0/24 as CIDR block, and 64516 as Amazon Side BGP ASN:

Figure 5. Creation of the Transit Gateway using the AWS Management Console

aws ec2 --region us-west-1 create-transit-gateway --options TransitGatewayCidrBlocks="10.24.10.0/24",AmazonSideAsn=64516

To get the ID of the newly created Transit Gateway, we will filter the results using the configured Transit Gateway CIDR Block of 10.24.10.0/24 and the Amazon side BGP ASN of 64516:

TGWID=$(aws ec2 describe-transit-gateways --region us-west-1 | jq -r '.TransitGateways[] | select(.Options.TransitGatewayCidrBlocks | index("10.24.10.0/24")) | select(.Options.AmazonSideAsn == 64516).TransitGatewayId')

Step 3: Transit Gateway association to the Direct Connect gateway

We need now to associate the Transit Gateway with the Direct Connect gateway. As you see in the image and CLI command below, we are adding the Transit Gateway CIDR block in the “Allowed prefixes” section. We are announcing this CIDR block to on premises, so when we are creating the VPN tunnels, the on-premises router can know that it needs to route the request to the private outside IP addresses via the Transit VIF.

Figure 6. Transit Gateway association to the Direct Connect gateway using the AWS Management Console

aws directconnect create-direct-connect-gateway-association --direct-connect-gateway-id $DXGWID --gateway-id $TGWID --add-allowed-prefixes-to-direct-connect-gateway cidr="10.24.10.0/24" --region us-west-1

In this example, $DXGWID is the ID of our Direct Connect gateway created in Step 1, while $TGWID is the ID of the Transit Gateway created in Step 2. These variables can be seen by performing:

echo "AWS Transit Gateway: $TGWID\nAWS Direct Connect gateway: $DXGWID"

Step 4: Create the Customer gateway (CGW)

Once we have created the Transit Gateway and we have associated it to the Direct Connect gateway, we can proceed to create the Site-to-Site Private IP VPN connection using the Direct Connect gateway and Transit VIF as underlying transport. But first, we will need to create the Customer gateway (CGW).

Figure 7. Creation of the Customer gateway using the AWS Management Console

aws ec2 create-customer-gateway --ip-address 192.168.0.1 --bgp-asn 65051 --type ipsec.1 --region us-west-1

The IP address and BGP ASN to configure in the Customer gateway should be from your on-premises environment (and different from this example). Make sure that the IP address does not overlap with the Transit Gateway CIDR block or other VPC CIDRs in AWS.

To get the ID of the new Customer gateway, we filter on the configured IP address of 192.168.0.1 and the ASN of 65051:

CGWID=$(aws ec2 describe-customer-gateways --region us-west-1 | jq -r '.CustomerGateways[] | select((.IpAddress == "192.168.0.1") and .BgpAsn == "65051") | .CustomerGatewayId')

Step 5: Create the AWS Site-to-Site Private IP VPN connection

And finally, the Private IP VPN connection. As variables, we need to pass the Customer gateway ID created in Step 4, the Transit Gateway ID created in Step 2, and the Transit Gateway attachment of the underlying transport – which is the Direct Connect gateway association we created in Step 3.

Figure 8. Creation of the Site-to-Site Private IP VPN connection using the AWS Management Console

For the CLI command, to get the Direct Connect attachment ID, we will filter for attachments that are in both the associated state and available state:

ATTACHID=$(aws ec2 describe-transit-gateway-attachments --region us-west-1 | jq -r '.TransitGatewayAttachments[] | select((.ResourceType == "direct-connect-gateway") and (.Association.State == "associated") and (.State == "available")) | .TransitGatewayAttachmentId')

We can then use the following CLI command to create the VPN connection:

aws ec2 create-vpn-connection --customer-gateway-id $CGWID --transit-gateway-id $TGWID --options "{\"OutsideIpAddressType\":\"PrivateIpv4\", \"TransportTransitGatewayAttachmentId\":\""$ATTACHID"\"}" --type ipsec.1 --region us-west-1

With this, we have our Private IP VPN connection and VPN Transit Gateway attachment created. The tunnel configuration from on premises, and the propagation and association of the VPN attachment will work as with a normal AWS Site-to-Site VPN connection.

To get the details of the new IPSec VPN connection, we will filter using the specific Direct Connect attachment ID:

aws ec2 describe-vpn-connections --region us-west-1 | jq -r --arg ATTACHID "$ATTACHID" '.VpnConnections[] | select(.Options.TransportTransitGatewayAttachmentId == $ATTACHID)'

Cleanup

If you followed the steps above to test the creation of a Private IP VPN, remember that you need to clean the following resources to avoid undesired costs. In this section we are only showing the CLI commands needed for the clean-up. If you followed the walkthrough using the AWS Management Console, follow the same order when deleting the resources.

• The Site-to-Site VPN connection (and attachment).

VPNID=$(aws ec2 describe-vpn-connections --region us-west-1 | jq -r --arg ATTACHID "$ATTACHID" '.VpnConnections[] | select(.Options.TransportTransitGatewayAttachmentId == $ATTACHID).VpnConnectionId')

aws ec2 delete-vpn-connection --vpn-connection-id $VPNID --region us-west-1

• The Customer gateway (CGW).

aws ec2 delete-customer-gateway --customer-gateway-id $CGWID --region us-west-1

• To remove the Transit Gateway, first you need to disassociate the Transit Gateway to the Direct Connect gateway.

DXGWASSOCIATEID=$(aws directconnect describe-direct-connect-gateway-associations --direct-connect-gateway-id $DXGWID | jq -r --arg TGWID "$TGWID" '.directConnectGatewayAssociations[] | select(.associatedGateway.id == $TGWID) | .associationId')

aws directconnect delete-direct-connect-gateway-association --association-id $DXGWASSOCIATEID --region us-west-1

• As this disassociation can take a few minutes, you can check the state using:

aws directconnect describe-direct-connect-gateway-associations --direct-connect-gateway-id $DXGWID

• Once the disassociation is done, you can proceed to delete the Transit Gateway.

aws ec2  delete-transit-gateway --transit-gateway-id $TGWID --region us-west-1

• Finally, you will need to remove the Direct Connect gateway.

aws directconnect delete-direct-connect-gateway --direct-connect-gateway-id $DXGWID

Things to know

• The throughput of the Site-to-Site Private IP VPN connection is the same as a regular Site-to-Site VPN connection: 1.25 Gbps. You can use ECMP (Equal Cost Multi-path) across multiple VPN connections to increase this bandwidth – remember that you need to use dynamic routing in the VPN connection for ECMP to work. Also, take into account that you can only use ECMP between Private IP VPN connections over the same Direct Connect gateway attachment.
• Site-to-Site Private IP VPN connections support static routing, as well as dynamic routing using BGP. We recommend the use of BGP-capable devices as your customer gateway, as the BGP protocol offers robust liveness detection checks that can assist failover to the second VPN tunnel if the first one goes down.
• The only underlying transport available for the Private IP VPN are Direct Connect connections. As such, you need to specify a Direct Connect attachment ID while configuring the VPN connection. Multiple VPNs can use the same Direct Connect attachment as transport.
• You need to make sure the Transit Gateway CIDR block does not overlap with the VPC CIDRs attached to the Transit Gateway. The IP addresses of the VPN tunnel in the AWS-side are assigned from this Transit Gateway CIDR block. Therefore, if this block overlaps, you may not get the desirable routing behavior.
• The VPN Outside Tunnel IPs only support IPv4 addressing and transport. The VPN Inside IP addresses (for the BGP session) support IPv6.
• Regarding the monitoring and visibility of the VPN connection, you can view the metrics and events for this attachment just like any other VPN attachment. Same about AWS Site-to-Site VPN Flow Logs.
• Both Transit Gateway and Site-to-Site Private IP VPN connection must be owned by the same AWS account.
• Regarding the pricing:
  • The Site-to-Site VPN connection has an hourly VPN connection charge and a per-GB Data Transfer Out charge for data egressing over the Direct Connect connection.
  • The Private IP VPN Transit Gateway attachment carries an hourly charge which is the same as a regular VPN attachment.
  • There are no AWS Transit Gateway data processing charges, as those are assumed by the underlaying Direct Connect transport attachment.

Conclusions

With this new feature, you can now create AWS Site-to-Site VPN connections on top of a Direct Connect connection using private IPs. Previously, customers had to use Public VIFs to achieve this traffic encryption, and therefore were forced to use public IP addresses for VPN endpoints. The usage of public IPs increases the probability of external attacks compelling customers to deploy additional security equipment for network protection. The Private IP VPN feature provides end-to-end private connectivity in addition to traffic encryption, improving the overall security posture.

Support for Private IP VPNs is available in all AWS regions where AWS Site-to-site VPN is available. For additional information, visit the AWS Site-to-site VPN product page, documentation, and pricing page.

A correction was made on February 6, 2024: An earlier version of this post included AWS Direct Connect quotas information which were revised in the meantime. The post has been updated to refer readers to the quotas within the AWS Direct Connect User Guide instead.

About the authors