Creating a single internet exit point from multiple VPCs Using AWS Transit Gateway

In this post, we show you how to centralize outbound internet traffic from many VPCs without compromising VPC isolation. Using AWS Transit Gateway, you can configure a single VPC with multiple NAT gateways to consolidate outbound traffic for numerous VPCs. At the same time, you can use multiple route tables within the transit gateway to maintain VPC-to-VPC isolation. This hub-and-spoke design enables you to manage all your outbound internet communication securely from one place.

Without AWS Transit Gateway, you have to combine an internet gateway with NAT gateways or NAT instances for each VPC needing outbound internet access. However, if you have more significant numbers of VPCs, the management of multiple internet gateways and NAT gateways and instances adds labor and costs. In that case, you can save overhead by centralizing outbound traffic with AWS Transit Gateway.

While this design allows for a centralized architecture with a pair of shared NAT gateways, it is possible to modify this architecture to replace the NAT gateways with other security appliances to carry out such tasks as traffic capture, policy enforcement, web filtering etc as long as those appliances are capable of performing NAT function and have required route configurations.

Solution overview

This architecture shows you how to use AWS Transit Gateway to centralize outbound internet traffic from multiple VPCs using hub-and-spoke design. This design includes two NAT gateways, as illustrated in the following diagram.

Figure 1: Architecture diagram showing AWS Transit Gateway to centralize outbound internet traffic from multiple VPCs

Prerequisites

To complete this walkthrough, you need:

An AWS account
An IAM user with access to AWS resources, including AWS Transit Gateway

Deploying the example

In this section, we demonstrate how to deploy AWS Transit Gateway and three VPCs with non-overlapping IP space in a single Region and attach the transit gateway to these VPCs. Within the hub VPC, we show you how to deploy NAT gateways (one in each Availability Zone) and an internet gateway. Finally, we demonstrate how to configure the route tables in all the VPCs and use a test instance in the spoke VPC to verify outbound internet connectivity.

If you prefer to automate the entire test setup, you can follow the instructions here to deploy all the components. This automates the next three stages of the deployment, enabling you to proceed directly to “Testing the Deployment” section. Make sure to choose the desired Region for this stack before you deploy the components.

Creating and configuring the VPCs

First, we show you how to deploy the essential components of this design, including the VPCs, subnets, an internet gateway, NAT gateway and route tables. The following instructions deploy this example in the N. Virginia (us-east-1) Region. However, you can deploy this procedure in any Region by adjusting the Region and AZ parameters.

Create the following three VPCs: Egress-VPC, App1-VPC, and App2-VPC. Provide values for each, as shown in the following table. For more information, see Getting Started with Amazon VPC.

VPC name tag	IPv4 CIDR	IPv6 CIDR	Tenancy
Egress-VPC	192.168.0.0/16	No IPv6 CIDR Block	Default
App1-VPC	10.0.0.0/16	No IPv6 CIDR Block	Default
App2-VPC	10.1.0.0/16	No IPv6 CIDR Block	Default

Create the subnets in each of the VPCs as described in the following table. For more information, see Creating a Subnet. In the next steps, you configure the route tables to make some of these subnets public.

Subnet name tag	VPC	AZ	IPv4 CIDR
Egress-Public-AZ1	Egress-VPC	us-east-1a	192.168.1.0/24
Egress-Public-AZ2	Egress-VPC	us-east-1b	192.168.2.0/24
Egress-Private-AZ1	Egress-VPC	us-east-1a	192.168.3.0/24
Egress-Private-AZ2	Egress-VPC	us-east-1b	192.168.4.0/24
App1-Private-AZ1	App1-VPC	us-east-1a	10.0.1.0/24
App1-Private-AZ2	App1-VPC	us-east-1b	10.0.2.0/24
App2-Private-AZ1	App2-VPC	us-east-1a	10.1.1.0/24
App2-Private-AZ2	App2-VPC	us-east-1b	10.1.2.0/24

Create and attach an internet gateway to the VPC Egress-VPC. Use IGW as the Name tag for this internet gateway. For more information, see Creating and Attaching an Internet Gateway.
Create a NAT gateway in the VPC Egress-VPC. For more information, see NAT gateways. Create only one NAT gateway for this example. In your production environment, you should create a NAT gateway for every Availability Zone in which you have a subnet and use the NAT gateway in the same Availability zone.
- For Subnet, enter Egress-Public-AZ1.
- For Elastic IP Allocation ID, choose Create new EIP.
Create two new route tables in Egress-VPC. For Name tags, use Egress-Public-RT and Egress-Private-RT.
Add a new default route in the route table Egress-Public-RT, with the destination set to 0.0.0.0/0. Associate the route with the internet gateway IGW. For more information, see Adding and Removing Routes from a route table. Then edit the subnet association and add both the Egress-Public-AZ1 and Egress-Public-AZ2 subnets to this route table.
Add a new default route in the route table Egress-Private-RT, with the destination 0.0.0.0/0. Associate the route with the NAT gateway. Then edit the subnet association, adding both the Egress-Private-AZ1 and Egress-Private-AZ2 subnets to this route table.

Deploying and configuring the transit gateway

Next, we show you how to deploy a new transit gateway and attach it to all three VPCs, routing traffic to the internet through the NAT gateway. You accomplish this by connecting the transit gateway to all the three VPCs:

In the VPC console, choose AWS Transit Gateway and create a new transit gateway. Use the name TGW-Internet, add a suitable description, and make sure to uncheck Default route table propagation and Default route table association.

Choose Transit Gateway Attachments and create the attachments described in the following table.

AWS Transit Gateway ID	Attachment type	Attachment name tag	Subnet IDs
TGW-Internet	VPC	Egress-Attachment	Egress-Private-AZ1 Egress-Private-AZ2
TGW-Internet	VPC	App1-Attachment	App1-Private-AZ1 App1-Private-AZ2
TGW-Internet	VPC	App2-Attachment	App2-Private-AZ1 App2-Private-AZ2

Choose AWS Transit Gateway Route tables and create two route tables. Name the route tables Egress-RouteTable and App-RouteTable and associate both route tables with the TGW-Internet transit gateway.
Under AWS Transit Gateway route tables, choose App-RouteTable, Associations, Create association. Associate both App1-Attachment and App2-Attachment to this route table.
On the same route table, choose Routes, Create route, enter the 0.0.0.0/0 route, and choose the attachment: Egress-VPC.
Add these additional routes: 192.168.0.0/16, 172.16.0.0/12 and 10.0.0.0/8 as Blackhole to make sure VPCs can’t communicate with each other through the NAT gateway.
Under AWS Transit Gateway route tables, choose Egress-RouteTable, Associations, Create association. Associate Egress-Attachment to this route table.
On the same route table, choose Routes, choose Create route, and enter 10.0.0.0/16 with the attachment App1-Attachment. Then enter a second route for 10.1.0.0/16 with the attachment App2-Attachment.
In the left navigation pane, choose Route Tables and edit the default route table associated with App1-VPC and App2-VPC, adding a 0.0.0.0/0 route and set TGW-Internet as the target.
Edit the Egress-Public-RT route table associated with the Egress-VPC and add 10.0.0.0/16 and 10.1.0.0/16. Set TGW-Internet as the target.

Launch test instances

To test this setup, launch three EC2 instances. Launch the first in a public subnet in the Egress-VPC as the bastion host. Launch the remaining two instances in VPCs App1-VPC and App2-VPC. For more information, see Launch an Instance.

Launch an EC2 Instance in the Egress-VPC with the following configuration:

AMI: Amazon Linux 2 AMI (HVM)
Instance Type: t2.large
Network: Egress-VPC
Subnet: Egress-Public-AZ1
Auto Assign Public IP: Enabled
Tags: Add a tag with Key: Name and Value: Bastion
Security Group: Create a new security group to allow SSH traffic from your Public IP address (you can find your current IP address by browsing to www.myipaddress.com)

Launch two EC2 instances, one in App1-VPC and the other in App2-VPC with the following configuration:

AMI: Amazon Linux 2 AMI (HVM)
Instance Type: t2.large
Network: App1-VPC
Subnet: App1-Private-AZ1
Auto Assign Public IP: Disabled
Tags: Add a tag with Key: Name and Value: App1VM
Inbound Security Group: Create a new security group to allow SSH & All ICMP – IPV4 traffic from 10.0.0.0/16, 10.1.0.0/16 & 192.168.0.0/16.

AMI: Amazon Linux 2 AMI (HVM)
Instance Type: t2.large
Network: App2-VPC
Subnet: App2-Private-AZ1
Auto Assign Public IP: Disabled
Tags: Add a tag with Key: Name and Value: App2VM
Inbound Security Group: Create a new security group to allow SSH & All ICMP – IPV4 traffic from 10.0.0.0/16, 10.1.0.0/16 & 192.168.0.0/16.

Testing the deployment

Log in to the bastion host instance using SSH. Then use SSH to connect to App1VM. To verify that AWS Transit Gateway routes traffic correctly to the NAT gateway in the Egress-VPC, use curl to connect to several websites. Then use SSH to connect from App1VM to the App2VM. Although allowed by security group policy, this connectivity should fail. This failure confirms that AWS Transit Gateway isn’t routing between the App1VM and App2VM instances. The following diagram illustrates this test workflow.

Figure 2: Diagram showing the test setup to test Transit Gateway for egress internet traffic

In the EC2 console, identify the public IP address or public DNS name of the bastion host you launched in the previous step.

Use SSH to connect to the bastion host:

ssh -i (path to key) ec2-user@(Public IP of the Bastion Host)

Copy the SSH key for this Region to the bastion host to log in to App1VM:

vi sshkey.pem
Press i for putting vi in insert mode.
Copy the private key you downloaded earlier and paste it into vi.
Press Esc
Type :wq to save and exit.

Use SSH to connect to the App1VM instance from the bastion host:
```
ssh -i ./sshkey.pem ec2-user@(Private IP of App1VM Instance)
```

Test the connectivity by using curl for the following URLs:

curl http://www.amazon.com
curl http://calculator.s3.amazonaws.com/index.html

Use SSH/ping to signal the IP address of App2VM from App1VM and verify that connectivity fails:
```
Ping (Private IP of the App2VM)
ssh (Private IP of the App2VM)
```
Alternatively, you can verify that connectivity fails with the following SSH command:
```
ssh ec2-user@(Private IP of the App2VM)
```

Cost implications

In most cases, a centralized NAT gateway provides a better return on your investment than multiple gateways. Even considering the NAT gateway expense and the data transfer cost per GB, the process should reduce your per-hour charges for multiple NAT gateways and simplify solution management.

If you already use NAT gateways, the amount of data processed by the NAT gateway remains constant, whether it is channeled through a single NAT gateway or multiple NAT gateways.

The following items affect the cost of this solution:

Transit gateway attachment (per hour)
Transit gateway data processing (per GB)
NAT gateway (per hour)
NAT gateway data processing (per GB)

Refer to our AWS Transit Gateway pricing page and NAT gateway pricing page for more information.

Design considerations

No design can take account of all use cases. However, cost and performance issues might make this solution ill-suited to some use cases.

Cost

There is an additional $0.02/GB data processing charge compared to running the NAT gateways in the local VPC. Depending on the number of VPCs you run and volume of data transferred, you might prefer to deploy a local NAT gateway.

Performance

When deploying this solution, also consider throughput. As of this post, a single NAT gateway supports 10 Gbps of bandwidth and automatically scales up to 45 Gbps. If you deploy two NAT gateways (one in each Availability Zone) in a production environment, your actual bandwidth should be much higher than a single NAT gateway. If your cumulative bandwidth requirements exceed the capacity of two NAT gateways, consider connecting your high-throughput VPCs to local NAT gateways.

A NAT gateway can support up to 55,000 simultaneous connections (or approximately 900 connections per second/55,000 connections per minute) to each unique destination. If you need more capacity than this, consider creating multiple NAT gateways to distribute the load.

Before concluding, we should also draw your attention to two other critical design elements:

Blackhole routes

While setting up your transit gateway, we configured all of the RFC1918 IP space (192.168.0.0/16, 172.16.0.0/12 and 10.0.0.0/8) as blackhole destinations in the App route table. Because we are advertising 0.0.0.0/0 route for each of the App VPCs, traffic destined to another VPC routes to NAT gateway, which reroutes the traffic to the destined VPC. As a result, communication among VPCs succeeds even though AWS Transit Gateway does not directly route this traffic. Adding the blackhole routes prevents undesired routing and makes sure VPCs remain isolated from each other.

Security inspections

The design presented in this post enables you to centralize outbound traffic through one VPC hub supported by two shared NAT gateways. You can also use this design to replace the NAT gateways with other security appliances that help manage outbound traffic—for example, carrying out deep security inspection, traffic capture, policy enforcement, and web filtering.

However, you are ultimately responsible for designing high availability and scalability into these appliances. AWS uses a distributed architecture to maintain the fault tolerance and scalability of each NAT gateway. If you replace the NAT gateways with Amazon EC2 instances, your design routes to only one instance per prefix and doesn’t perform health checks on static routes. Make sure that any replacement security appliance can perform the NAT function and has the required route configurations.

Conclusion

In this post, we showed you how to use AWS Transit Gateway to build this centralized exit point to the internet. By consolidating your outbound traffic, you can manage outbound communications security, scaling, and configuration in one place. If you run more substantial numbers of VPCs, we believe that this centralization can save you work, money, and stress. However, while this design reduces the number of NAT gateways and internet gateways you have to manage, bear in mind that it also requires you to deploy and manage transit gateways at scale.

Lastly, we have only shown you traffic output through a NAT gateway to the internet. You can expand on this design and replace the NAT gateways with third-party devices that can perform additional filtering capabilities like URL filtering, malware detection, and outbound protocol inspection.

		Blog: Using AWS Client VPN to securely access AWS and on-premises resources
		Learn about AWS VPN services
		Watch re:Invent 2019: Connectivity to AWS and hybrid AWS network architectures

Networking & Content Delivery