AWS Public Sector Blog

AWS Hybrid Connectivity: Sharing AWS Direct Connect with AWS GovCloud (US) and commercial Regions

To establish network connectivity between on-premises data centers, branch locations, and cloud resources, organizations use a hybrid network. This technical walkthrough explains how to implement hybrid connectivity from your premises to AWS GovCloud (US) and commercial AWS Regions using a dedicated private network connection provided by AWS Direct Connect (DX). Customers looking to implement hybrid connectivity from their premises to commercial AWS Regions exclusively should refer to the AWS Hybrid Connectivity whitepaper.

Customers that host sensitive data, regulated workloads, and need to meet the most stringent US government security and compliance requirements choose AWS GovCloud (US). Some AWS GovCloud (US) customers also host non-sensitive data and run secure workloads in commercial AWS Regions. This multi-region architecture allows them to optimize costs by selecting the Region that best meets their compliance needs, while benefiting from Amazon Web Services (AWS) offerings not yet available in AWS GovCloud (US).

Customers that implement this multi-region architecture often require hybrid connectivity from their premises to each AWS Region. Those that require increased network throughput and a more consistent network experience than internet-based connections leverage AWS Direct Connect (DX). Additionally, customers that do not require a dedicated network connection for each AWS Region can optimize costs by sharing a dedicated network connection to AWS using AWS Direct Connect Gateway (DXGW).

Architecture

Regulated customers are optimizing cloud operating costs by strategically placing workloads within Regions based on their compliance and business needs. Designing for cost optimization is consistent with architectural best-practices and the AWS Well-Architected Framework.

The AWS Well-Architected Framework provides a consistent set of best practices to evaluate architectures based on five pillars: operational excellence, security, reliability, performance efficiency, and cost optimization. The Cost Optimization Pillar includes the ability to run systems that deliver business value at the lowest price point. This multi-Region and shared DX architecture allows you to adopt a consumption model where you pay only for the computing resources that you require. You can increase or decrease usage depending on business and/or compliance requirements.

Figure 1 illustrates a common connectivity model used for hybrid connectivity. Connectivity models refer to the communication pattern between on-premises networks and cloud resources in AWS. This particular model originates from the DXGW with AWS Transit Gateway, Multi-Regions, and AWS Public Peering connectivity model discussed in the AWS Hybrid Connectivity whitepaper—with one key difference: there’s no AWS Transit Gateway Peering since AWS does not allow communication between AWS GovCloud (US) and commercial Regions by design. All other model attributes and considerations discussed (scale, service limits, data transfer costs) in the whitepaper apply.

This connectivity model contains the following:

  • Multiple AWS Regions inclusive of AWS GovCloud (US)
  • Dual Direct Connect connections to independent DX locations
  • Single on-premises data center with dual connections to AWS
  • Transit Virtual Interface (VIF) for connectivity to AWS Transit Gateway (TGW) via AWS DX
  • Public Virtual Interface (VIF) for connectivity to AWS public services via AWS DX
  • AWS Direct Connect Gateway (DXGW) with AWS Transit Gateway
  • Scale to hundreds of AWS Virtual Private Clouds (VPCs) per Region

 

Figure 1 – AWS DX hybrid connectivity to AWS GovCloud (US) and AWS commercial Region.

Figure 1 – AWS DX hybrid connectivity to AWS GovCloud (US) and AWS commercial Region.

AWS GovCloud (US) considerations

AWS GovCloud (US) Regions are physically isolated and have logical network isolation from all other AWS Regions, which is why Transit Gateway Peering between AWS GovCloud (US) Regions and commercial Regions is not allowed. Transit Gateway Peering is allowed between commercial Regions, as well as between AWS GovCloud (US) Regions.

When you create accounts in the AWS GovCloud (US) Region from AWS Organizations, an associated account in the AWS commercial Region is automatically created for billing and support purposes. The account in the commercial Region and the account in the AWS GovCloud (US) Region are linked. The linked account in the commercial Region becomes an important concept when implementing connectivity models. It provides the mechanism that you use to share AWS services between AWS GovCloud (US) and commercial Regions; for example, AWS DXGW and AWS DX Transit VIF.

Figure 2 illustrates the relationship between the AWS GovCloud (US) account and the linked account in the commercial Region where you provision the AWS network services needed to implement the connectivity model represented in Figure 1.

Figure 2 – AWS GovCloud (US) account linked to commercial Region account.

Figure 2 – AWS GovCloud (US) account linked to commercial Region account.

Planning

AWS Accounts

Using multiple AWS accounts to isolate and manage your business applications and data can help you align across most of the AWS Well-Architected Framework. The Organizing Your AWS Environment Using Multiple Accounts whitepaper covers this topic in detail; however, in this blog we focus on the use of a network account. The network account typically exists within the Infrastructure Organizational Unit (OU), centralizing management of many networking resources.

IP Addressing

IP addresses enable resources in your Amazon Virtual Private Cloud (Amazon VPC) to communicate with each other, and with resources over the Internet. A VPC is a virtual network within an AWS Region that closely resembles a traditional on-premises network, with the benefits of cloud scalability on AWS.

Each AWS Region can have multiple VPCs and each VPC can have multiple subnets. You should plan to allocate sufficient IP address space that doesn’t overlap across AWS Regions or with on-premises networks. For example, if your on-premises network uses the 10.0.0.0/16 CIDR block, you can use the 10.1.0.0/16 and 10.2.0.0/16 CIDR blocks for two AWS Regions—as illustrated in Figure 3.

Figure 3 – Example of non-overlapping IP address allocation for hybrid connectivity.

Figure 3 – Example of non-overlapping IP address allocation for hybrid connectivity.

Implementation

Assumptions

  • AWS GovCloud (US) network account and linked AWS commercial Region network account are provisioned.
  • AWS VPCs and subnets have been defined within all AWS Regions.
  • Both AWS DXs are provisioned at different DX locations and terminated to your premises. If not, you can use the AWS Direct Connect Resiliency Toolkit to get started.

Note: AWS Direct Connect can be provisioned in any account since it is only a billing mechanism; however, the Transit VIF needs to be deployed in the AWS commercial Region network account that is linked to the AWS GovCloud (US) network account since that is where the AWS Direct Connect Gateway also needs to be provisioned.

Instructions

The following step-by-step instructions guide you through the implementation; however, since detailed instructions already exists for implementing AWS services, I refer to those instructions when applicable.

Complete the following steps in the AWS commercial Region network account that is linked to the AWS GovCloud (US) network account:

Figure 4 – Network services in linked commercial network account.

Figure 4 – Network services in linked commercial network account.

1. Create the Transit Gateway and make sure to select a unique Amazon side ASN.

2. Once the Transit Gateway is available, attach your VPCs to your Transit Gateway by creating Transit Gateway Attachments.

3. Once the Transit Gateway is attached to your VPCs, add routes between the Transit Gateway and your VPCs.

4. Test the Transit Gateway by sending data (ICMP) between two attached VPCs.

5. Create a Direct Connect Gateway, select a unique Amazon side ASN. The ASN for the DXGW and TGW needs to be different. Associate the TGW, and specify the allowed prefixes.

Note: As of publication, there is a limit of 20 prefixes per AWS Transit Gateway from AWS to on-premise on a Transit Virtual Interface. This limit cannot be increased.

6. To connect your DX connections to the Transit Gateway, create a Transit Virtual Interface for each DX connection. The Transit VIF needs to be created from the account that owns the DX connection.

a. If the linked commercial network account owns the DX connection, you can select My AWS account and specify the Direct Connect Gateway to which the new virtual interface is attached.

b. If the linked commercial network account is not the DX connection owner, you need to specify it as the VIF owner by selecting Another AWS account. The linked commercial network account then needs to accept the newly created VIF.

Expand Additional settings to configure your own BGP peering network and BGP authentication key.

7. Create a public virtual interface for each DX connection. A public virtual interface can access all AWS public services using public IP addresses. When you create a public virtual interface, it can take up to 72 hours for us to review and approve your request.

Complete the following steps in the AWS GovCloud (US) Region network account:

Figure 5 – Network services in AWS GovCloud (US) network account.

Figure 5 – Network services in AWS GovCloud (US) network account.

1. Create the Transit Gateway, select a unique Amazon side ASN, and attach your VPCs. The ASN for the TGW and DXGW needs to be different.

2. Once the Transit Gateway is available, attach your VPCs to your Transit Gateway by creating Transit Gateway Attachments.

3. Once the Transit Gateway is attached to your VPCs, add routes between the Transit Gateway and your VPCs.

4. Test the Transit Gateway by sending data (ICMP) between two attached VPCs.

5. Click on the Transit Gateway that was created and associate the existing Direct Connect Gateway. The Direct Connect Gateway is visible via My account because it was provisioned in the commercial Region network account that is linked to the AWS GovCloud (US) network account.

Complete the following steps on your premises:

1. Download the router configuration files for your on-premises hardware from each virtual interface. These configurations provide the 802.1q encapsulation and BGP peering details for each virtual interface.

2. Verify IP connectivity, BGP neighbor state, and BGP learned routes

3. Redistribute BGP learned routes into your Interior Gateway Protocol (IGP)

4. Verify bidirectional connectivity between your premises and AWS Regions to complete this hybrid connectivity design.

Conclusion and next steps

Security Reminder: The hybrid connectivity model discussed in this blog provides a single shared path to both AWS GovCloud (US) and AWS commercial Region(s). Customers need to be aware of their compliance requirements that require data to reside solely in AWS GovCloud (US) when using this connectivity model.

This blog covered a common hybrid connectivity model; however, there are other connectivity models and considerations that may impact your final network design. For example, what if you are utilizing more than three Regions inclusive of AWS GovCloud (US)? As of publication, three Transit Gateways per AWS Direct Connect Gateway is a limit that cannot be increased. In this case, consider using a dedicated DX connection for AWS GovCloud (US).

Understanding how AWS network services are shared between AWS GovCloud (US) and commercial Regions will help you design and govern a connectivity model that best suits your compliance and business needs. We have included references below that provide deeper insight into the various topics discussed throughout this blog. Also, if you prefer to talk through AWS network design or anything AWS related, please contact your Account Manager and/or Solutions Architect.

AWS References

Subscribe to the AWS Public Sector Blog newsletter to get the latest in AWS tools, solutions, and innovations from the public sector delivered to your inbox, or contact us.