AWS Public Sector Blog

Hybrid connectivity to AWS GovCloud (US) and commercial Regions using AWS Direct Connect

AWS branded background with text overlay that says "Hybrid connectivity to AWS GovCloud (US) and commercial Regions using AWS Direct Connect"

To establish network connectivity between on-premises data centers, branch locations, and cloud resources, organizations use a hybrid network. This blog post explains how to implement hybrid connectivity from your premises to AWS GovCloud (US) and commercial AWS Regions using a dedicated private network connection provided by AWS Direct Connect (DX). Customers looking to implement hybrid connectivity from their premises to commercial AWS Regions exclusively should refer to the AWS Hybrid Connectivity whitepaper.

Customers that host sensitive data, regulated workloads, and need to meet the most stringent US government security and compliance requirements choose AWS GovCloud (US). Some AWS GovCloud (US) customers also host non-sensitive data and run secure workloads in commercial AWS Regions. This multi-region and multi-partition architecture allows them to optimize costs by selecting the Region and partition that best meets their business and compliance needs, while benefiting from Amazon Web Services (AWS) offerings not yet available in AWS GovCloud (US). What exactly is an AWS partition? An AWS partition logically and physically separates groups of AWS Regions. This provides data, network, and machine isolation from AWS Regions in other AWS partitions. AWS GovCloud (US) Regions reside within the AWS GovCloud (US) partition whereas commercial AWS regions reside within the AWS standard partition. For more information regarding AWS partitions please refer to the AWS GovCloud (US) or standard? Selecting the right AWS partition blog post.

Customers that implement this multi-region and multi-partition architecture often require hybrid connectivity from their premises to each AWS Region and partition. Those that require increased network throughput and a more consistent network experience than internet-based connections leverage AWS Direct Connect (DX). Additionally, customers that do not require a dedicated network connection for each AWS Region and AWS partition can optimize costs by aggregating DX access using AWS Direct Connect gateway (DXGW)—a globally available resource that allows connectivity to any Region (excluding AWS China Regions).

Architecture

Regulated customers are optimizing cloud operating costs by strategically placing workloads within AWS Regions and partitions based on their business and compliance needs. Designing for cost optimization is consistent with architectural best-practices and the AWS Well-Architected Framework.

The AWS Well-Architected Framework provides a consistent set of best practices to evaluate architectures based on six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability. The Cost Optimization Pillar includes the ability to run systems that deliver business value at the lowest price point. This multi-region and multi-partition architecture allows you to adopt a consumption model where you pay only for the computing resources that you require. You can increase or decrease usage depending on business and/or compliance requirements.

Figure 1 illustrates a common connectivity model used for hybrid connectivity. Connectivity models refer to the communication pattern between on-premises networks and cloud resources in AWS. This particular model originates from the DXGW with AWS Transit Gateway, Multi-Regions, and AWS Public Peering connectivity model discussed in the AWS Hybrid Connectivity whitepaper—with one key difference: there’s no AWS Transit Gateway Peering between AWS GovCloud (US) and commercial Regions since they are in separate partitions.

This connectivity model contains the following:

  • Multiple AWS Regions inclusive of AWS GovCloud (US)
  • Dual AWS Direct Connect connections to independent DX locations
  • Single on-premises data center with dual connections to AWS
  • Transit Virtual Interface (VIF) for connectivity to AWS Transit Gateway (TGW) via AWS DX
  • Public Virtual Interface (VIF) for connectivity to AWS public services via AWS DX
  • AWS Direct Connect gateway (DXGW) with AWS Transit Gateway
  • High scale of Amazon Virtual Private Clouds (VPCs) per AWS Region

All other model attributes and considerations discussed (scale, service limits, data transfer costs) in the whitepaper apply.

Figure 1. AWS DX hybrid connectivity to AWS GovCloud (US) Region and AWS commercial Region.

AWS GovCloud (US) considerations

AWS GovCloud (US) Regions are physically isolated and have logical network isolation from all AWS Regions in other partitions. This is why Transit Gateway Peering between AWS GovCloud (US) Regions and commercial Regions is not supported, as they exist in separate partitions. Transit Gateway Peering supports inter-Region and intra-Region peering only within the same partition as shown in Figure 2.

Figure 2. Transit Gateway Peering relationships between AWS Regions and partitions

Creating accounts from within AWS Organizations operates differently in the AWS GovCloud partition compared to the standard AWS partition. Creating GovCloud accounts directly from within the AWS GovCloud partition is not supported and requires action from the management account of the organization in the standard partition.

When accounts are provisioned into the AWS GovCloud (US) partition from the management account of the organization, an associated account in the AWS standard partition is automatically created for billing, service, and support purposes. The account in the standard partition and the account in the AWS GovCloud (US) partition are linked. For more information on AWS Standard Account Linking, please refer to the AWS GovCloud (US) User Guide. The linked account in the standard partition becomes an important concept when implementing multi-partition connectivity models. It provides the mechanism that you use to share AWS services between AWS GovCloud (US) and standard partitions; for example, AWS DXGW and AWS DX Transit VIF.

Note: Customers already leveraging AWS Organizations with an existing network account within a commercial region (i.e., standard partition) will need to create the AWS GovCloud (US) account using the existing network account’s root credentials using the same steps as if creating an AWS GovCloud (US) from a standalone AWS account. This will allow customers to create a new AWS GovCloud (US) account that is linked to the existing network account within a commercial region.

Figure 3 illustrates the relationship between the AWS GovCloud (US) account and the linked account in the commercial Region where you provision the AWS network services needed to implement the connectivity model represented in Figure 1.

Figure 3. AWS GovCloud (US) account linked to commercial Region account.

Planning

AWS Accounts

Using multiple AWS accounts to isolate and manage your business applications and data can help you align across most of the AWS Well-Architected Framework. The Organizing Your AWS Environment Using Multiple Accounts whitepaper covers this topic in detail; however, in this blog we focus on the use of a network account. The network account typically exists within the Infrastructure Organizational Unit (OU), centralizing management of many networking resources.

IP Addressing

IP addresses enable resources in your Amazon Virtual Private Cloud (Amazon VPC) to communicate with each other, and with resources over the Internet. A VPC is a virtual network within an AWS Region that closely resembles a traditional on-premises network, with the benefits of cloud scalability on AWS.

Each AWS Region can have multiple VPCs and each VPC can have multiple subnets. You should plan to allocate sufficient IP address space that doesn’t overlap across AWS Regions or with on-premises networks. For example, if your on-premises network uses the 10.0.0.0/16 CIDR block, you can use the 10.1.0.0/16 and 10.2.0.0/16 CIDR blocks for two AWS Regions—as illustrated in Figure 4.

Figure 4. Example of non-overlapping IP address allocation for hybrid connectivity.

Implementation

Assumptions

  • AWS GovCloud (US) network account and linked AWS commercial Region network account are provisioned.
  • AWS VPCs and subnets have been defined within all AWS Regions.
  • Both AWS DXs are provisioned at different DX locations and terminated to your premises. If not, you can use the AWS Direct Connect Resiliency Toolkit to get started.

Note: AWS Direct Connect can be provisioned in any account since it is only a billing mechanism; however, the Transit VIF needs to be owned by the AWS commercial Region network account that is linked to the AWS GovCloud (US) network account since that is where the AWS Direct Connect gateway also needs to be provisioned.

Instructions

The following step-by-step instructions guide you through the implementation; however, since detailed instructions already exists for implementing AWS services, I refer to those instructions when applicable.

Complete the following steps in the AWS commercial Region network account that is linked to the AWS GovCloud (US) network account:

Figure 5. Network services in linked commercial network account.

  1. Create the Transit Gateway and make sure to select a unique Amazon side ASN.
  2. Once the Transit Gateway is available, attach your VPCs to your Transit Gateway by creating Transit Gateway Attachments.
  3. Once the Transit Gateway is attached to your VPCs, add routes between the Transit Gateway and your VPCs.
  4. Create a Direct Connect gateway, select a unique Amazon side ASN. The ASN for the DXGW and TGW needs to be different. Associate the TGW, and specify the allowed prefixes.
  5. To leverage your DX connections with Transit Gateway, create a Transit Virtual Interface for each DX connection. Each Transit VIF needs to be created from the account that owns the DX connections.
    • If the linked commercial network account owns the DX connection, you can select My AWS account and specify the Direct Connect Gateway to which the new virtual interface is    attached.
    • If the linked commercial network account is not the DX connection owner, you need to specify it as the VIF owner by selecting Another AWS account. The linked commercial network account then needs to accept the newly created VIF.Expand Additional settings to configure your own BGP peering network and BGP authentication key.
  6. Create a Public Virtual Interface for each DX connection. A Public Virtual Interface can access all AWS public services using public IP addresses. When you create a public virtual interface, it can take up to 72 hours for us to review and approve your request.

Complete the following steps in the AWS GovCloud (US) Region network account:

Figure 6. Network services in AWS GovCloud (US) network account.

  1. Create the Transit Gateway, select a unique Amazon side ASN, and attach your VPCs. The ASN for the TGW and DXGW needs to be different.
  2. Once the Transit Gateway is available, attach your VPCs to your Transit Gateway by creating Transit Gateway Attachments.
  3. Once the Transit Gateway is attached to your VPCs, add routes between the Transit Gateway and your VPCs.
  4. Navigate to the Direct Connect service page and associate Transit Gateway to the existing Direct Connect gateway. The Direct Connect gateway is visible via My account because it was provisioned in the commercial Region network account that is linked to the AWS GovCloud (US) network account.

Complete the following steps on your premises:

  1. Download the router configuration files for your on-premises hardware from each virtual interface. These configurations provide the 802.1q encapsulation and BGP peering details for each virtual interface.
  2. Verify IP connectivity, BGP neighbor state, and BGP learned routes
  3. Redistribute BGP learned routes into your Interior Gateway Protocol (IGP)
  4. Verify bidirectional connectivity between your premises and AWS Regions to complete this hybrid connectivity design.

Conclusion and next steps

Security reminder: The hybrid connectivity model discussed in this blog provides a single shared path to both AWS GovCloud (US) and AWS commercial Region(s). Customers need to be aware of their compliance requirements that require data to reside solely in AWS GovCloud (US) when using this connectivity model.

This blog covered a common hybrid connectivity model; however, there are other connectivity models and considerations that may impact your final network design. For example, what if you are utilizing several Regions inclusive of AWS GovCloud (US)? When this blog was first published, three Transit Gateways per AWS Direct Connect gateway was the service limit, which would have compelled customers to leverage a dedicated DX connection for AWS GovCloud (US).

Update (April 2023): AWS Direct Connect now supports 4 transit virtual interfaces per dedicated connection and AWS Direct Connect gateway now supports six Transit Gateways. These service limit increases offer customers more flexibility when designing connectivity models and is why customers are encouraged to check service quotas and limits when building on AWS. Customers can now use an additional transit virtual interface to provide logical network segmentation for GovCloud (US) and commercial Region communication across the same AWS Direct Connect.

Understanding how AWS network services are shared between AWS GovCloud (US) and standard partitions will help you design and govern a connectivity model that best suits your compliance and business needs. We have included references below that provide deeper insight into the various topics discussed throughout this blog. Also, if you prefer to talk through AWS network design or anything AWS related, please contact your Account Manager and/or Solutions Architect.

AWS References

Subscribe to the AWS Public Sector Blog newsletter to get the latest in AWS tools, solutions, and innovations from the public sector delivered to your inbox, or contact us.