Achieving traffic segmentation in multi-AWS Region environments using AWS Transit Gateway and AWS Cloud WAN

In June 2022, we announced AWS Cloud WAN in general availability, a managed wide area networking (WAN) service that makes it easy for you to build, manage, and monitor a global network connecting your resources across your AWS Regions and on-premises environments. With Cloud WAN, you have a central place to create and manage your global routing configuration by creating a policy, and achieving traffic segmentation between resources in several Regions.

Alongside the general availability announcement, we also announced the ability to peer Cloud WAN with AWS Transit Gateway. Although it may seem that Cloud WAN can simply be used instead of Transit Gateway, there are different use cases where it may make sense to have both together and simply peer them:

1. Migrating your resources from using Transit Gateway to Cloud WAN (Federation model). In this use case, the Transit Gateways and Cloud WAN’s Core Network are managed centrally by the same team (in the same AWS Account), and the peering is used to gradually migrate from using Transit Gateway (and Transit Gateway’s inter-region static peerings) to Cloud WAN’s dynamic inter-region peering. This use case is explained in the AWS Cloud WAN and AWS Transit Gateway migration and interoperability patterns post.
2. Distributed Transit Gateways and Cloud WAN acting as a central, global network. Another use case is where, in your organization, different teams want to have full control over their routing capabilities between VPCs (for example to centralize certain services, or traffic inspection), but the inter-Transit Gateway communication is managed by a central team (Transit Gateways are in different AWS accounts). A central AWS Account can manage Cloud WAN to control the traffic flow and network configuration between Transit Gateways (connected to the core network via peering) and possibly also on-premises environments.
3. Traffic Segmentation in multi-Region environments. Currently, there’s no native way to achieve traffic segmentation with Transit Gateway static peerings, as this connection is “shared” by any traffic traversing two Transit Gateways – regardless of the routing domains created in each Transit Gateway. Current solutions to avoid communication between routing domains when traversing a Transit Gateway added via static peering are: 1, creating more specific routes in the route tables – so that only the CIDR block of the other Region’s routing domain is configured in the route table, or 2, adding a central inspection VPC in each Transit Gateway that is peered. Both options add complexity and cost when the final goal is simply to extend the Transit Gateway route tables between two (or more) Transit Gateways. The peering with Cloud WAN allows the extension of Transit Gateway route tables to Cloud WAN segments, allowing for a native way to achieve traffic segmentation when having multi-Region environments.

In this post, we’ll show Traffic Segmentation in multi-Region environments as the example use case to dive deep into peering between Cloud WAN and Transit Gateway. We’ll also review the different steps needed to create the peering (and the resources to create), and we’ll show the end-to-end solution that achieves traffic segmentation in a multi-Region environment.

Cloud WAN and Transit Gateway peering concepts

When peering Cloud WAN with a Transit Gateway, the following resources are needed:

• Cloud WAN peering: it allows you the interconnection of your Core Network Edge (CNE) with a Transit Gateway in the same Region. This peering supports dynamic routing with an automatic exchange of routes using Border Gateway Protocol (BGP), and traffic segmentation is achieved by using policy-based routing over the peering. Note that the routing exchange uses External BGP (eBGP), which means that the Transit Gateway should use a different Autonomous System Number (ASN) with the Core Network’s edges.
• Transit Gateway policy table: to achieve dynamic routing over the Cloud WAN – Transit Gateway peering, you must associate a Transit Gateway policy table to the Transit Gateway’s peering attachment. The table contains policy rules for matching network traffic by policy attributes, and then it maps the traffic that matches the rule to a target route table. When you associate the policy table with the attachment, it populates the table automatically with the policy rules.
• Transit Gateway route table attachment: this attachment extends a Transit Gateway route table to a Cloud WAN segment. It needs a peering as an underlay and, if a Transit Gateway policy table is associated to the peering, then the routes propagated in the Transit Gateway route table and the Cloud WAN segment are shared between each other. You can create several Transit Gateway route table attachments over the same peering. Therefore, you can create one of these attachments per routing domain that you want to segment (extending a specific Transit Gateway route table to a Cloud WAN segment).

Now let’s build the use case that we’ll cover today: end-to-end segmentation in multi-Region environments, between VPCs in different routing domains, using Transit Gateway and Cloud WAN.

Walkthrough

In this example, we have two networks in the North Virginia (us-east-1) and Ireland (eu-west-1) Regions, each of them with two different routing domains (Prod and Non-Prod) configured as Transit Gateway route tables – as shown in the following figure. The end goal will be peering the Transit Gateways to Cloud WAN’s core network to achieve native traffic segmentation between the routing domains.

Figure 1. Network architecture (initial state)

We’re also assuming that the Global Network and Core Network are already created. And we’re using the Core Network policy as shown in the following JSON format. This policy creates the following:

• The Core Network is created in two Regions (us-east-1 and eu-west-1), which are the same as where we have our hub and spoke networks created via Transit Gateways, as shown in the previous figure.
• Two segments are created – Prod and Non-Prod – both of them in the two edge locations indicated earlier.
• Two attachment policies are defined, so that we can automate the association of any new attachment to our segments. Later in this section, we’ll discuss how this works in depth.

{
    "version": "2021.12",
    "core-network-configuration": {
        "vpn-ecmp-support": true,
        "asn-ranges": ["64512-64520"],
        "edge-locations": [
            {"location": "us-east-1"},
            {"location": "eu-west-1"}
        ]
    },
    "segments": [
        {
            "name": "Prod",
            "edge-locations": [
                "us-east-1",
                "eu-west-1"
            ],
            "require-attachment-acceptance": false
        },
        {
            "name": "NonProd",
            "edge-locations": [
                "us-east-1",
                "eu-west-1"
            ],
            "require-attachment-acceptance": false
        }
    ],
    "attachment-policies": [
        {
            "rule-number": 100,
            "condition-logic": "or",
            "conditions": [
                {
                    "type": "tag-value",
                    "operator": "equals",
                    "key": "env",
                    "value": "prod"
                }
            ],
            "action": {
                "association-method": "constant",
                "segment": "Prod"
            }
        },
        {
            "rule-number": 200,
            "condition-logic": "or",
            "conditions": [
                {
                    "type": "tag-value",
                    "operator": "equals",
                    "key": "env",
                    "value": "nonprod"
                }
            ],
            "action": {
                "association-method": "constant",
                "segment": "NonProd"
            }
        }
   ]
}

For this walkthrough, we’ll use only the AWS Management Console, although you can also use the AWS Command Line Interface (AWS CLI) or AWS SDK. If you want an example of how to create a similar architecture using Infrastructure-as-Code (IaC), then check the following aws-samples repository on GitHub. We’ll only show the configurations required to create the resources in the us-east-1 Region, but note that the same steps will be needed to create the resources in the eu-west-1 Region. We’ll proceed as follows:

1. Create the Transit Gateway policy tables.
2. Create the peering between the Transit Gateway and Cloud WAN’s core network.
3. Create the Transit Gateway route table attachments (one per routing domain).
4. At this step, we’ll review how the dynamic route propagation is done between the Transit Gateways and the core network.
5. Lastly, as an optional step, we’ll register our Transit Gateways in the AWS Network Manager global network for centralized visibility and management.

Before starting, note that the peering between Transit Gateway and Cloud WAN – as well as the communication between network edges in the core network – uses eBGP. Make sure the Transit Gateway ASNs are different from the ASNs used in the Cloud WAN core –network. You can update the ASN of the Transit Gateway using the modify-transit-gateway AWS CLI command.

1) Create Transit Gateway policy tables

From the VPC console, under the “Transit Gateways” tab at the left-hand side, you can move to the “Transit gateway policy tables” section to create a policy table. Remember that this resource is used to achieve dynamic routing over the Cloud WAN – Transit Gateway peering.

Figure 2. Amazon VPC console view – Transit gateway policy tables section

Figure 3. Creating Transit gateway policy table (in us-east-1)

Remember to also create the policy table for the Transit Gateway in eu-west-1.

2) Create Transit Gateway – Cloud WAN peerings

Once our Transit Gateway policy tables are created, we can proceed to create the peerings. From the Network Manager console, under the “Core network” tab at the left-hand side, you can move to the “Peerings” section to create the peering connection.

Figure 4. AWS Network Manager console view – Peerings section

Figure 5. Creating Cloud WAN – Transit Gateway peering (in us-east-1)

As you can see in the previous figure, when creating a peering connection from the console, you can also configure the following:

• Creation and association of a policy table. In this example, we’ve already created the policy table in the previous section. Therefore, we’re associating it at the same time that we create the peering. Even though in the console this step is shown as a single action, on the backend this workflow requires several API calls (peering creation, policy table creation, and policy table association). Therefore, take this into account when creating these resources using the AWS CLI, AWS SDK, or any IaC solution.
• You can register the Transit Gateway to peer with the global network. We aren’t doing this in this step, as we’ll cover this action later.

When you create the peering, if you selected the creation or association of the policy table, then you’ll see the following. Allow the workflow to complete before proceeding.

Figure 6. Cloud WAN – Transit gateway peering creation process (from the AWS Management console)

Remember to also create the peering connection between the Transit Gateway in eu-west-1 to the core network.

3) Create Transit Gateway Route Table attachments

Once our peerings are created, and the policy tables are associated with those attachments, we can proceed to create the Transit Gateway Route Table attachments. These attachments will extend a specific Transit Gateway route table to a Core Network’s segment, and we’ll achieve the segmentation that we’re looking for.

From the Network Manager console, under the “Core network” tab at the left-hand side, you can move to the “Peerings” section to create the peering connection. We must create two attachments per Region – one for Prod and the other for Non-Prod traffic.

Figure 7. AWS Network Manager console view – Attachments section

Figure 8. Creating Transit Gateway Route Table attachments (in us-east-1)

Note the creation of the tags in the attachments. Checking the “attachment-policies” section in the Core Network’s policy (shown in the following), we automate the association of attachments to the segments if this attachment contains a tag with the format: key = “env” and value = “prod”/“nonprod”. Therefore, when the attachments are created, they will automatically be associated with the specific segment, and the network will be updated automatically, as shown in the following figure. Note that you can also specify if attachment acceptance is required prior to the network update.

"attachment-policies": [
    {
        "rule-number": 100,
        "condition-logic": "or",
        "conditions": [
            {
                "type": "tag-value",
                "operator": "equals",
                "key": "env",
                "value": "prod"
            }
        ],
        "action": {
            "association-method": "constant",
            "segment": "Prod"
        }
    },
    {
        "rule-number": 200,
        "condition-logic": "or",
        "conditions": [
            {
                "type": "tag-value",
                "operator": "equals",
                "key": "env",
                "value": "nonprod"
            }
        ],
        "action": {
            "association-method": "constant",
            "segment": "NonProd"
        }
    }
]

Figure 9. Transit Gateway Route Table attachments

Remember to also create the Transit Gateway route table attachments in eu-west-1. With this, the peering connectivity between Cloud WAN and the Transit Gateways is done, and we can proceed to check the dynamic exchange of the routes. In the following figure, you’ll see the final status of our architecture, with the resources created highlighted with arrows.

Figure 10. Network architecture (final state)

4) Review route exchange in Transit Gateway and Cloud WAN

If you move to any of the Transit Gateways and check the route tables (for example we’ll check the Non-Prod route table in us-east-1), then you can see that the peering connection propagates in the Transit Gateway route table (as shown in the following figure). However, in the next figure we can see that not all of the routes from the other Region are propagated – only the ones from the same routing domain.

Figure 11. Transit Gateway NonProd route table propagations (us-east-1)

Figure 12. Transit Gateway NonProd routes (us-east-1)

To have a global view of the routing exchange in the network, you can also move to Cloud WAN’s console, and under “Core network”, you can select the “Routes” tab to filter by segment and the Region to see which routes have been exchanged. The following figure shows the NonProd segment in eu-west-1. The routes where you see a Dash (“-“) under the “Destination” column are the ones that came from attachments in the same Region as the network edge that we are in. Meanwhile, routing exchange from other Regions is shown with the specific Region name (and segment name), for example, nonprod | us-east-1.

Figure 13. Cloud WAN’s routes (us-east-1)

5) (Optional) Add your Transit Gateways in AWS Network Manager

One optional step is registering the Transit Gateways in the Global Network. By doing so, with AWS Network Manager you can centrally manage your global network across AWS Accounts, Regions, and on-premises locations.

From the Network Manager console, under the “Transit gateway network” tab at the left-hand side, you can move to the “Transit gateways” section to do the registration. Select the Transit Gateways from the list and select Register Transit Gateway to complete the step as shown in the following figure.

Figure 14. AWS Network Manager console view – Transit gateways section

Figure 15. Registering Transit gateways to Network Manager

To register Transit Gateways from other AWS Accounts, you must make sure that you’ve enabled multi-account support for Network Manager. If you go back to the “Transit gateway network” section, then you can verify that the Transit Gateways are now visible in your inventory as show in the following figure.

Figure 16. AWS Network Manager overview

Considerations

• Regarding the peering between Cloud WAN and Transit Gateway.
  • As mentioned earlier, the Transit Gateway ASN should be different from the ASN used in Cloud WAN’s Core Network edges, as well as from the other Transit Gateway ASNs. You can update the ASN of the Transit Gateway using the modify-transit-gateway CLI command.
  • A Transit Gateway used for peering must be in the same Region as the core network.
  • Refer to the Cloud WAN documentation for the route evaluation order.
• You can use AWS Resource Access Manager (AWS RAM) to share a Core Network across accounts or across your AWS Organization. You must share it from the N. Virginia (us-east-1) Region so that other Regions can see the resource.
  • A shared Core Network currently only supports Amazon Virtual Private Cloud (Amazon VPC) and Transit Gateway route table attachments.
  • When creating a peering from a shared Core Network, you must make sure that the account you use to create the peering has the correct permissions. Refer to the Cloud WAN documentation to find the required permissions.
• Transit Gateway policy table association is only supported in Cloud WAN – Transit Gateway peerings.
• If you want to automate the association of a Transit Gateway route table attachment to a Cloud WAN’s segment, then make sure that you tag this attachment when creating it. For more information about the Core Network policy and the different actions that you can perform in Cloud WAN, check the Cloud WAN documentation.

Conclusion

In this post, we’ve reviewed different use cases where Transit Gateway and Cloud WAN can be used together, for example traffic segmentation, and walked through the steps of how to setup. From the creation of the peering, to the association of a Transit Gateway route table attachment to a Cloud WAN segment, we’ve presented an end-to-end scenario to achieve traffic segmentation in multi-Region environments. Visit the Cloud WAN product page, documentation, and pricing page for additional information.

About the authors