Networking & Content Delivery
Designing geo-segmented AWS Cloud WAN with service insertion
Building a geo-segmented AWS Cloud WAN network with service insertion means meeting two requirements at once. First, network segmentation must respect Regional data residency: traffic that originates in Europe should stay in Europe, and North American workloads should stay in North America. Second, security teams want centralized traffic inspection for east-west (segment-to-segment), cross-geography, and north-south (internet egress) flows, using a consistent policy everywhere.
Meeting both requirements at once is where the design gets hard. The controls for data residency and the controls for centralized inspection interact, and those interactions shape every design decision in this post. AWS Cloud WAN addresses the first requirement with geo-restricted segments (segments pinned to specific edge locations). It addresses the second with AWS Cloud WAN service insertion and Network Function Groups (NFGs).
This post walks through a deployable four-segment, multi-Region reference architecture that combines them using dual-hop inspection. Along the way we cover the decisions this design forces. These include dual-hop versus single-hop trade-offs, NFG reuse rules, segment-pair limits, and an inspection virtual private cloud (VPC) routing requirement worth getting right the first time. All findings are lab-validated, and a ready-to-deploy AWS CloudFormation template is available in an accompanying aws-samples repository.
Combining geo-segments and NFG service insertion
Existing AWS Cloud WAN service insertion guidance treats segments and NFGs in isolation. A realistic multi-geo deployment needs all three inspection patterns at once. They are intra-geo inter-segment (production to non-production within a geography), cross-geo (North America production to Europe production), and north-south (each segment’s internet egress). Expressing all three in one policy with geo-restricted segments is where the constraints appear.
Reference architecture
The architecture, shown in Figure 1, uses a single global network with three Core Network Edges (CNEs) and four geo-restricted segments:
| Segment | Edge locations | Purpose |
NorthAmericaProd |
us-east-1, us-west-2 | NA production |
NorthAmericaNonProd |
us-east-1, us-west-2 | NA dev/test |
EuropeProd |
eu-west-1 | EU production |
EuropeNonProd |
eu-west-1 | EU dev/test |
Restricting each segment’s edge-locations is what enforces data residency. An attachment in a segment can only be created in that segment’s edge locations, so European workloads can’t accidentally land on a North American CNE.
All inspection here is inter-segment or egress, so the default segment configuration works.
Inspection is provided by Network Function Groups. This post recommends dual-hop mode, which inspects in both AWS Regions and needs five NFGs:
| NFG | Scope | Used in |
NorthAmericaNFG |
Intra-NA east-west inspection | send-via (dual-hop) |
EuropeNFG |
Intra-EU east-west inspection | send-via (dual-hop) |
CrossGeoNFG |
Cross-geography inspection | send-via (dual-hop) |
NorthAmericaEgressNFG |
NA internet egress inspection | send-to |
EuropeEgressNFG |
EU internet egress inspection | send-to |
The three east-west NFGs use send-via for inter-segment inspection. The two egress NFGs use send-to for internet egress. The first three follow from the geo design plus the segment-pair constraint. The egress split is forced by a dual-hop reuse constraint covered later.
An attachment can belong to a segment or a network function group, but not both. Each NFG also needs its own dedicated inspection VPC. A VPC holds only one core network attachment to a given core network. You can’t attach the same inspection VPC to multiple NFGs on different subnets. Each NFG can also have only one attachment per Region. That is why Figure 1 uses six separate inspection VPCs for five NFGs: CrossGeoNFG needs one in each Region.
The core network policy expresses that design as seven segment actions: four send-to (one internet egress per segment through its geographic egress NFG) and three send-via dual-hop (NA Prod↔︎NA Non-Prod, EU Prod↔︎EU Non-Prod, and NA Prod↔︎EU Prod cross-geo). Here is the core of the policy:
This snippet shows the segments, NFGs, and segment actions. It omits the attachment-policies block for brevity, though no attachment is associated without it. Attachment-to-segment and attachment-to-NFG mappings are driven by AWS Cloud WAN attachment tags. Rules are evaluated in rule-number order, lowest first. The full tag-based rule set is in the aws-samples template. Two behaviors are worth noting: send-via is bidirectional, so a rule from segment A to segment B automatically covers B to A and you don’t write the reverse rule. And send-to takes no destination CIDR as it steers the segment’s default route, which in a dual-stack segment means both 0.0.0.0/0 and ::/0.
Why cross-geo needs its own NFG: The segment-pair constraint
You might expect to route North America production to Europe production through NorthAmericaNFG, which already inspects North America traffic. The console blocks it at policy-creation time with a validation error:
A segment pair can only use ONE NFG for all send-via actions between them. Error: Segments X and Y are configured to use network function group A but are also configured to use network function group B
The limit is explicit: a given segment pair can use exactly one NFG across all its send-via actions. NorthAmericaProd can still use NorthAmericaNFG for its inspection with NorthAmericaNonProd. That is a different segment pair. But for the North America production ↔︎ Europe production pair, both geo NFGs are candidates. A dedicated CrossGeoNFG, with inspection attachments in both Regions, keeps cross-geo inspection explicit and the geo NFGs focused on intra-geo work.
Dual-hop versus single-hop
AWS Cloud WAN offers two send-via modes. Dual-hop inspects traffic in both the source and destination Regions. Single-hop inspects it once, in a single deterministically chosen Region.
The trade-offs, validated in the lab:
| Aspect | Dual-hop (recommended) | Single-hop (lighter alternative) |
| NFGs in this design | 5 | 3 |
| Inspection passes | 2 (both ends, across Regions) | 1 (one Region for the pair) |
| Per-Region control | Each Region inspects independently | One Region inspects for the pair |
NFG reuse with send-to |
Blocked (needs separate egress NFGs) | Allowed |
| Cross-geo latency overhead | Higher (double inspection) | Lower (single inspection) |
| Policy complexity | Higher | Lower |
Dual-hop is the recommended default because inspecting at both ends gives each Region its own inbound and outbound enforcement point. This is the symmetric model most compliance and security teams expect. The cost is more NFGs and a second inspection pass. That second pass applies only across Regions: within one Region the NFG route table resolves straight to the destination attachment, so traffic is inspected once per direction.
The reason this design needs five NFGs in dual-hop rather than three is a documented constraint that you might not notice until the policy fails:
An NFG used in a dual-hop send-via action can’t also be used for a send-to action. Error: the network function group X can’t be used in both dual-hop and either single-hop mode or send-to action
As a result, the three inspection NFGs can’t carry send-to egress, and the design needs two dedicated egress NFGs, for five in total.
A related constraint blocks mixing modes on a single segment:
A segment can’t participate in both single-hop and dual-hop send-via modes simultaneously. Error: segment X is used with multiple modes in send-via segment actions
NorthAmericaProd uses dual-hop for its intra-geo inspection, so its cross-geo action must also be dual-hop. Decide the mode per segment before you build.
When to choose single-hop: Choose single-hop when you only need single-sided inspection, for example an on-premises hairpin model where traffic is already inspected once.
The constraint that fails silently: Inspection VPC routing
The preceding constraints surface as console validation errors at policy-creation time. The inspection VPC routing requirement, by contrast, produces no error when misconfigured. The policy deploys cleanly and attachments move to the Available state. Traffic is still dropped silently. The cause is ordinary VPC routing. After the appliance re-emits an inspected packet, the inspection subnet’s route table needs an explicit route back to the core network, or the VPC drops it.
An east-west inspection VPC (one attached to an NFG used by send-via) needs two route tables.
┌──────────────────────────────────────────────────────────┐
│ North America east-west inspection VPC (NorthAmericaNFG) │
│ │
│ AWS Cloud WAN Subnet (hosts the AWS Cloud WAN ENI) │
│ Route: 10.0.0.0/8 → inspection endpoint │
│ Route: 2001:db8:48ea:4000::/56 → inspection endpoint │
│ Route: 2001:db8:4d9d:bf00::/56 → inspection endpoint │
│ ↓ │
│ Inspection Subnet (Gateway Load Balancer endpoint, │
│ AWS Network Firewall endpoint; │
│ appliance ENI in the lab) │
│ Route: 10.0.0.0/8 → Core Network │
│ Route: 2001:db8:48ea:4000::/56 → Core Network │
│ Route: 2001:db8:4d9d:bf00::/56 → Core Network │
│ │
└──────────────────────────────────────────────────────────┘The AWS Cloud WAN subnet forwards traffic to the appliance, which inspects it and then needs a route back to the Core Network. Without that return route on the inspection subnet, for either address family, the appliance still receives and re-emits the packet. The VPC then has nowhere to send it. There is no log line, no dropped-packet metric on the attachment, no policy error. You discover it only through connectivity testing.
Two more documented requirements complete the picture:
- Turn on appliance mode on the inspection VPC attachment, so both directions of a flow pin to the same Availability Zone (symmetric routing).
- Turn on IPv6 support on the attachment in a dual-stack design. Without it, the VPC’s IPv6 prefixes never appear in the segment or NFG route tables. This happens even though the VPC, its subnets, and the appliance interfaces all have IPv6 addresses. Like the missing return route, it produces no error.
For centralized IPv6 egress, translate the workload’s IPv6 source address on the egress appliance.
Inter-geo east-west, and what the packets show
Figure 2 shows an AWS Cloud WAN core network spanning us-east-1 and eu-west-1, with NorthAmericaProd, EuropeProd, and cross-geography NFG bands. A forward flow is inspected in a CrossGeo inspection VPC in each Region before reaching the EuropeProd VPC. The segment route table and both regional NFG route tables appear alongside.
(A) The NorthAmericaProd VPC route table matches 10.102.0.0/24 with its AWS Cloud WAN attachment as the target, sending the packet to the core network.
(B) The NorthAmericaProd segment route table at the us-east-1 edge matches 10.102.0.0/24 with the us-east-1 CrossGeo inspection VPC as the target.
(C) The appliance in us-east-1 inspects and returns the packet to the core network.
(D) The attachment now belongs to CrossGeoNFG, and this is where dual-hop earns its name: the NFG route table at the us-east-1 edge resolves 10.102.0.0/24 not to the workload but to the CrossGeo inspection VPC in eu-west-1.
(E) The appliance in eu-west-1 inspects and returns the packet to the core network.
(F) A CrossGeoNFG route table lookup at the eu-west-1 edge matches 10.102.0.0/24 with the EuropeProd VPC attachment as the target. Return traffic follows the same path in reverse and is inspected in both Regions.
IPv6: the path is identical, with 2001:db8:0a18:6c00::/56 in place of 10.102.0.0/24 at steps B, D, and F.
The same path with IPv6:
Both flows completed with no packet loss. Each appliance shows the packet arriving (In) and being re-emitted (Out). Removing the inspection subnet’s return route in us-east-1 doesn’t change that capture: the request still arrives and is still re-emitted, but the packet is dropped in the VPC afterward and no reply comes back. The eu-west-1 appliance captures nothing at all because the second inspection never happens, so neither appliance reveals the fault.
When to consider separate global networks instead
If cross-geography traffic is rare or already flows over AWS Direct Connect and on premises, a separate global network per geography avoids these constraints entirely. Each geography uses one NFG, with no cross-geo segment actions and no mode conflicts. The trade-off is that cross-geo connectivity moves outside AWS Cloud WAN (AWS Transit Gateway peering or an on-premises hairpin), which you design and operate yourself.
Deploy it
A complete, deployable reference is published in the sample-cloudwan-geo-segments-nfg-service-insertion repository. It includes CloudFormation templates for the core network and policy, parameterized workload and inspection VPCs, and Amazon Elastic Compute Cloud (Amazon EC2) packet forwarders standing in for firewalls. The repo’s docs/nfg-constraints.md lists every documented and lab-validated constraint with its exact error message. The template deploys the dual-hop policy described here. To run the single-hop variant, change each send-via mode and point the send-to actions at the geographic NFGs. In production, replace the EC2 forwarders with AWS Network Firewall or an appliance fleet behind a Gateway Load Balancer. The AWS Cloud WAN policy and inspection VPC routing are identical either way.
Cost considerations
AWS Cloud WAN bills per Core Network Edge, per attachment, and per GB of data processed. Inspection appliances bill separately. Dual-hop costs more on three axes. First, it needs more NFGs, and therefore more inspection VPC attachments. Second, cross-geo flows pass through an inspection VPC attachment in both Regions rather than one. You pay data-processing charges for two attachment hops instead of one. Third, cross-geo traffic crosses Regions between Core Network Edges. That adds inter-Region data transfer charges per GB on top of data processing. If your security posture doesn’t require per-Region inspection, single-hop removes the second pass and two NFGs. For current rates, refer to the AWS Cloud WAN pricing page.
Conclusion
Combining geo-restricted segments with NFG service insertion means planning for a handful of constraints up front. The clean policy has five parts:
- Geo-restricted segments for data residency.
- Dual-hop
send-viathrough three inspection NFGs, so each Region inspects independently. - Two dedicated egress NFGs for internet-bound traffic.
- Dual route tables in every east-west inspection VPC.
- Appliance mode turned on throughout.
Deploy the reference template in a test account, trace the flows against your own route tables and captures. Then swap the EC2 forwarders for AWS Network Firewall or your own appliance fleet. To learn more, refer to the AWS Cloud WAN service insertion documentation and its policy examples.