Visualize and diagnose network reachability across AWS accounts using Reachability Analyzer

It is common to encounter network designs on AWS with resources that belong to multiple AWS accounts. For example, you may have several AWS accounts with Virtual Private Clouds (VPCs) in those accounts connected to an AWS Transit Gateway in a central networking account. You may need to determine or diagnose network reachability between AWS resources that are in different AWS accounts. For example, you might want to determine network reachability between a pair of Amazon Elastic Compute Cloud (Amazon EC2) instances that are in different AWS accounts in your AWS organization.

In this post, we show how to use Reachability Analyzer in a network that spans multiple AWS accounts. First, we describe a sample network design.

Sample multi-account network on AWS

You can deploy workloads into multiple AWS accounts to realize several benefits, such as the ability to group workloads by business purpose and ownership, or the ability to apply distinct security controls by environment. For example, you can apply different security, network, and operational policies for the non-production and production environments for a given workload by deploying them into separate AWS accounts. Furthermore, you may have a security policy to make sure that resources in the non-production environments don’t have a network path to resources in the production environment.

The diagram in the following figure shows an example multi-account setup with four VPCs that are each owned by a different AWS account. Moreover, all of these VPCs are connected to a Transit Gateway, which is deployed in a Network Admin account. There’s a web-server in Account-1’s VPC that must communicate with an database instance deployed in Account-4’s VPC.

Figure 1: Example multi-account deployment

Now that we understand the sample network, imagine you must determine whether the web server can connect to the RDS database. Reachability Analyzer can now trace network paths across resources deployed in multiple AWS accounts. In this post, we demonstrate how to set up Reachability Analyzer in a multi-account deployment.

How multi-account Reachability Analyzer works

Reachability Analyzer is an Amazon VPC feature that analyzes your network configuration and answers the question of whether or not a network path exists between a source and destination resource in your network. You can specify various AWS resources as sources and destinations, such as EC2 instances, Network Interfaces, VPC Endpoints, Transit Gateways, etc. For a complete list of supported resources, see the VPC Reachability Analyzer documentation.

With support for AWS Organizations in VPC Reachability Analyzer, you can now analyze network reachability between a source resource and a destination resource that are in different AWS accounts in your organization.

Let’s look at the steps needed to use Reachability Analyzer in a multi-account deployment.

Step 1: Enable trusted access for Reachability Analyzer from AWS Organizations

First, enable trusted access for Reachability Analyzer from your organization’s management account. Enabling trusted access authorizes Reachability Analyzer to create the AWS Identity and Access Management (IAM) service-linked roles required to run the network analyses across the AWS accounts in your organization.

To do so, sign in to your organization’s management account, navigate to  console, and select “Settings” under Reachability Analyzer from the left navigation. The following figure shows this process:

Figure 2: Enable Reachability Analyzer trusted access

Step 2: Select Delegated Administrator accounts for Reachability Analyzer

You can designate certain AWS accounts in your organization as the delegated administration account for Reachability Analyzer. Only the delegated administration accounts and the organization’s management account can run cross-account network reachability analyses.

To select delegated administrator accounts, from the organization’s management account, select “Register delegated administrator”, and select the accounts to register as the delegated administrators from the drop-down. The following two figures show the process.

Figure 3: Register delegated administrator

Figure 4: Choose delegated administrator account

Registering a delegated administrator for Reachability Analyzer is a global action. Once you assign an AWS account as a delegated administrator, you can use Reachability Analyzer from this account to trace networking paths in any AWS Region in that partition.

You can now run network reachability analyses across AWS accounts in your organization from the organization’s management account, or the selected delegated administrator AWS accounts.

Step 3: Analyzing network paths

In this section, we’ll create and analyze the network path from a delegated administrator AWS account. Here’s the sample network topology for which we’ll create the path.

Figure 5: Network Topology

The previous diagram shows the various resources that VPC Reachability Analyzer will check. For example, security groups attached to each server’s network interface, Network Access Control Lists (NACLs) attached to each subnet, each subnet’s routing tables, and Transit Gateway’s routing tables. These configuration resources are colored purple in the figure, and some of these resources are highlighted in yellow circles with numbers. We refer to these numbers in the circles later in this post.

To create the path, navigate to VPC -> AWS Network Manager -> Monitoring and Troubleshooting -> Reachability Analyzer. Select Create and analyze path. Select the Source type, Source account, Source, Destination type, Destination account, and Destination, and select Create and analyze path. In this case, the source is an EC2 instance and the destination is the elastic network interface of the RDS database server. We also included TCP port 3306 in the path definition, as this is the port on which the webserver would communicate with the database. The following figure shows the details of the path.

Figure 6: Reachability Analyzer Path details

After this, Reachability Analyzer analyzes the path between the source and the destination, and provides the outcome of the analysis on the console. In this example, the analysis reveals that the path is unreachable. The Path details section shows the error code, as shown in the following figure.

Figure 7: Path Analysis result

The numbers in yellow circles in the previous figure correspond to the numbers in the network diagram in Figure 5. In this case, Reachability Analyzer traced the network path across resources deployed in different accounts, and discovered that there is no communication between the web server and the RDS database instance. The explanation code of “ENI_SG_RULES_MISMATCH” means that the communication between the source and the destination is blocked by a Security Group. Whenever the analysis discovers a blocked path, VPC Reachability Analyzer highlights it in red color on the path details page. In the example in the previous figure, it was the Elastic Network Interface attached to the RDS database instance where the path was blocked. In case you have multiple intermediate resources blocking the network path, VPC Reachability Analyzer shows all of these resources in a single path analysis.

Furthermore, you can view the results of the path analysis as a JSON object by expanding the Details dropdown within the Explanations section, as shown in the following figure. The machine readable JSON detail includes the explanationCode field that can be used with automation to respond to the analysis result.

Figure 8: Path Analysis result as a JSON object

Now that we know the explanation for why the source instance can’t reach the RDS database, we can fix it by updating the rules in the security group associated with the RDS database instance. After the updates, the same path can be analyzed again, as shown in the following figure.

Figure 9: Path Analysis result after updating Security Group

As you can see, Reachability Analyzer shows that there is a network path between the web server and the RDS database instance. Select Path Details to view the entire path between the source and the destination.

In this example, we discussed how Reachability Analyzer can help with troubleshooting an end-to-end networking path in a multi-account deployment, wherein the source resources, destination resources, and all of the intermediate networking resources are all deployed in different AWS accounts.

Conclusion

In this post, you learned how to use VPC Reachability Analyzer to evaluate network paths that span resources in multiple AWS accounts, giving you better insights into your network configuration. We showed an example of evaluating a path to determine reachability between resources that span three AWS accounts. You can extend this idea by creating multiple paths and periodically analyzing these to evaluate compliance with network security policies.
This feature is available today, so try it out and let us know if you have any questions about this feature or post by contacting AWS Support.

About the Authors