Managing IP pools across VPCs and Regions using Amazon VPC IP Address Manager

Since the inception of IP networks, network engineers and operators have sought systems, solutions, and procedures to help them efficiently plan and manage IP spaces. AWS recently launched a new service named Amazon VPC IP Address Manager (IPAM) to make it easier for you to plan, track, and monitor IP addresses for your AWS workloads.

Today, the demand for IP has grown exponentially. Technologies like IoT and the Cloud have pushed the demand far beyond the border of remote offices, but to tens, hundreds, or even thousands of networks hosted in a Cloud. This exponential growth in demand requires an automated way to make use of your IP space.

We have found that many of our customers use spreadsheets or homegrown tools to manage their IP addresses. These methods are time-consuming to use and require manual updates and synchronization across inter-connected networks.

Introducing Amazon VPC IP Address Manager (IPAM)

Amazon simplifies IP address management through the new Amazon VPC IPAM service. IPAM enables you to plan, track, and monitor your IP spaces across your AWS accounts, VPCs, and AWS resources.

We built IPAM to serve as a single source of truth for all IP address-related usage information across all of your AWS accounts. It provides five key features and benefits. These are:

• IP Planning & Automated Allocation of IP Addresses helps you automate IP address allocations across hundreds of accounts and VPCs based on configurable business rules.

• IP Address Usage Tracking & Monitoring helps you to monitor your IP addresses and provides alerts when it detects potential issues such as depleting IP addresses that can stall your network’s growth or overlapping IP addresses that cause erroneous routing.

• IP Network Observability provides you with a single pane of glass/dashboard to view your entire IP address space across all your AWS accounts and VPCs.

• IP Address Auditing allows you to use automatically retained IP address monitoring data to do retrospective analysis and audits of your network security and routing policies.

• Network Troubleshooting, such as figuring out if a network connectivity issue was caused by IP address misconfiguration, is easier and faster.

The Amazon VPC IPAM for multi-VPC environments

To get started, create an IPAM instance in your AWS Account. This IPAM now forms the parent class that all IP addresses are allocated and managed from. The various IPAM constructs are described and shown in the following figure (Figure 1).

Figure 1: Amazon VPC IPAM Logical constructs

Amazon VPC IPAM scopes

When creating an IPAM, we provide you with two IPAM Scopes. An IPAM Scope is a top-level container that represents a single network and encapsulates the IP space that serves that network.

The creation of an IPAM provides you with two default scopes, private and public. As the name suggests, private scopes are used for your private IP address space (that is your non-Internet routable IP space, as denoted by RFC 1918).

Public scope is used to manage internet routable IP addresses.

While planning your IP space, we recommend using a unique IP space as much as possible to prevent overlaps. If overlapping IP spaces must be used for unconnected networks, these must be carefully designed. In such cases, IPAM scopes provide benefits of IP space reusability. If you have unconnected networks, then you can create a scope for each unconnected network and reuse the same IPs in across these different scopes.

Amazon VPC IPAM pools

The creation of an IPAM Pool is the next step after creating your IPAM Scope.

An IPAM Pool is a collection of contiguous IP address ranges. IPAM creates a top-level pool and then sub-divides it into a hierarchy of Regional pools. AWS Resources in that Region, like Amazon VPCs, queries the Regional pool for an available CIDR range. This eliminates or reduces the probability of resources having overlapping IP spaces. You can implement an overlap event workflow to mitigate potential routing issues.

Pools make it possible for you to organize IP addresses according to your routing and security needs. For example, if you have separate routing and security needs for ‘prod’ and ‘dev’ applications, you can create pools for each environment.

That said, you can create parent pools that create specific pool types if needed. These parent pools will act as a super-net for the child pools.

Figure 2: Multi-VPC, multi-Region AWS Environment with IPAM

In figure 2, we configured the IPAM service to support a multi-Region, multi-VPC environment for IP address allocation and management. The environment had existing VPCs (VPC B, C, and D in Region-A) before we started using IPAM. In such an environment, a network administrator must follow several steps to configure IPAM and use it with new and existing VPCs. In the scenarios that follow, we walk through the configuration of this sample environment using the AWS Management Console. We will use us-east-1 and us-west-2 Regions for illustration. Please refer to the supported Regions in the Amazon VPC IPAM launch documentation.

Scenario 1: Creating Amazon IPAM Pool structure for your environment

IPAM provides a lot of flexibility in creating IPAM pools for your environment. To create an IPAM pool to allocate and manage IP addresses for your environment, we recommended creating a well-designed hierarchy of IPAM pools that reflect the segregation of applications environments on AWS. You can either sub-divide the top-level pool into child pools corresponding to different business units or application environments or Regions. Some of the recommended patterns are described in Example IPAM Pool plans in AWS documentation.

You can use the following steps to create the required components in IPAM. We follow the approach of creating Regional pools for our sample AWS environment.

These steps are:

1. Create an IPAM instance
2. Create a Top-level IPAM Pool for your AWS workloads
3. Create IPAM Pools for different Regions
4. Provision the IPv4 CIDRs to the pools

Step 1: Create an Amazon IPAM instance

Create an IPAM instance using the AWS Command Line Interface (CLI) or the AWS Management Console as shown in the following snapshot (Figure 3):

Figure 3: Amazon VPC IPAM Console user interface for Create IPAM

When you create an IPAM instance, AWS automatically:

• Returns a globally unique resource ID (IPAM ID) for the IPAM.
• Creates a default public scope and a default private scope. The public scope is intended for public IP addresses that are going to be accessed via public internet. The private scope is intended for private IP addresses that are not accessed directly from the public internet.

In the scenarios we show in this post, we focus on Private IP address management and use the default private scope to configure the IPAM pools. However, the concepts shown here apply to public IP address management as well.

Since we want IPAM to manage and govern resources in both us-east-1 and us-west-2 Regions we have used the operating-Region parameter to specify additional Regions.

While you can add additional scopes if you are managing disparate networks with your own IP requirements, we work with the default private scope.

We create pools and provision IP address ranges from this IPAM instance to those pools in later steps.You can validate the status of IPAM instance creation instance creation using AWS CLI or with AWS Management Console as shown in the following screenshot (Figure 4):

Figure 4: AWS Console user interface showing IPAM instance

Step 2: Create top level IPAM pools for your AWS workloads

In the preceding architecture, we created an IPAM pool that encapsulates the IP address space that our workloads within AWS should use. We created this as a top-level-pool using the private scope of IPAM. Since us-east-1 is the primary Region, we create this pool in that Region using the AWS CLI or the AWS Management Console, as shown in the following screenshot (Figure 5):

Figure 5: AWS Console user interface for IPAM Pool creation

You can validate the status using AWS CLI or with AWS Management Console, as shown in the following snapshot (Figure 6). Notice that the Locale setting is “None”. When you create an IPAM pool, the pool belongs to the AWS Region of the IPAM instance by default. Leaving this value to None allows you to create child pools under this top-level pool for different Regions.

Figure 6: AWS console user interface for IPAM Pool

Step 3: Create IPAM pools for different Regions

In this scenario, we would like to extend the use of this pool to other Regions. We do this by creating a child pool under this parent “top-level-pool” as shown in Figure 7. The IPAM control plane is associated with the Region you created your IPAM instance, so in order to create IPAM pools in different Region, we use the Locale setting. You can use the locale option when you create a pool to make the pool available to services in a Region other than the Region of the IPAM.

Figure 7: AWS Console user interface for IPAM Pool creation for regional pool

Now the VPC resources in us-east-1 can use this IPAM pool for CIDR allocation.

As you can see from the above steps, the top-level pool is not used to allocate IP CIDR prefixes to resources directly. To do this, we create child pools underneath to manage allocations. Therefore, for the top-level pool, the setting “Use this pool to allocate CIDRs to resources such as VPCs” is not turned on. We have turned on this option for Regional pools as we create VPCs using CIDR allocations from these pools. Note that AWS recommends creating an additional hierarchy of pools based on your needs and configure allocation rules from those pools, but we do not discuss that here for brevity.

Once this feature is turned on, you can configure rules for the allocations that are made within the IPAM pool.

Allocation rules makes it possible to configure the following:

• Whether IPAM should automatically import CIDRs into the IPAM pool if it finds them within this pool’s CIDR range. Notice that the parameter “Automatically Import discovered resources” has value “No”. This means that if VPCs are created without using IPAM, IPAM cannot map that VPC to the IP pool allocations it is managing and cannot detect any IP address overlaps or conflicts.
• The required netmask length for allocations within the pool.
• The required tags for resources within the pool. You can set an allocation rule stating that any resource that wants a CIDR from this pool must have a tag that matches the allocation rule tag requirements.
• The required locale for resources within the pool. The locale is the AWS Region where an IPAM pool is available for allocations. When you create a VPC, the pool that the VPC draws from must be in the same Region as the VPC.

Step 4: Provision the IPv4 CIDRs to the top-level and Regional pool

You can provision CIDR block to the top-level pool using AWS CLI or with AWS Management Console as shown in the following snapshot (Figure 8):

Figure 8: AWS Console user interface for CIDR provisioning for IPAM Pool

For each Regional IPAM pool sourced from top-level pool, we can provision a subset of the top-level pool’s CIDR block and check if the CIDR block is provisioned (Figure 9):

Figure 9: AWS Console user interface for IPAM Pool CIDR provisioning for regional pool

You can check the status of ipv4 CIDR Block association to Regional IPAM pool using the AWS CLI or with AWS Management Console as shown in the following snapshot (Figure 10):

Figure 10: AWS Console user interface for IPAM Pool details

You must repeat Steps 3 and 4 for the us-west-2 Regional pool as well.

Scenario 2: Creating new VPCs in various Regions using Amazon IPAM

Once you finish setting up IPAM pools as described in Scenario 1, you can create VPCs using those pools without having to worry about IP address conflicts or policy violations. Create a VPC and assign an IPv4 CIDR block to the VPC from the pool using the AWS CLI or with AWS Management Console, as shown in the following snapshot (Figure 11):

Figure 11: AWS Console user interface for VPC creation using IPAM

The CIDR block is chosen for this VPC based on the allocation policy for the specified pool and ensuring that the same CIDR block is not allocated to other resources.

Similar operations are performed to create VPCs in us-west-2 Region.

You can check the status of the VPC resource within IPAM using the AWS CLI or with the AWS Management Console (Figure 12). This is useful to see the VPCs that are mapped to each VPC pool, the IPAM Resource status, and IPUsage.

Figure 12: AWS Console user interface for VPC resource status within IPAM

Scenario 3: Discovering VPC Resources created without using Amazon IPAM

IPAM not only creates VPC resources using IPAM pool-specified allocations but it also detects VPCs created without using IPAM. You can configure IPAM to automatically import any VPCs you have in your environment that fall within the CIDR range in the pool.

To activate automatic import on a pool, change the pool using the AWS CLI or with AWS Management Console, as show in the following snapshot (Figure 13):

Figure 13: AWS Console user interface for IPAM Pool – activating automatic import

Consider four VPCs in Region-A (us-east-1), VPC A, VPC B, VPC C and VPC D in Figure 2.

• VPC A is created with CIDR block of 10.0.0.0/24 using IPAM.
• VPC B is created with CIDR block of 10.0.1.0/24 without using IPAM.
• VPC C is created with 10.0.0.0/24 CIDR range without using IPAM.
• VPC D was created with a CIDR block of 172.31.0.0 without using IPAM.

Since your IPAM pools are configured with “auto-import” = true, Amazon IPAM detects these VPC configurations for VPC B, C and D.

• It maps VPC B to the us-east-1 Regional IPAM pool as the CIDR block belongs to that pool and un-allocated. The status will be shown as “Compliant” and the overlap status as “Nonoverlapping”.
• VPC C is also mapped to the IPAM pool and marked with a compliance status of “Unmanaged” as it does not have a CIDR allocated from an IPAM pool and is not being monitored by IPAM for potential CIDR compliance with pool allocation rules. Since the CIDR block conflicts with VPC A, the overlap status will be shown as “Overlapping.”
• As soon as VPC C is created with a CIDR block that overlapped with VPC A, the overlap status of VPC A was changed to “Overlapping”. However, its compliance status is “Compliant” since the CIDR block was allocated from the IPAM Pool in compliance with allocation rules and it will continue to be monitored for overlaps.
• VPC D is not mapped to the IPAM pool as its CIDR block is outside the IPAM Pool’s provisioned CIDR and shows the status as “Unmanaged.”

We can view these using the AWS CLI or with AWS Management Console, as shown in the screenshot that follows (Figure 14):

Figure 14: AWS Console user interface for IPAM Pool resource status view

If you have VPCs in your environment that are not part of any pool, or cannot be mapped to any pool, then the status is “unmanaged”.

Scenario 4: Observing your multi-VPC environment for IP Address Management

IPAM automatically starts monitoring the usage of IP addresses across accounts and VPCs. We can set alarms for IP address utilization and rule compliance to fix IP address related issues proactively. Finally, we can use the current and historical data made available by IPAM to speed up network troubleshooting and audits.

VPC address allocation:

Monitor the CIDR utilization of your VPC by Querying the IPAM pool for each respective Region using the AWS CLI or with AWS Management Console as show in the following snapshot (Figure 15):

Figure 15: AWS Console user interface for IPAM pool allocations

IP Usage utilization:

Monitor the IP utilization of the following resources using the AWS CLI or with the AWS Management Console, as shown in the following snapshot:

Subnets

IP usage is defined as the percent of IP space used in the subnet. In the example that follows (Figure 16), Subnet (10.0.1.0/26) has total 64 IP, out of which five are reserved by default, so 5/64 =.0781

Figure 16: AWS Console user interface for IPAM Pool resource view for subnets

Historical insights for an IP address or CIDR:

IPAM automatically keeps your IP address monitoring data. You use the historical data to analyze and audit your network security and routing policies. You search for historical insights using the AWS CLI or with the AWS Management Console for the following resources as shown in the following snapshots (Figure 17 & 18):

• VPCs
• VPC subnets
• Elastic IP addresses
• EC2 instances
• EC2 network interfaces

What resources are currently associated with 10.0.0.0/24?

Figure 17: AWS Console user interface for IPAM – IP historical insights

What resources were associated with 10.0.0.0/24 between 1am and 6pm?

Figure 18: AWS Console user interface for IPAM – IP historical insights

Summary

As you scale up your AWS environment to support the needs of your business, you need to connect & manage multiple VPC environments across Regions and on-premises networks. Amazon VPC IPAM provides a single pane of glass visibility into IP allocation to resources across your networks and management to ensure efficient use of IP space as well as avoid conflicts. This ensures that IP planning does not impede your growth goals.