American Tower private network using hybrid infrastructure

The American Tower logo is a registered trademark of and all rights are reserved by ATC TRS V LLC. Used by permission

Introduction

5G technology is expected to revolutionize the retail industry by providing a platform for retailers to engage with customers in new ways and drive operational efficiencies. With 5G as a catalyst, we are entering a period of hyper-connectivity and anticipate continued proliferation of connected devices. 5G, together with edge computing, can enable retailers to improve the in-store customer experience in terms of personalization and engagement with products. It also serves as a catalyst for operational efficiency improvement, such as inventory, store operations, and labor productivity. Retailers can also use advanced artificial intelligence (AI) and machine learning (ML) to personalize pricing and promotions in store. On the operations side, there are 5G retail use cases that set a higher standard for inventory productivity, such as real-time out-of-stock, shelf replenishment, and shrink reduction.

Figure 1 AWS Cloud continuum

AWS edge services such as AWS Outposts, AWS Outposts Servers and AWS Snowball Edge have been proven to be instrumental in building 5G Private Networks, and deliver data processing, analysis, and storage close to endpoints, allowing deployment of APIs and tools to locations outside of AWS data centers. These services help extend the cloud closer to the applications, enabling ultra-low latency, intelligent, and real-time responsiveness. In addition, automation tools as described in the later sections make these Private Network Deployments that leverage AWS Edge services highly repeatable.

Experience consistency from edge to cloud

American Tower, a leading digital infrastructure provider, leveraged the AWS global infrastructure on a recent pilot private 5G standalone network at a premier Las Vegas shopping mall. American Tower chose Athonet 5G core software to deploy 5G Network functions (NFs) on the AWS Cloud.

One design principle of 5G is Control and User Plane Separation (CUPS). CUPS is essential to 5G networks because it allows operators to separate the core network functions into a control plane that can be hosted in a centralized location (Availability Zone (AZ) in an AWS Region) or AWS Local Zone, and for the user plane to be placed closer to the application it is supporting. This minimizes network travel time or latency between the user and the application.

• AWS Region: American Tower used AWS Regions as a centralized location where they deployed the core network Control Plane (CP). The CP carries the administrative and signaling traffic, such as control commands and authentication and authorization of users. There are functions that communicate with control or signaling traffic that are not relatively latency sensitive, such as the Access and Mobility Function (AMF), the Session Management Function (SMF), and the Unified Data Management (UDM) function. Furthermore, the control plane can communicate with multiple user planes in multiple locations.
• AWS Outposts servers: American Tower deployed User Plane Functions (UPF) on-premises, closer to the end users at this pilot private 5G standalone network. By stretching the 5G network using AWS Outposts server, American Tower could manage all the infrastructure and 5G core network from a single pane of glass using the AWS Management Console.

The Outposts server 2U form factor can be easily hosted in the IT server room. The network can seamlessly extend existing Amazon Virtual Private Cloud (VPC) to AWS Outposts servers in an on-premises location. After installation, a user can create a subnet in its regional VPC and associate it with an Outpost, just as the user associates subnets with an AZ in an AWS Region. Instances in Outpost subnets communicate with other instances in the AWS Region using private IP addresses, all within the same VPC. Outposts servers have a Local Network Interface (LNI) that provides a Layer 2 presence on a local network for AWS service endpoints.

Figure 2 AWS Outposts server 2U

Under the shared responsibility model, AWS is responsible for the hardware and software that run AWS services. This applies to AWS Outposts, just as it does to an AWS Region. For example, AWS manages security patches, updates firmware, and maintains the Outpost equipment. AWS also monitors the performance, health, and metrics for the Outpost and determines whether any maintenance is needed.

Architecture

Figure 3 Hybrid deployment architecture

For the Control Plane: A central CP runs on Amazon Elastic Compute Cloud (Amazon EC2) in a private subnet in the AWS Region.

• Connectivity through service links stretches the Amazon VPC over the AWS Region and on-premises location, enabling seamless communication over private IPs between the 5G Core control and user plane functions.
• AWS Site-to-Site VPN enables secure communication between the CP in the AWS Region and radio access network (Next generation NodeB; gNB) in the mall. The 5G N2 interface handles the communication between the gNB and the AMF, and optionally the 5G N3 interface, which handles the user traffic between the gNB and user plane hosted in the AWS Region in case of non-latency sensitive use cases.

For the User Plane functions, we have two Data networks (DN) each pointing to UPF1 (edge) and (UPF2) Region. DNN1: Local_ Mall_Applications, and DNN2: Cloud_Applications

• UPF1 hosted on Outposts server on-premises serves DNN1 Leveraging the Local LNI network, the UPF breakouts the internet traffic to the local ISP provider on-premises or communicate locally with MEC applications or on-premises applications.
• UPF2 hosted on AZ: DNN2: Alternatively, connect to the internet using an Internet Gateway (IGW) via the Outposts VPC subnet.

High availability architecture

Figure 4 High availability – Hybrid setup with Outposts server and separated CU/UP

To achieve Application Redundancy, it is recommended to configure at least two Control Planes and two User Plane functions, as standalone active/active nodes. In addition, NG-Flex as defined in (3GPP TS 38.401) can provide the capabilities to interface with multiple Control Planes and User Planes. The NG interface connects NG-RAN (Next Generation Radio Access Network) to 5GC (5G Core). 3GPP TS 38.413 defines the communication protocol for this interface. A gNB can be connected to multiple AMFs (Access and Mobility Function) in NG-Flex configuration, as shown in Figure 3. The selection of AMF by a particular gNB for a UE(user equipment) is done based on service requirements, slice information, and other parameters. Further, Ng-Flex allows for load balancing across the AMF pool, as shown in Figure 4.

Hence, 5G Core Network Application redundancy is achieved using the NG-flex capability on the RAN side, and standard 3GPP selection procedures on the Core side.

In NG-Flex configuration, each gNB is connected to all AMFs within an AMF Region, providing redundancy in case of failure in one of the Control Planes.

Walkthrough

The Athonet software Amazon Machine Image (AMI) ships with all the 5G NFs, which are enabled/disabled based on how the EC2 instance is intended to run. This enabled American Tower to deploy the same AMI anywhere and simply activate only the needed NFs, such as an All-in-One 5GC, or split UP/CP.

This approach not only simplifies the deployment of the Athonet software with the AWS infrastructure, but also further simplifies the management and orchestration of the software, as it is the same software that runs whether the EC2 instance acts as the 5G control plane or the 5G user plane.

An additional benefit is that the network can be built to suit the initial small capacity requirements and expand with additional UP or CP additions as the network grows with network demand.

Automation

Having the right automation solution becomes essential when scaling to more than 1500 sites. AWS provides an automation framework built on AWS serverless services that scales automatically, provides built-in high availability, and a pay-for-use billing model to increase agility and optimize costs.

The AWS automation framework deploys applications and network workloads, such as Athonet 5G Core components, across AWS infrastructure – Region, Local Zones, or Outposts, and leverages the consistent experience that the AWS cloud continuum provides.

Figure 5 AWS serverless automation framework

When deploying a new private network, the operator specifies the desired location for the CP and UP components. The AWS automation framework handles executing the needed set of steps to deploy the solution. Additionally, the AWS automation framework can also deploy MEC applications, achieving a fully automated end-to-end deployment of the solution.

The following is a non-exhaustive list of steps that are part of the automated deployment workflow:

1) Creating the networking and launching the EC2 instance

1. a. Creating the VPC
2. b. Setting up Security Groups
3. c. Setting up Route Tables
4. d. Instantiating the Athonet 5G Core

2) Configuring the Athonet 5G core

1. a. Configuring 5G network interfaces
2. b. Configuring AMF, SMF, UDR, and UPF
3. c. Creating subscriber profiles in the UDM
4. d. Provisioning subscribers

3) Deploying applications

Traffic separation and parametrization of the network design

Separation of different traffic types is achieved for security and performance purposes. The network design is also templatized to speed the deployment of multiple sites in different locations programmatically:

• OAM – can serve the 5GC management purposes.
• Telco Signaling – reserved for traffic signaling purposes.
• User Traffic – toward the DN (ideally 1 VRF for each Data Network), this makes sure of separation of traffic at the Data Network Name (DNN) levels, when multiple DNNs are needed.

A network plan is designed for each running instance of the Athonet software with a common network plan that can cater for any network configuration types with all interfaces with a pre-assigned IP address scheme.

Figure 6 Scaling UPF servers

Conclusion

In this post, we discussed the AWS cloud continuum and how customers seamlessly deploy private 4G/5G core networks on multiple AWS infrastructure options, managed with a single pane of glass. Customers benefit from a wide AWS partner ecosystem to deploy end user applications that support their mission, with examples such as video analytics at the edge for safety and security, industrial automation, mission critical communications, robotics, and augmented reality/virtual reality (AR/VR). AWS also recently launched Integrated Private Wireless on AWS, which is designed to provide enterprises with managed and validated private wireless offerings from leading Communications Service Providers (CSPs). The offerings integrate CSPs’ private 5G and 4G LTE wireless networks with AWS services across AWS Regions, AWS Local Zones, AWS Outposts, and AWS Snow Family.