Zero touch deployment of multi-vendor private 5G networks

Analysts forecast that the market for private LTE and 5G networks will more than quadruple by 2026 (IDC). Wireless networking use cases in verticals such as manufacturing, industrials, and logistics will be significant drivers of this expansion. These are precisely the verticals that our AWS Customer Experience Center in Munich, Germany was built to help our customers explore. Over time, as we added more novel systems to the experience center, we noticed that an increasing number make provision for connection to private 5G networks.

Moreover, it has become quite common for mobile private networks to be deployed on AWS, thereby utilizing the elasticity of the cloud, and the operational overheads savings that can be achieved. This is a parallel development to our own AWS Private 5G service, which at the time of writing is available only in the US on CBRS spectrum.

Accounting for these trends, it seemed natural for us to add our own private 5G network to the Customer Experience Center to understand the state of the art. In this post, we’ll discuss the process and describe what we learned along the way.

Network design goals

Before we started building, we set the following high-level design goals for the network:

• Modular and extensible: the network will initially comprise one core software vendor and one integrated gNodeB vendor. However, it should be extensible to allow additional vendors, as well as an Open RAN architecture with separate CU, DU, and RU components. It shall be possible to change components independently of each other.
• Fully automated: the environment should support experimentation and prototyping—for example with new vendors—but it must also be possible to revert quickly to a “known good” state for customer demos. This implies that all resource creation and destruction must be fully automated.
• Reusable: there’s already significant interest within AWS to replicate the 5G-enabled Experience Center, as well as customer interest. Components must be reusable in other accounts and regions without modification and with minimal configuration change.
• Performant: to enable the widest possible set of use cases, the throughput of the 5G physical layer should be the only limiting factor. To enable very low latency use cases, the deployment of the 5GC and RAN software to a co-located AWS Outposts server instead of an AWS Region should be a run-time choice.
• Secure: all principles of cloud and network security will be followed strictly, including least privilege access, control and user plane separation, and end-to-end traffic encryption.

Because the Customer Experience Center is in Germany, we can apply to use the 3.7-3.8GHz spectrum that is made available by Bundesnetzagentur für Telekommunikation (BNetzA), the regulator, for private networks. This lies within the n77 5G band that extends from 3.3GHz to 4.2GHz (also encompassing the US CBRS, or n48, band). The availability of both the network equipment and devices that support this band is generally good and improving as of the time of writing (August 2022).

Previous work

In a previous post, we talked about the automated deployment of the Athonet Mobile Core on AWS. Since this was the first 5G Core that we wished to use, it was natural to extend the automation developed there to encompass the RAN software as well. Another related piece of work is the deployment of private networks using the AWS Snow Family of rugged, disconnectable devices to simultaneously run both private network cores and novel computer vision applications.

Architecture

The deployment and lifecycle operations of the end-to-end (E2E) private 5G network are self-service and automated through AWS native serverless compute services. A collection of AWS Step Functions state machines define the tasks to be executed to, for example, instantiate the E2E private 5G network (see the figure above).

The E2E private 5G network instantiation state machine starts by creating the cloud infrastructure on which the 5G service components will be deployed. This includes resources like the VPC, or subnets for Operations and Management (OAM), control plane, and data plane. In the next stage, Athonet 5G core and Airspan RAN management functions are deployed and activated in parallel tasks. The last stage deploys 5G web services that will be exposed to end users through the 5G access network. Both the cloud infrastructure and private 5G network components are defined as code in the form of AWS CloudFormation templates and made available for self-service as products by AWS Service Catalog. We use the AWS Service Catalog chaining feature to combine multiple smaller CloudFormation templates into a single, high-level product, such as the cloud infrastructure. This allows platform and application teams to easily build services using independently maintained and versioned products.

The E2E automation for private 5G networks has been modeled as an AWS Cloud Development Kit (AWS CDK) application. AWS CDK is an open-source software development framework to define cloud infrastructure and application resources using modern programming languages, such as Typescript, and provision them through CloudFormation. The AWS CDK application code is version controlled and stored in a code repository using AWS CodeCommit.

Customers can easily integrate the private 5G network automation framework with existing OSS/BSS systems using RESTful APIs as specified by ETSI NFV SOL005 API to perform Lifecycle Management Operations on Network Services. The SOL005 adapter shown in the following figure is implemented using Amazon API Gateway and AWS Lambda. The SOL005 adapter performs operations, such as Create Network Service (NS) resources, Instantiate NS instance, and Delete NS instance, that manage both 5G Core and 5G RAN network functions.

Lessons learned and further work

The overall experience of building the 5G lab has been hugely rewarding, and it has amply substantiated the well-known adage, “there is no compression algorithm for experience”. Here are some of the lessons we learned:

‘Right first time’ for physical infrastructure

In software and cloud infrastructure, we become accustomed to rapid iteration, with many build-run-debug cycles per hour. However, when it comes to physical infrastructure, there are additional considerations. For instance, one of our gNodeB shipments was delayed for several days in German customs, and subsequently some of the units also had to be returned and replaced, incurring further shipping delays.

Because we had established VPN connectivity between our cloud environment and the manufacturer, we could continue work using gNodeBs hosted in their facilities. In hindsight, the overall project would have been smoother if the exact devices to be shipped had been staged and tested remotely, and then shipped in a known-good state.

Regulatory interactions should start early

When dealing with regulatory authorities, such as the BNetzA, it pays to start the interaction as soon as possible. Other users of spectrum may need to be consulted about, or at least informed of, the application to use spectrum. This can cause delays to the application processing.

A remedy that we used in this case was to apply both for a long-term, unconditional license, and also for a short-term test and development license with additional conditions. We could use the latter while waiting for the former to be granted. A more general approach is the use of an automated Spectrum Access System, as adopted for CBRS in the US and under consideration by some European regulators for shared access spectrum bands.

Don’t underestimate integration

A key feature of the 3GPP 5G System architecture is the open, standardized interfaces between components. The O-RAN Alliance extends open interfaces further into the radio access network. The intended benefit of these open interfaces is the ability for operators to create networks using components from different vendors, while still having the assurance of interoperability. Regulators have also been strong supporters of these standards, because they ensure the overall health of the supplier ecosystem by lowering the barriers to entry.

One issue which has often been raised, particularly in the context of Open RAN designs, is whether or not integrating systems from multiple suppliers to build a network creates a burden on the operator. Our experience so far is that integration shouldn’t be underestimated, and it currently requires a technical authority to convene working sessions between multiple suppliers. As the market segment matures further, this requirement should abate. However, in the interim, note that the recently launched AWS Private 5G allows private network users to delegate this responsibility to AWS.

Further work

One of the most interesting pieces of work yet to be completed is comparing running the 5G Core in the Region with running it on an Outposts server that’s physically in the lab. By avoiding the network round trip time to the Region and back, users and applications should be able to experience lower packet delay, which may be important for certain control systems. However, the packet core handling of user plane traffic will be responsible for some of the delay. Understanding the origin of packet delay will be important for matching network designs to real-world use cases.

Conclusion

Here at the Customer Experience Center in Munich, pictured above, we integrate new use cases and devices with our private 5G network on a regular basis. Examples include laser etching systems, robotic arms and other industrial IoT systems. If you’re interested in seeing the Experience Center for yourself, then contact your account team to discuss a visit. You can also explore more about cloudified private and public 5G networks at aws.amazon.com/telecom.