Building Multi-Edge Data Architectures on AWS Wavelength and MongoDB

By Robert Belson, Developer Advocate – AWS
By Steve Dalby, Director, Industry Solutions – MongoDB

MongoDB

With the advent of high-speed 5G networks, enterprises have sought to deliver low-latency experiences across industrial Internet of Things IoT (IIoT), media and entertainment, automotive, and beyond.

At the core of these low latency experiences is data—whether it be a terrestrial robot, consumer smartphone, or autonomous car. The seamless exchange of data from 5G producer to consumer, such as machine learning (ML) or even generative artificial intelligence (AI) technology, is critical to powering these new experiences.

Ensuring that required data is made available to consumers with the lowest possible latency in any given location (for example via multiple AWS Wavelength zones), while also ensuring there is a single view of the entire dataset, is no easy task.

Since 2020, MongoDB and Amazon Web Services (AWS) have worked together to define reference patterns for low-latency, distributed database patterns across AWS hybrid edge technologies. In particular, this post builds on an earlier demonstration of MongoDB Realm on AWS Wavelength by amplifying the existing architecture with three new major features:

Edge-to-cloud replication: Edge-to-cloud replication in a multi-edge environment is enabled via Atlas Device Sync.
Dynamic edge discovery: Atlas Functions implementation directs mobile clients to the optimal instance of MongoDB Realm running at the edge.
Containerized application: Amazon Elastic Kubernetes Service (Amazon EKS) native deployment exposes the MongoDB Realm database directly to the carrier network.

We will also demonstrate in this post how to use the AWS Load Balancer Controller to deploy an Application Load Balancer (ALB) in front of the database.

MongoDB is an AWS Data and Analytics Competency Partner and AWS Marketplace Seller. A developer data platform company, MongoDB empowers innovators to unleash the power of software and data.

AWS Wavelength Use Cases for MongoDB

From smart manufacturing and preventive maintenance to agriculture, there are no shortages of low-latency applications that benefit from mobile edge computing.

There are a few major benefits to extending existing workloads on MongoDB to the edge:

Security: Today, responses from database queries must traverse—encrypted or otherwise—the public internet to return back to the mobile client. To protect highly-sensitive workloads from the vast majority of malicious actors, mobile edge computing offers the opportunity for all mobile traffic to never egress the radio access network (RAN) to the public internet.
Performance and reliability: Minimizing the incremental hops to the application endpoint(s), particularly in moments of high congestion, can significantly decrease the latency budget of the application. Moreover, for high-volume, transaction-based applications, a mere 10ms in savings compounded over one million transactions could result in over two hours of latency savings.

AWS Wavelength Reference Architecture

To architect this solution, we will need to extend the AWS region to an AWS Wavelength Zone of choice, where we’ll deploy a containerized Realm application to communicate back with MongoDB Atlas in the parent region.

Figure 1 – AWS Wavelength and MongoDB architecture.

Amazon VPC

For each region in which we have a running MongoDB Atlas cluster or AWS Wavelength zone application, we must have a corresponding virtual private cloud (VPC).

In scenarios where a high number of Amazon VPCs are deployed, or multiple carriers’ Wavelength Zones (Verizon, Vodafone, Bell Canada) are leveraged, configure VPC peering and ensure that bi-directional peering is configured.

You can find the complete list of regions with available Wavelength zones can be found on the AWS Wavelength locations page.

MongoDB Atlas

Using MongoDB Atlas, a fully managed database offering, we can instantiate a MongoDB Atlas cluster in any region of choice. Supported Atlas regions can be found in the MongoDB Atlas documentation.

MongoDB Realm

In a previous post, we deployed self-managed instances of MongoDB running on Amazon Elastic Compute Cloud (Amazon EC2) instances and managed by MongoDB Cloud Manager. However, in a multi-Wavelength zone environment, the complexity of synchronizing data between those zones increases. This complexity can be addressed by the introduction of MongoDB Realm and Atlas Device Sync, which automates the synchronization of data between Realm-based endpoints and Atlas.

MongoDB Atlas Functions

To determine the optimal edge endpoint, we can utilize an edge discovery APIs, such as the Verizon 5G Edge Discovery Service, in a thin-client model to offload north-south service discovery to the application, rather than the device.

Amazon EKS

The ability to orchestrate applications from a single, centralized control plane reduces the management overhead of configuring multiple standalone clusters or auto scaling groups. For the edge compute database, we’ll use Amazon EKS to create a highly-scalable environment to run multi-edge workloads.

Within the EKS cluster, we’ll deploy containers including an application, such as an MQTT broker, alongside the MongoDB Realm database.

To learn more about Amazon EKS on AWS Wavelength, see the AWS blog post on deploying geo-distributed EKS clusters on AWS Wavelength.

Solution Walkthrough

In this section, we’ll provide a step-by-step guide to deploy each of the relevant components—starting first with the edge discovery service registry, core infrastructure in AWS, followed by the MongoDB cluster and containerized application.

Figure 2 – MongoDB solution overview.

Step 1: Create Edge Discovery Automation

To start, we need a way to ensure our edge discovery service mesh is always up to date with the latest carrier-facing endpoints. Thus, prior to launching any edge infrastructure, we must configure an automated workflow that dynamically registers AWS Wavelength endpoint. This can be done via Amazon EventBridge and AWS Lambda and is described in this AWS blog post.

Step 2: Create Atlas Cluster

Next, create the parent region database on AWS. Using MongoDB Atlas, MongoDB will maintain the high-availability of the cluster in the region you select.

Step 3: Create Amazon EKS Cluster

To deploy your AWS Wavelength application, you will first need the parent region resources instantiated. As an example, the Amazon EKS cluster must be launched with two subnets in the parent region (for example, us-east-1a, us-east-1b).

Step 4: Create Realm App

For our edge application, you can use MongoDB Atlas App Services to create an IoT application which contains the application code for your static web page and CSS files. As a sample application, you can follow the directions in this MongoDB blog post to take advantage of low-latency innovation with MongoDB Atlas, Realm, and AWS Wavelength.

In parallel, for your dynamic application logic, create a container image that writes to a Realm database instance within the container itself; as an example, you can visit this MQTT proxy to Realm on GitHub. Leveraging this design allows us to decouple the latency-tolerant experiences, such as the application user interface, from the latency-critical data plane of IoT messages.

Step 5: Push Realm App

Next, we need to configure the MongoDB Realm application with the application code. To do so, we’ll push the remote configuration (from our pre-developed application) as we did in this AWS blog post you can follow along with.

Next, we need to configure our Atlas Function, a serverless event-driven function that invokes our edge discovery service to route to the optimal MongoDB Realm endpoint.

In this function, we extract the calling IP address so we can invoke the Edge Discovery Service (EDS) API on the mobile client’s behalf. Next, we authenticate to the edge discovery service and query the optimal serviceEndpoint records which returns the optimal Carrier IP address corresponding to the closest Wavelength Zone (mecresponse.data.serviceEndpoints[0].serviceEndpoint.IPv4Address).

Please note the EDS keys, appKey, and secretKey should be treated as sensitive and outside the code, such as using AWS Secrets Manager.

exports = async function({ query, headers, body}, response) {
    const ipAddr = headers["X-Forwarded-For"][0];
    console.log("Using IP: " + ipAddr);  
    try {
      var tokenresponse = await axios.request(
        { 
          url: "https://5gedge.verizon.com/api/ts/v1/oauth2/token",
          method: 'post',
          headers: {"Content-Type": "application/x-www-form-urlencoded"},
          data: qs.stringify({"grant_type" : "client_credentials"}),
          auth: {"username": appKey, "password": secretKey}
        }
      );
    } catch(ex) {
      console.log(ex);
    } 
    try {
      var mecresponse = await axios.request(
        { 
          url: "https://5gedge.verizon.com/api/mec/eds/serviceendpoints?serviceEndpointsIds="+serviceEndpointsId+"&UEIdentityType=IPAddress&UEIdentity="+ipAddr,
          method: 'get',
          headers: {"Content-Type": "application/x-www-form-urlencoded", "Authorization": "Bearer "+tokenresponse.data.access_token},
          data: qs.stringify({"grant_type" : "client_credentials"})
        }
      );
      
    return mecresponse.data.serviceEndpoints[0].serviceEndpoint.IPv4Address; 
    } catch(ex) {
      console.log(ex);
    }
};

Step 6: Launch Self-Managed Nodes

To complete the deployment of the EKS cluster, deploy self-managed worker nodes to the AWS Wavelength Zone(s) of your choice. To learn more about configuring self-managed nodes in Wavelength, check out the AWS documentation which covers how to use eksctl for deployment in Wavelength zones.

Step 7: Deploy Realm Apps

Next, we create a deployment manifest, realm.yaml, which includes a Deployment and ClusterIP service objects.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: realm-database
  labels:
    app: realm-iot
spec:
  replicas: 1
  selector:
    matchLabels:
      app: realm-iot
  template:
    metadata:
      labels:
        app: realm-iot
    spec:
      containers:
      - name: realm-app-demo
        image: <your-realm-app-image>
        ports:
        - containerPort: 80    
      nodeSelector:
        failure-domain.beta.kubernetes.io/zone: <your-wavelength-zone-ID>

---
apiVersion: v1
kind: Service
metadata:
  name: realm-service
spec:
  selector:
    app: realm-iot
  ports:
    - port: 80
      targetPort: 80

Next, we can configure the AWS Load Balancer Controller to provision an Application Load Balancer when you create an ingress object. See this AWS blog post to learn more about configuring ALBs in Wavelength zones.

Conclusion

To learn more about MongoDB and AWS Wavelength, you can review a hands-on workshop from AWS re:Invent or browse use cases, reference architectures, and whitepapers on MongoDB at the network edge:

Realizing the Benefits of 5G with MongoDB and Verizon 5G Edge: In this whitepaper, learn how the speed, throughput, reliability, and responsiveness of the Verizon network, paired with the sophistication of the next generation MongoDB developer data platform, is poised to transform a variety of industries.
Delivering Data Faster with Verizon 5G Edge and MongoDB: Learn the basics of mobile edge computing, edge discovery API workflow, and the MongoDB Atlas architecture.
Computing in the Real World: Explore the industry use cases for mobile edge computing (MEC) across industrial IoT, smart manufacturing, automotive, and beyond.

You can also learn more about MongoDB in AWS Marketplace.

.

.

MongoDB – AWS Partner Spotlight

MongoDB is an AWS Data and Analytics Competency Partner and developer data platform company that empowers innovators to unleash the power of software and data.

Contact MongoDB | Partner Overview | AWS Marketplace | Case Studies