Managing edge-aware Service Mesh with Amazon EKS for AWS Local Zones

Introduction

In a previous post, Deploy geo-distributed Amazon EKS cluster on AWS Wavelength, we introduced how to extend Amazon Elastic Kubernetes Service (Amazon EKS) clusters closer to end-users for 5G network-connected applications. However, this exact pattern of using self-managed node groups also applies to AWS Local Zones, an infrastructure solution that brings AWS compute and storage to metropolitan centers. Across both AWS Wavelength and AWS Local Zones, one of the pieces of collective feedback from our customers was to simplify east-west traffic communication – irrespective of the customer’s selection of nodes.

To build on the foundations of Amazon EKS at the edge, this post provides a reference implementation of this multi-edge deployment pattern and how service mesh technology can be used to abstract away the complexity of the edge itself. This post demonstrates how an Amazon EKS cluster’s data plane – spanning across AWS Local Zones, AWS Wavelength Zones, and the parent AWS Region – can be used with a service mesh to simplify edge aware routing.

Hub-and-spoke networking at the edge

With the advent of AWS Local Zones and AWS Wavelength Zones together, developers have more choices than ever for geographies where lower latency access can be introduced. Even though Wavelength Zones are embedded directly within Telecommunications’ providers 5G networks and Local Zones are internet-accessible, both AWS Edge Computing services share key networking characteristics that prevent inter-zone VPC connectivity. AWS Wavelength and AWS Local Zones operate in a hub-and-spoke architecture where all spokes can only communicate within their zone itself or back to the hub (i.e., Parent Region). This introduces challenges for developers looking to take advantage of low-latency geo-distributed applications at scale.

To illustrate this challenge with an example, consider a two-tier application deployed across three Local Zones may encounter the following scenario:

• Traffic originating from the web service in Local Zone 3 may seek to connect to the app Within the cluster, Kubernetes domain name system DNS may see five pods corresponding to this service across three Local Zones (LZs) and two Availability Zones (AZs).
• While DNS could direct this traffic flow to the app pod running in Local Zone 3, resulting in the lowest latency, DNS could also try to direct this traffic to app pods on Local Zone 1 or Local Zone 2, which would result in an unreachable endpoint. Alternatively, DNS could try to direct the traffic to an app pod in the AWS Region (e.g., AZ1 or AZ2). While this is possible, the traffic would incur a latency penalty to traverse over the service link from the Local Zone to the parent AWS region.

As a result, workload placement is incredibly important for applications with low latency requirements to ensure that all latency-sensitive components are scheduled to each edge zone, as desired by the customer. To illustrate the impact of workload placement in a geo-distributed setting, consider the architecture of an automotive company wishing to reduce the latency for a machine learning-based workflow for automatic pedestrian detection or vehicle-to-anything (V2X) connectivity. Perhaps the two-tier web app consists of an Application Programming Interface (API) proxy and machine learning inference service, each geo distributed across every available AZ and LZ to create uniform, low latency access for vehicles within a given AWS Region.

Beyond workload placement, traffic routing is an equally important design consideration. Today, customers seeking to deploy that same two-tier web app above might need 10 manifest files: one for each instance of the service to ensure that each resource is scheduled appropriately to the right edge location or AZ. Secondarily, development teams would need specialized config files that deterministically point each service to the next logical service (e.g., “direct traffic from API service in Local Zone 2 to ML inference service in Local Zone 2“).

Customers asked for a Kubernetes native reference pattern that could introduce edge-awareness to the cluster. This edge-awareness could understand this unique hub-and-spoke design without causing developers to re-architect their entire application.

As one such solution, we’ll demonstrate how service mesh solutions can provide control of east-west traffic — across an arbitrary number of Local Zones or Wavelength Zones – without making any changes to the application itself.

Solution overview

To illustrate a more realistic microservices environment, consider the reference architecture of HashiCups, HashiCorp’s demo e-Commerce application.

• Frontend: This is a service we’ll expose in each Availability Zone and Local zones. That means that, if there are two Local Zones and two Availability Zones, we’ll need at least four replicas.
• Public API and Product API: Just like the frontend service, we’ll likely want at least one replica scheduled to each node for high availability.
• Postgres: For the database itself, we may only need the data source sporadically and, thus, we can keep in the region with a primary and single read replica.

So how can we schedule the workload easily and ensure that, as an example, the product-api service in Local Zone 1 never routes to the public API service in Local Zone 2?

Namespace control and zone-based routing

Now that a desired deployment is selected, how could Kubernetes ensure cross-LZ traffic isn’t attempted?

One approach is Kubernetes namespaces and node selectors. Namespaces create logical isolation of your deployments and allow DNS resolution to natively route services by the service-name (e.g., public-api) without the fully qualified domain name (FQDN) such as public-api.my-namespace.svc.cluster.local, since DNS queries that don’t specify a namespace are limited to the Pod’s namespace. This is a great solution in cases where all four microservices are scoped within a namespace. In our architecture, if we had 3 namespaces (region-namespace, edge1-namespace, edge2-namespace) each with a copy of the four microservices, all of the routing could be handled natively by Kubernetes DNS (CoreDNS).

The second we introduce complexity by way of a single postgres deployment within the region. As an example, this approach isn’t viable. When a product-api pod seeks to connect to the postgres data referencing the namespace-agnostic FQDN (e.g., postgres rather than postgres.namespace.svc.cluster.local), it connects to CoreDNS seeking to resolve postgres and receives nothing because no such postgres workload exists in that namespace. In these types of cases, an additional routing overlay is needed to manage complex rules and scenarios based on the given service or namespace.

Service Mesh at the edge

While developers can use a variety of service mesh technologies with Amazon EKS, such as AWS App Mesh or HashiCorp Consul. HashiCorp Consul is a solution to automate network configurations, discover services, and enable secure connectivity. You can use HashiCorp Consul’s service mesh to ensure that traffic is appropriately routed to the appropriate Availability Zone, Local Zone, or Wavelength at any given point.

By mirroring Kubernetes namespaces to a Consul namespaces, each service routes to its closest upstream service in the same Consul namespace and you can borrow from existing native Kubernetes benefits (90%+ of routing decisions) while adding additional routes of interests.

apiVersion: consul.hashicorp.com/v1alpha1
kind: ServiceResolver
metadata:
  name: postgres
spec:
  redirect:
    service: postgres
    namespace: demo-app-region

In this example config, a ServiceResolver is defined that routes all traffic destined for postgres specifically to the service available in the ‘demo-app-region’ namespace. This same configuration manifest needs no adjustments for each namespace and, thus, is designed to scale to any number of Local Zones. In this way, service resolvers allow Consul namespaces to resolve to shared services in the parent region.

Edge failover to region

In the event of a node in either Local Zones rebooting, application deployment error or human misconfiguration, what would happen if one of the services scheduled to the edge (e.g., product-api)were to temporarily disappear?

Surely enough, if you were to visit your application, you would see an error because Kubernetes has no native, deterministic way to failover to a replica of product-api in another namespace (and topology) if that in the current namespace is unavailable. What’s happening here is that, either Kubernetes has no knowledge of a product-api replica in another namespace or would round-robin you to a replica in a different Local Zone, which is blocked.

This is where the ServiceResolver, a CustomResourceDefinition (CRD) offered by HashiCorp Consul comes back in. Let’s create a file, failover-resolver.yaml with the following:

cat > postgres-resolver.yaml <<EOF
apiVersion: consul.hashicorp.com/v1alpha1
kind: ServiceResolver
metadata:
  name: product-api
spec:
  failover:
    "*":
      service: product-api
      namespace: demo-app-region
EOF

kubectl apply -f failover-resolver.yaml -n edge1-namespace

By configuring this failover rule, the service mesh explicitly routes traffic to the product-api service in the AWS Region namespace if the product-api service in the Local Zone were to become unavailable. In fact, these changes are edge-agnostic. Once you create the manifest, you can apply it as many times as needed with dynamic failover. And remember, this cares for the worst-case scenario. Most of the time, this additional configuration won’t even be used, as the public-api service in a given Local Zone connects to the product-api service within that same Local Zone.

Conclusion

In this post, we showed how Amazon EKS clusters deployed across multiple AWS Local Zones can be used with a service mesh to simplify edge aware routing. While Consul Service Mesh addresses a critical routing limitation by geo-distributed hub-and-spoke architectures, there remains a visible customer pain point around geo-distributed deployments. In Kubernetes, it’s easy to deploy any number of replicas for a given service but edge-aware replicas are not natively supported. Today, it would be the developer’s responsibility to create and maintain taints, affinities, and tolerations for each service, which is burdensome as the number of Local Zones and Wavelength Zones scale.

As a direct result of this Consul integration, a much stronger case to developers can be made why edge applications can be developed from the start with at least two Edge Zones at any given point. By abstracting away the complexities of the network itself, developers can focus on building their applications, while we reduce the undifferentiated heavy lifting of building at the network edge.

By introducing edge-awareness into edge applications, we can extend the AWS Well-Architected Framework to the edge in the following ways:

• Edge-awareness enhances reliability: by introducing incremental failure domains either via new Availability Zones within the carrier network (e.g., AWS Wavelength) or metro edge (e.g., AWS Local Zones), customers have additional choice for where or how to create failover strategies
• Edge-awareness reinforces security posture: as the incremental or cumulative hops through the public internet decreases (particularly north-south traffic from end-device to cloud), the security posture of customer workloads increases
• Edge-awareness drives performance efficiency: beyond scheduling workloads in an edge-aware way (i.e., in a microservices environment, which services must live on the edge vs. the region), the routing of inter-cluster traffic (i.e., east-west traffic) can be done in an edge-aware way to find the lowest-latency path

To learn more about edge-awareness for AWS Edge Computing environments, visit Architecting Geo-Distributed Mobile Edge Application With Consul or the Demo application for using Consul on GitHub.