Approach to migrate Spring Cloud microservices applications to Amazon EKS

In this blog, we will look into how enterprises can approach migrating on-prem Spring Cloud microservices to Amazon Elastic Kubernetes Service (Amazon EKS). With managed service offerings from AWS, developers can eliminate the need to run and manage cross-cutting services like Service Registry, Config Server and API Gateway. By using AWS services, developers can focus on building business code to accelerate release velocity and maximize business value.

Problem Statement

Many enterprises that build microservices in on-prem data centers leverage Spring Cloud libraries. Spring Cloud provides tools for developers to quickly build some of the common patterns in distributed systems (e.g., Configuration Management, Service Discovery, Circuit Breakers, Intelligent Routing, Micro-proxy, Leadership Election). Coordination of distributed systems leads to boiler plate patterns. Using Spring Cloud allows developers to quickly stand-up services and applications that implement those patterns. They work well in a distributed environment, including the developer’s laptop and bare metal data centers.

The problem with enterprise on-prem deployment is the requirement to manage Config Server, Service Registry, Spring Cloud Gateway services, and microservices applications. This takes away precious time from developers to deliver business outcomes and slows down the release velocity. The Spring Cloud Services deployment model has limitations on how you can register, discover, and optimize the resource utilization of the services.

Solution Overview

This solution will cover how to replace Spring Cloud module capabilities within a microservices application using Amazon EKS and other AWS services. The architecture will implement resiliency using AWS App Mesh. Monitoring will be leveraged using AWS X-Ray, and Amazon OpenSearch Services. Amazon API Gateway will be used as an API Gateway solution to manage the APIs.

Basic Spring Cloud Architecture

Let’s look at a fictional pet clinic application that is composed of microservices using Spring Cloud. Components include:

Customers Service – Lists and manages all the customers and their details
Vets Service – Lists and manages the vets
Visits Service – Manages visits by customers for their pets with a vet

All the incoming user traffic is routed through the Spring Cloud API Gateway for request routing, throttling, and security enforcement. Microservices register with the service registry implemented with Spring Cloud Netflix for discoverability. Microservices configuration is externalized by using a Spring Cloud Config server backed by a Git repository. Service invocation telemetry data is generated and ingested into the Zipkin distributed logging service. All the microservices and cross-cutting services are deployed on servers in the data center.

Figure 1: This is a sample microservices built with Spring Cloud. Spring Cloud Gateway as an API Gateway, Spring Cloud Config to externalized the configurations, Zipkin for tracing services and Spring Cloud Netflix for discovering the services..

Basic Amazon EKS Architecture

For enterprises to migrate microservices to the AWS Cloud, AWS provides service offerings that will assist in the process. When an application is migrated to Amazon EKS, each microservice is deployed as a container running in a pod inside a Kubernetes cluster. By deploying to Amazon EKS, AWS manages Kubernetes control plane. Incoming traffic is routed through Amazon API Gateway, which is responsible for API management. Microservices are registered with the Kubernetes service registry. Microservices are exposed using an Amazon Elastic Load Balancer (ELB) to balance incoming traffic. AWS App Mesh is responsible for implementing some of the cross-cutting concerns, such as circuit breaker and failover. Amazon CloudWatch provides centralized logging, and AWS X-ray covers distributed tracing of the requests to identify performance issues.

Figure 2: The sample application is deployed on Amazon EKS. Spring Cloud components are replaced with AWS Services.

Benefits of migrating to Amazon EKS:

Amazon EKS is a fully managed Kubernetes service that eliminates the operational burden of installation, configuration, and management of the Kubernetes platform. Benefits include:

Configuration management and service discovery features
Self-healing and auto-scaling capabilities
Out of the box support for multiple programming languages
Resiliency and high availability can be configured with the application deployment configuration.
A rich ecosystem of open tooling to support innovation.
Amazon EKS applications can be deployed to any conformed Kubernetes platform, providing portability.

Next let’s review how and which AWS services provide the features that Spring Cloud Services offer.

Comparison Matrix:

Capability	Spring Cloud	AWS
API Management	Spring Cloud Gateway	Amazon API Gateway
Externalize Configuration	Spring Config Server	Amazon EKS and AWS Secrets Manager
Service Registry and Discovery	Eureka	External DNS with Amazon Route 53, AWS Cloud Map
Circuit Breaker for Resiliency	Netflix	AWS App Mesh
Distributed Tracing	Sleuth	AWS X-ray, Amazon OpenSearch Service

API Management:

Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. API Gateway supports containerized and serverless workloads, as well as web applications.

Through the API Gateway console, you can define the REST API and its associated resources and methods, manage the API lifecycle, generate client SDKs, and view API metrics. You can also use the API Gateway console to define your API usage plans, manage developer API keys, and configure throttling and quota limits. All of the same actions are available through the API Gateway APIs.

Current Spring application APIs are defined as Swagger API definitions. You can use the open source Swagger importer tool to import your Swagger API definitions into API Gateway. With the Swagger importer tool, you can create and deploy new APIs as well as update existing ones.

API Gateway logs API calls, latency, and error rates to Amazon CloudWatch. The metrics are also available through the API Gateway console in a REST API dashboard. API Gateway meters utilization by third-party developers; the data is available in the API Gateway console and through the APIs. You can use the CloudWatch console to set up custom alarms. By default, API Gateway monitors traffic at a REST API level. Optionally, you can enable detailed metrics for each method in your REST API from the deployment configuration APIs or console screen. Detailed metrics are also logged to CloudWatch and are charged at the CloudWatch rates.

Externalize Configuration:

Application configuration is managed through name-value pairs like API keys and DB connection details. Applications reference configuration through application.yaml or properties files with Spring Cloud Config Service in a source code repository.

Amazon EKS provides the capability to manage the configuration as ConfigMaps and Secrets. ConfigMaps are used to externalize the configuration of any name-value pairs. Secrets are used to save sensitive data like DB details and API keys. Amazon EKS provides mechanisms to mount configurations and secrets to the containers file system. This enables an application to ingest configuration without knowledge of where it is running.

Application configuration is defined and managed as a YAML file in the source code repository of choice. This configuration can be versioned and reviewed before it is applied to a running cluster for security, compliance and audit purposes. DevOps practices are a popular way of automating this process to apply the latest configuration to a running cluster. Kubernetes provides a command-line interface (kubectl) to interface with the cluster. You can apply the configuration using the following commands:

apiVersion: v1
kind: ConfigMap
metadata:
  name: sample-app-config
  namespace: default
data:
  name: sample-app
  pool.size.core: 1
  pool.size.max: 16

kubectl apply -f ./sample-app-configmap.yaml

ConfigMap can also be created with the existing application properties file. The following command allows you to generate ConfigMap from the properties file:

kubectl create configmap sample-app-config-env-file \ --from-env-file=configure-pod-container/configmap/sample-app-env-file.properties

Starting with Kubernetes v1.23, kubectl command supports the –from-env-file argument to be specified multiple times when creating a ConfigMap from multiple data sources. Use the following command to load multiple configuration properties files in a single execution:

kubectl create configmap config-multi-env-files \ --from-env-file=configure-pod-container/configmap/sample-app-env-file.properties \ --from-env-file=configure-pod-container/configmap/sample-app-ui-env-file.properties

The Spring Cloud Kubernetes library enables loading the configuration with minimum to no changes to your application. By adding the spring-cloud-starter-kubernetes-config library to pom.xml, configuration is loaded from ConfigMaps and Secrets. When the configuration is changed, spring cloud Kubernetes reloads the changed configuration. Example:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-kubernetes-config</artifactId>
</dependency>

Please refer to Spring Kubernetes documentation for further implementation details.

Service Registry and Discovery:

Microservices discovery is implemented as service discovery where microservice is defined with an alias name and invoked by looking them up in a service registry. Spring Cloud provides this capability by offering the Service Registry Service.

Amazon EKS provides the ability to look up and invoke a service based on the service name defined during the creation time by configuring the DNS Server. Amazon EKS provides options to configure the external DNS server with either Route 53 or AWS Cloud Map. Please refer to the documentation Setting up ExternalDNS for Services on AWS and Setting up ExternalDNS using AWS Cloud Map API for more details.

Using native Kubernetes service discovery ensures compatibility with additional tooling, such as AWS App Mesh. App Mesh is capable of load balancing, circuit breaker, failover, and much more.

The caller service then need only refer to names resolvable in a particular Kubernetes cluster. A simple implementation might use a spring RestTemplate that refers to a fully qualified domain name (FQDN), such as https://{service-name}.{namespace}.svc.{cluster}.local:{service-port}.

Circuit Breaker for Resiliency:

Microservices have backing services that can either be a database, messaging service, or another microservice. When invoking the backing services to avoid service degradation and cascading failures, resiliency is provided through a circuit breaker and retry implementation.

App Mesh in conjunction with Amazon EKS allows developers to implement resiliency at the cluster level. There are multiple App Mesh features in addition to resiliency. Let’s focus on circuit breaker and retry implementations in this section.

App Mesh supports the circuit breaker pattern to protect a downstream service from being invoked by the upstream service during a partial or complete downtime. App Mesh provides a default connection pool configuration and leverages outlier detection functionality to implement circuit breaker capabilities. Using the circuit breaker pattern creates applications that are resilient to connection errors.

By using the connection pool configuration for your service mesh, you can limit the number of simultaneous connections or requests to your application endpoints. You can apply the connection pool settings on a virtual gateway with the following code: Additional details in the documentation

“connectionPool”: {
    “http”: {
        “maxConnections”: 100,
        “maxPendingRequests”: 10000
    }
}

The outlier detection feature allows a service to track the health status of individual hosts in each of its upstream services. It will temporarily stop routing traffic to hosts that exhibit elevated errors.

{
   "virtualNodeName": "sample-node",
   "spec": {
      "listeners": [
         {
            .....
            },
            "outlierDetection": {
               "maxServerErrors": 5,
               "maxEjectionPercent": 100,
               "interval": {
                  "value": 30,
                  "unit": "s"
               },
               "baseEjectionDuration": {
                  "value": 10,
                  "unit": "s"
               }
            }
         }
      ]
}

The connection pool configuration and outlier detection together allow you to limit the impact of failures, latency spikes, and network fluctuations in your application.

The retry mechanism provides resiliency to recover from transient issues. It helps to overcome short lived network blips or short-term interruptions on the server services. You can add the ability to retry connections between services using App Mesh based on HTTP error codes.

"httpRoute": {
    "action": {
        "weightedTargets": [
                {
                    "virtualNode": “sample—node",
                    "weight": 1
                }
       ]
    },
    "match": {
        "prefix": "/"
   },
    "retryPolicy": {
        "maxRetries": 10,
        "perRetryTimeout": {
            "unit": "s",
            "value": 2
        },
        "httpRetryEvents": ["server-error"]
        } 
    },
 "priority": 1
 }

Distributed Tracing:

Microservices need the ability to capture logs and metrics to provide visibility into the performance of the services and pinpoint bottlenecks.

AWS X-Ray provides application-tracing functionality, giving deep insights into all microservices deployed. X-Ray integration with Amazon EKS can be enabled by configuring the X-Ray daemon. Applications can then leverage the X-Ray SDK to send trace data to the daemon. Trace data includes information about incoming HTTP requests served by the application, and calls made to downstream services by the application. Additional information in the Application Tracing on Kubernetes with AWS X-Ray blog.

Alternatively, you can continue to leverage Spring Cloud Sleuth or Micrometer tracing libraries to generate traces and spans from the application.

The Amazon OpenSearch Service is used for performing log analytics and application monitoring. You can also use Trace Analytics, which is part of the OpenSearch observability plugin for application-tracing .

Key Considerations

Microservices architectures need the ability to implement mutual TLS and automatically generate, rotate and renew certificates. AWS Certificate Manager (ACM) makes it easy to manage certificates for microservices applications.

Microservices with backing service access keys and passwords should be managed in a secure vault for security reasons. AWS Secrets Manager centrally manages the lifecycle of all the secrets and makes it easy to integrate with applications deployed to Amazon EKS Clusters. A key requirement for some enterprise applications is to rotate the secrets on a periodic basis to comply with regulatory demands. Secrets Manger automates the rotation of secrets based on the policies defined.

For Spring Boot applications deployed to Amazon EKS, the applications should be containerized to manage the lifecycle of the container dependencies.

AWS App2Container(A2C) is a command line tool for migrating and modernizing Java and .NET web applications into container format. AWS A2C analyzes and builds an inventory of applications running in bare metal, virtual machines, Amazon Elastic Compute Cloud (EC2) instances, or in the cloud.

App2Container generates Kubernetes resource files for the containerized application following AWS best practices. AWS A2C creates CI/CD pipelines for Amazon DevOps services, which you can customize to integrate with your existing CI/CD workflow.

Amazon Elastic Container Registry (Amazon ECR) is a fully managed container registry offering high-performance hosting, so you can reliably deploy application images and artifacts anywhere. Amazon ECR scanning helps in identifying software vulnerabilities in images to comply with your security requirements.

Conclusion

The migration of microservices from Spring Cloud to Amazon EKS eliminates the operational burden of managing Kubernetes, while allowing you to take advantage of Kubernetes rich opensource tooling. Integration with AWS App Mesh benefits managing the migration in a phased manner. AWS X-ray helps you trace user requests and identify bottlenecks to improve application performance. All of these services together afford developers time to focus on delivering business features, which in turn leads to a higher release velocity.

Resources:

How to get started: Learn about Amazon EKS and integration with AWS services through a set of hands-on exercises.

Explore the EKS Best practices guides to effectively manage Amazon EKS Clusters.

About the authors:

AWS Cloud Operations Blog