An Optimized Approach to Modernize an Integration Platform on AWS

By Dipanjan Sengupta, Sr. Director, Technology – Cognizant
By Samriddha Chatterjee, Architect – Cognizant
By Pratip Bagchi, Partner Solutions Architect – AWS

Cognizant

Enterprise customers are modernizing existing workloads to the cloud for various reasons. Cost optimization has always been a conversation starter, as that leads to application modernization and operational excellence, which brings agility to application development and enables faster go-to-market.

The evolution of cloud service providers has helped modern applications achieve fluctuating and high demands of infrastructure, software-defined networking and storage, and seamless integration with managed services.

The case study highlighted in this post describes how to bridge the gap between infrastructure-as-a-service and integration-platform-as-a-service models for implementing an API.

This solution uses compute, infra, and managed services provided by Amazon Web Services (AWS), as well as an implementation model that uses declarative configurations, reusable components, and code generation for accelerated and uniform service delivery.

Cognizant’s customer in this scenario is one of the world’s most-used business credit report providers, based in Europe and serving 365 million businesses worldwide.

Currently, services to consumers of credit information are provided over access-controlled RESTful APIs hosted on a specialized platform. This allows consumers to manage their master data with up-to-date business information, accelerate and streamline customer and supplier onboarding and qualification processes, and get alerted of credit rating changes.

Cognizant is an AWS Premier Consulting Partner with six AWS Competency designations. Cognizant is also a member of the AWS Managed Service Provider (MSP) and AWS Well-Architected Partner Programs.

Existing Architecture

The customer’s existing API design, implementation, and delivery were accomplished using integration software from an integration solution provider and was hosted on their platform, offered as a service.

Broadly, APIs (services) in the API-led connectivity approach are categorized into three distinct layers:

• System APIs: Typically accesses the core system of records (SOR) and provide a means of insulating the user from the complexities of, or any changes to, the underlying systems.
• Process APIs: Interacts with and shape data within a single system or across multiple systems by integrating with one or more System APIs.
• Experience APIs: Renders data in a way that is easily consumed by its intended consumer.

Following are key characteristics of services in this customer scenario:

• Services in all three categories are grouped by functional domains or business areas, and are independently consumed by external clients (invoking the Experience APIs, which are typically available over the internet) and other enterprise applications (invoking the Process and System APIs).
• Service endpoints (distinct methods or operations within a service) typically implement a synchronous request-response message exchange pattern.
• Individual endpoints implement an array of tasks (namely validation, routing transformation, and data aggregation) defined by subflows, an integration solution-specific construct written using a domain-specific language (DSL).
• Existing integration use cases predominantly include RESTful and Simple Object Access Protocol (SOAP) service invocations (invoking external service endpoints), database interactions, and object store service interactions.

Implementation Challenges

The decision to migrate to AWS was made by Cognizant’s customer and driven by the need to scale out with growing demand while reducing operating expenses.

The implementation challenges to overcome arose primarily from the fact the team was migrating integration applications from an integration-platform-as-a-service provider to a scalable platform that provides infrastructure-as-a-service and a vast array of managed services to choose from to design the right solution.

Integration platforms typically support a set of components and constructs to define and specify integrations flows in a DSL that is unique to the platform.

The applications in this modernization scope were no exception:

• Cognizant needed a comprehensive configuration-driven integration flow builder (equivalent to the flow and subflow constructs of integration solutions) and a common endpoint definition and specification model. The interaction of endpoints across different layers represents the abstraction of the integration flow. This solution required a highly available and reliable store for persisting the individual endpoint model documents with minimum latency in loading it for execution.
• Cognizant also needed to adopt a development approach with support for a uniform interaction pattern between the application layers, a set of reusable components for cross-cutting tasks (like validation, transformation, or routing) and a common interface for integration over different protocols (with SOAP and RESTful endpoints, databases, and object stores), a viable equivalent of prebuilt connectors of integration solutions.

Given the large number of interfaces in all three layers and their corresponding implementations to be rewritten, Cognizant needed to come up with a modernization strategy that adopted the fail-fast philosophy.

The time to market was governed by the existing software provider’s license renewal deadline; hence, the developers needed all possible tools and techniques to accelerate the development lifecycle.

With a wide array of managed and infrastructure services to choose from for compute, network, and storage, designing an elastic, highly-available, cost-effective, and scalable solution architecture was a critical success factor for the modernization project.

Modernization Strategy

To realize an end-to-end business function, invocation of an Experience API endpoint by a client typically entails, among other tasks, downstream invocations of one or more Process (and in some cases System) API endpoints, which in turn, invokes one or more System API endpoints.

A complete request-response roundtrip across all of the layers, including the interactions with the core system of records, for the realization of a particular business function is what Cognizant calls a use case.

Figure 1 – Use case diagram.

Although the services in the Experience, Process, and System layers are designed, developed, and packaged as independent deployable units, they achieve little business functionality in isolation.

Thus, from a modernization standpoint, Cognizant believes a use case-based migration strategy is more prudent than a service-based migration. Following are some of the benefits of this approach:

• Allows testing of migrated use cases. Service-based migration does not allow testing of migrated services in isolation.
• Helps identify the endpoints in a service, which are actually used and need to be migrated. Endpoints in an existing service that are not participating in any use case are not used or deprecated and don’t need to be migrated.
• Enables faster and gradual adoption of the new API delivery platform. This includes phased release of migrated endpoints to help mitigate any challenges of the new platform.
• Adopts the fail-fast design philosophy, which is critical in a time-sensitive modernization project.

In this scenario, Cognizant used Amazon API Gateway as a façade that intercepts requests going to the existing backend system. With such a façade, existing features can be migrated to the new system gradually, while consumers can continue using the same interface, unaware that any migration has taken place. This minimizes the risk of migration and spreads the development effort over time (Strangler Fig pattern).

The endpoint to route a request to is configured through Amazon API Gateway HTTP proxy integration so that, in case of deterioration of any existing quality of service or any other unforeseen issue with the newly implemented endpoint, the requests for that use case can be immediately routed back to the old implementation.

Solution Overview

At a high-level, the target solution Cognizant came up with consisted of the following components.

Endpoint Configuration Model

In this case, an endpoint represents a distinct method or operation within a service. The service can be in any of the three layers: Experience, Process, or System.

Each endpoint has a unique ID, which represents a specific endpoint (typically, a combination of the service name, version, operation or method name, and any other parameters that make the endpoint unique). In addition to the identifier, there are other attributes of interest for an endpoint, especially when it participates in and is an integral part of integration flows across multiple service layers to realize a specific use case.

Cognizant developed an endpoint configuration model, the schema of which comprehensively addresses the use cases they currently have, with the scope for extending it further, if required for other use cases.

The endpoint configuration model includes the following attributes:

Figure 2 – Endpoint configuration model class diagram.

A common endpoint configuration model across all services enables Cognizant to adopt a framework-based approach for implementing the individual use cases. Attributes declared in the configuration of an endpoint are interpreted and used uniformly by a common set of reusable components.

Framework-Driven Development

The sequence diagram below is a generic representation of services in any two random layers interacting with one another to perform their roles and implement their functionality within a given use case:

Figure 3 – Sequence diagram. Click to enlarge.

The sequence starts by loading the configuration of the entry-level endpoint for the given sequence and continues thereafter. This is driven largely by the attributes in the configuration that are executed by a set of common reusable components (shaded in grey in Figure 3).

The parser of the common endpoint configuration model documents and the components that perform common tasks like validation, executing routing decisions, payload transformation, and invoking downstream endpoints in the use case, are part of a framework that’s added as a dependency in all service implementations.

Common data models for reporting errors, auditable log entries, and access control parameters are also part of the common framework.

With the framework doing the bulk of the heavy lifting for the endpoint implementation of any service, any specific functional or business logic that an endpoint may need to implement is confined within a fixed set of classes (namely, the helper classes in Figure 3).

Code Generator

The declarative configurations, reusable framework artifacts, and the API-first development methodology (OpenAPI 3.0) enable Cognizant to generate a large part of the endpoint-specific classes. This includes the controller, corresponding delegator, and helper classes for containing the specific functional or business logic for a specific endpoint and all other downstream endpoints it interacts with.

The code generator solution Cognizant developed is well suited for the process of incremental generation of designated classes, preserving the custom business logic written and maintaining the source code versions by integrating with a Git-based version control system. This makes it best suited for a use case-based migration strategy.

The use of the code generator not only increases the speed of delivery, but also helps in standardization of application code. The code generator generates a significant amount of robust and fully tested code, which enforces uniform design patterns and best practices.

Individual developers are expected to focus only on business logic inside designated placeholders within the generated classes. The scope of application code review and possibility of defects are also minimized by the use of the code generator, improving design time governance and the overall quality of the application.

Solution Architecture

Cognizant’s modernized solution uses AWS Lambda and Amazon ECS on AWS Fargate, a fully managed serverless compute service with workload-aware cluster scaling logic, maintaining event integrations and managing runtimes.

Figure 4 – Architecture diagram. Click to enlarge.

The following table explains how services shown in the architecture diagram (Figure 4) solve a particular technical use case in the technical transformation journey:

Although access to the services in the Experience layer is strictly through Amazon API Gateway, this solution provides multiple access points for services in the Process and System layers.

The System layer endpoints are accessible through Amazon API Gateway and over direct Lambda function invocations using the native AWS SDK toolkit. Similarly, services in the Process layer have Amazon API Gateway endpoints as well as Application Load Balancer endpoints for integration interfaces. This provides the balance between the security and latency of the overall solution.

Cognizant’s solution depends heavily on the availability and reliability of configuration documents for each of the integration endpoints. It uses Amazon DynamoDB, a fully managed, multi-region, multi-active, durable database solution along with Amazon DynamoDB Accelerator (DAX) that delivers up to a 10x performance improvement even at millions of requests per second.

Observability always plays an important role in an application architecture, which helps to reduce mean-time-to-response (MTTR) for a production defect. The traceability of the integration between Amazon API Gateway to services in the Experience layer, through to downstream invocation of the system of records from the System layer services, is achieved through AWS X-Ray. This provides an end-to-end view of requests as they travel through different layers of the integrated architecture, and shows a latency map of the underlying components.

Container Insights and Lambda Insights, available in AWS CloudWatch, helps collect, access, and correlate logs and metrics from the Amazon ECS and AWS Lambda services.

Conclusion

In this post, we shared an enterprise customer case study and viable strategy for modernizing API implementations, migrating from a specialized integration platform-as-a-service provider to a highly available and scalable platform using AWS.

In this application modernization journey, Cognizant used the right selection of services and functionalities around compute infrastructure and managed services provided to build a robust, flexible, and high-performing architecture for the customer while working backwards from their challenges and requirements.

Adopting the right migration approach and using the right set of reusable components, tools, and techniques is important to accelerate the delivery of modernized platforms while maintaining a high level of design-time governance.

.

.

Cognizant – AWS Partner Spotlight

Cognizant is an AWS Premier Consulting Partner and MSP that transforms customers’ business, operating, and technology models for the digital era by helping organizations envision, build, and run more innovative and efficient businesses.

Contact Cognizant | Partner Overview

*Already worked with Cognizant? Rate the Partner

*To review an AWS Partner, you must be a customer that has worked with them directly on a project.