Migrating to Amazon API Gateway: A Datalex success story

Datalex is an industry leader of omni-channel retail solutions for airlines around the world. The Datalex product portfolio supports end-to-end retail capabilities that include pricing, shopping, and order management.

This year, Datalex’s multi-year deal with their API provider was up for renewal. As part of a best practice review, they considered other options. When the AWS Account team showed Datalex a demo of Amazon API Gateway, Datalex was impressed by its rich integrations, flexible pricing model, and compatibility with the OpenAPI Specification (OAS). Because Datalex APIs were consumed by applications out of Datalex’s direct control, any API migration had to be totally invisible and have no impact on existing customers.

This blog post explains how Datalex migrated REST APIs from another provider across all environments to AWS within two weeks and realized a savings of almost 98%.

Considerations

Datalex had to accomplish the initial migration of its REST APIs to Amazon API Gateway quickly, without any impact to their customers. In the API service that was being replaced, applications that consumed Datalex APIs were generally out of Datalex’s control. Speed, maintaining service to customers, and ensuring zero hands-on work were critical to the success of the migration project.

The two primary considerations were to import existing API keys and maintain the existing structure of HTTP requests.

Existing API keys

Datalex customers were using preexisting API keys in production environments. If a change to the API keys was required, the goal of a seamless migration could not be realized. Datalex used Amazon API Gateway APIs to import existing API keys to complete a seamless cutover and avoid the need for new API keys. Amazon API Gateway usage plans allowed Datalex to improve control by implementing quotas and throttles at the API key level. Datalex did not have this kind of control with the former API service. 

HTTP request structure

The existing structure of the HTTP request was to send the API key to the former provider in a format that was not supported by Amazon API Gateway.

Facilitating the invisible swap

Datalex used AWS Lambda custom authorizers to ensure calling clients were not impacted in the process. This made it possible to intercept the request and retrieve the API key without having to change the HTTP request structure.

API Gateway has two options for the source of the API key: HEADER or AUTHORIZER. Initially, Datalex used AUTHORIZER as the source for the API key. The Datalex migration team was able obtain the value of the API key in the Lambda custom authorizer code and send it back as usageIdentifierKey in the response payload. API Gateway uses this value to validate the API key against a usage plan to decide whether a request will be allowed to the backend.

After ensuring that all HTTP methods and resources were represented in the APIs they created in API Gateway and required an API key, Datalex executed existing tests against the newly built APIs in development. This proved everything worked as expected before they moved to production.

Figure 1: HTTP request with AUTHORIZER used as the API key source 

After validating success in dev/test environments, the migration team made a CNAME change to point Datalex’s domain name to API Gateway instead of their previous provider. With that, they achieved their goal of a zero-downtime migration that was invisible to customers and their end users. All Datalex customer API environments were now on AWS.

Thinking long term

The partnership between the Datalex API team and the AWS team was productive and useful. The AWS team provided consultative services throughout the project. In their existing API setup, Datalex was accepting requests to REST APIs from clients with the API key in a format not supported by API Gateway.

After a careful review with the AWS team, Datalex moved to a structure where the API key is sent in a natively supported method, the x-api-key header. Datalex was able to reduce costs by eliminating Lambda authorizer invocations to facilitate API key mapping.

After the successful invisible migration to API Gateway, Datalex sent depreciation notices to their clients, to get them migrated to the new HTTP request structure. In the words of Daniel Morrow:

“By representing our infrastructure as code, this allowed us to quickly clone the API and deploy a parallel version which sources the API key from the x-api-key header. We gave our customers a timeframe in which they needed to point their clients to the new API endpoint which takes the API key in a new format. During this time, both sources were supported. Once all customers were leveraging the new endpoint, the AWS CloudFormation stack representing the original API was deleted.”

Instead of invoking a Lambda function when a request arrives, Datalex can now fully rely on API Gateway usage plans for fine-grained quotas and throttling.

Figure 2: HTTP request to API Gateway with HEADER as the API key source

Business outcomes

Infrastructure as code
Previously, building APIs was a manual process. By using AWS CloudFormation, Datalex was able to develop a repeatable, baseline API architecture represented as YAML code. Developers can also use version control when they make changes to APIs. API Gateway and AWS CloudFormation have removed the heavy lifting of the previous process so developers can innovate at speed.

Fine-grained quota control
By using AWS usage plans and API keys in API Gateway, Datalex has more control over individual API keys. This allows Datalex to allocate a certain number of requests per month on a per-customer basis. It also enhances Datalex’s capability to monetize future APIs.

Cost savings
By migrating to API Gateway, the cost to operate the API solution has dropped by an order of magnitude from the previous implementation. Compared to the pricing agreement with their previous vendor, Datalex has saved almost 98% across all of their REST APIs.

Payment model
Instead of an up-front fixed payment, Datalex wanted a more flexible payment plan based on actual usage instead of projected committed usage. The AWS pay-as-you go pricing model better suits the Datalex business plan and is better tied to Datalex’s transaction-based pricing model.

Tight integration with AWS services
Through their partnership with AWS and use of API Gateway, Datalex realized benefits that include a rich set of integrations with AWS Lambda, HTTP endpoints, and direct integrations with Amazon Simple Queue Service (Amazon SQS), Amazon Simple Notification Service (Amazon SNS), and Amazon Kinesis. For information, see the Datalex and AWS Are Modernizing Airline Retailing blog post.

Future migrations
After more Datalex applications are migrated from the on-premises environment to AWS, API Gateway will facilitate a seamless, straightforward integration between Datalex applications.

Conclusion

Datalex was looking for a more cost-effective solution that would allow the company to scale as its business grows and use cases are added for their customers. The AWS native integrations in API Gateway, such as Lambda custom authorizers, allowed Datalex to make a clean, invisible migration without impacting customers and their end users. The AWS solution is a fraction of the cost of the previous implementation. It makes it possible for Datalex to integrate directly with AWS services and represent the Datalex API footprint as code in a pay-as-you-go model.

Voice of the customer

In the words of Daniel Morrow, the Engineering Director at Datalex, “We completed a seamless cutover from our previous provider to API Gateway. We are now fully cut over on all production and nonproduction systems and no longer have any use for our old provider in Datalex environments. The success of this project will have a big influence on our use of API Gateway when it comes to migration time from the datacenter.”

About the Authors

Ronan Prenty, Solutions Architect, AWS

Ronan Prenty is a Solutions Architect based in Dublin, Ireland. He works with companies of all sizes in Ireland to innovate in the cloud using the latest technologies. Ronan has expertise in the Serverless domain from his time on the AWS Support team. Ronan holds a degree in Computing from Dundalk IT. In his spare time, he is a massive boxing fan and enjoys getting outdoors.

Daniel Morrow, Engineering Director, Datalex

Daniel Morrow has extensive infrastructure engineering management experience and a background in software engineering, development and operations, and cloud engineering. Daniel has held previous roles as Infrastructure Engineering Manager, Hosting Architect, and Software Engineer at Datalex. He is a Computer Applications graduate from DCU and is currently studying part time for an MBA from National College of Ireland.

Alberto Galan, Infrastructure Engineer, Datalex

Alberto Galan has expertise on cloud infrastructure. Alberto has held previous roles in operations and support at enterprises in Ireland and Spain. He holds Red Hat and AWS certifications and has a degree in Computer Systems Engineering from Rey Juan Carlos University.