Deutsche Börse Group Boosts Trading Platform Testing and Lowers TCO in Collaboration with AWS

Deutsche Börse Group previously ran testing for T7 on premises, using a limited amount of retired production hardware. With a limited number of servers available, this led to bottlenecks, and tests were queued.

“We had some limitations on premises,” explains Christian Hellmann, senior software developer at Deutsche Börse Group. “We had dedicated servers for testing, which became overloaded in certain situations during a project where a lot of code delivery and compilation is done. Also, developers had to wait for the feedback of their delivery because a lot of smoke tests were going on.”

Deutsche Börse Group wanted to modernize and accelerate its testing capacity to meet rapidly changing expectations and to maintain the quality of software updates and changes. In 2017, the Group decided to move its T7 testing environment onto Amazon Web Services (AWS) to take advantage of the secure, resilient global cloud infrastructure and elastic, flexible resources.

As part of that move, Deutsche Börse Group decided to use Amazon EC2 Spot Instances. This provides flexible access to AWS compute resources and resilient testing capacity. A Spot Instance allows Deutsche Börse to take advantage of unused Amazon Elastic Compute Cloud (Amazon EC2) capacity up to a 90 percent discount compared to on-demand pricing, lowering Amazon EC2 costs and total cost of ownership (TCO) significantly.

Because AWS sometimes needs to reclaim the capacity for a Spot Instance, Deutsche Börse Group had to balance flexibility and price advantage against the risk of test workloads failing when an instance can be terminated before the test can be completed.

Tracking the termination rates of Spot Instances on a daily basis helped Deutsche Börse Group to overcome the possible termination of instances. This involved using the 30-day average regional figures from AWS—a way to identify the lowest termination rates and where best to place an instance for any given test workload, while allowing for flexibility in instance type and region. In the case of a Spot Instance being terminated, Deutsche Börse Group developed a process to either re-execute the tests using on-demand AWS products and services, or redistribute to other Spot Instances, depending on the type of test.

However, since Deutsche Börse Group started using Spot Instances for its testing capacity in 2017, termination rates have been very low: just 2.3 percent. And for instances that are terminated in the first hour, Deutsche Börse Group isn’t charged.

“Now that we don’t have the limitations of on premises there are no queues for testing anymore,” says Hellmann. “The scale-out capability of AWS has allowed us to reduce the testing overload situation, and this increased developer happiness and satisfaction, too.”

Moving the T7 platform testing capacity onto AWS allows Deutsche Börse Group to run T7 testing on single-host deployments, as well as to test multi-host deployment environments using the multi-cast feature in AWS Transit Gateway, which helps testers and developers to uncover issues earlier in the development and testing cycle.

The importance of this type of testing is critical. That’s because, after updates and new releases are in the production environment, the T7 platform uses hundreds of servers, all communicating with one another simultaneously via multi-cast networking for inter-process communication and low-latency messaging.

Migrating to AWS has enabled Deutsche Börse Group to run testing at scale and improve and shorten feedback cycles for its T7 platform without having to make significant investments in new infrastructure.

“Using the instant capacity that AWS provides has shortened the testing cycle for the T7 platform—in some cases, from five days to just a few hours—by giving us more flexibility for testing and allowing developers to get feedback quicker. Spot Instances also makes this architecture cost-effective. Extensive software testing is vital to maintain the quality and functionality of our trading platform,” says Christian Hellmann.

For example, for the most common type of test—smoke test—total costs have been significantly reduced. During April 2020, Deutsche Börse Group ran 1,061 tests and re-ran 90 tests—compared to running these tests on-demand, the Group was able to reduce costs by 70 percent using Spot Instances.

“The benefit of Spot Instances is the pay-as-you-go approach without the need for a long-term contract or complex licensing,” says Helge Harren. “This really allows you to be flexible in your consumption and architectural choices, and to make more use of the ‘best fit’ machines for the short periods they are required.”

This proved to be a benefit for the regression testing, which is more complex than other types of tests and involves the automatic execution of functional test conditions and specific scenarios.

In the old on-premises development environment, Deutsche Börse Group was able to run the regression tests only during the weekends, using 10–15 servers that were borrowed from the development team for this type of test. If a regression test failed over the weekend, that failure wouldn't be discovered until Monday and then it would not be possible to re-run the test until the following weekend.

Now, Deutsche Börse Group is able to execute its regression tests across 150 servers on Spot Instances overnight, significantly increasing the scope and scale of its testing capability, and delivering much faster feedback to testers. This means that, if a regression test fails, the test can now be run again the following night.

Using AWS, Deutsche Börse Group has achieved its goal of increasing the scale of software testing, while lowering TCO, to maintain the functional quality and performance of the T7 trading system.

As a result, Deutsche Börse Group is now in the process of bringing the testing and development environments even closer together: it plans to migrate development from on premises to AWS later in 2021.