Sperry Rail Improves Accuracy 50% Using AI and Gains Scalability with Steamhaus on AWS

Sperry Rail Service developed an artificial intelligence (AI) solution to automate the analysis of railroad track data—its core business—but couldn’t process and analyze data as fast as it could collect it. So it engaged AWS Partner Steamhaus to improve its tool to scale and run faster and more economically on Amazon Web Services (AWS), with the goal of freeing analysts from tedious and repetitive tasks and helping them to focus on potential problems flagged by the tool.

Sperry Rail Service is a rail inspection company founded in 1928. Based in the US, it works under contract to many leading railways to inspect and analyze the condition of rails. The company does most of its business in North America but operates globally. By identifying damage in the rails—including the precise location and nature of it—and by removing false positives, Sperry ensures that its railway maintenance business customers around the world can most efficiently and cost-effectively deploy their resources to fix the damage. The accuracy and depth of its analysis also helps to minimize disruption to rail travel.

To provide its services, Sperry runs specialized trains that scan rails and capture data, which is then analyzed. The company has a history of adopting new technology to improve the collection and processing of data. Sperry developed a new artificial intelligence (AI) tool called Elmer (named after founder Elmer Sperry) and built it on AWS to automate the analysis of rail data. The goal was to improve processing time and increase throughput, helping analysts to focus on the potential problems flagged by Elmer. “The data science was good,” says Bobby Gilbert, senior director of digital transformation at Sperry. “The problem was that we were getting huge backlogs of data waiting for processing.”

Gilbert began the development of Elmer as a proof of concept (PoC) to see how technology could help Sperry process the data it was collecting from the scanning trains. The PoC proved the concept could work, but it simply couldn’t scale. The scans, which are primarily ultrasound data, were collected, sorted, and examined by Elmer. The data was then made available to human analysts who could check the work. “Even in this early form, it was a real game-changer,” says Gilbert. “It helped analysts more easily identify problem spots. But we were a victim of our own success. We wound up with massive queues of data piled up with an 8-hour wait to be processed.”

Sperry wanted to access the data it was collecting more quickly and use fewer resources to process it. The company runs about 100 trains per night, each collecting data on 10–100 miles of track. The company approached its AWS account manager, who referred Sperry to AWS Partner Steamhaus to help it evaluate Elmer and identify ways to make it scalable. “When we met with Sperry, we undertook a Well-Architected review,” says Phil Horn, director at Steamhaus. “Sperry wanted to take Elmer from an minimum viable product (MVP) to an enterprise-grade solution and optimize costs.”

AWS Well-Architected helps cloud architects build secure, high-performing, resilient, and efficient infrastructure for a variety of applications and workloads. Steamhaus proposed short- and long-term improvements to help Sperry across the pillars of AWS Well-Architected, which include operational excellence, security, reliability, performance efficiency, and cost optimization. “The Well-Architected review helped us see where we were strong—such as in security—and where we could use help,” says Gilbert. “After the review, we decided to work with Steamhaus to put its advice into practice. It was clear they could help us reach our goals more quickly.”

Steamhaus recommended two major architectural improvements to Sperry. The first was to adopt more serverless services and abstract more resources. And the second was to implement a managed build, continuous integration, and load-test environment by taking advantage of the automation features in AWS services. “Sperry was really just using AWS as a compute service, and that got expensive as you scaled,” says Rob Greenwood, chief technology officer at Steamhaus. “The development cycle was also slow—slow to develop, test, and release new features to Elmer. So it was clear that we could offer value as advisers on AWS. They had the AI development figured out already.”

Sperry and Steamhaus containerized the company’s Amazon Elastic Compute Cloud (Amazon EC2) workloads using Amazon Elastic Container Service (Amazon ECS), a fully managed container orchestration service that helps you to more efficiently deploy, manage, and scale containerized applications. This solution uses AWS Fargate, a serverless, pay-as-you-go compute engine that lets users focus on building applications without managing servers. “We’ve now got access to more resources, managed for us, that scale,” says Gilbert. “We pay only for what we use. We’ve reduced the monthly cost of Elmer by about 35 percent and get more value.”

As a result of this work, Sperry can do better work, more quickly. Analysts are able to make more informed decisions because they are presented with 10 times the data in an intuitive format. Rather than making decisions based on a single set of ultrasound data, they now see the four most recent recordings with areas of concern identified by AI. They use Amazon Athena—a serverless, interactive analytics service built on open-source frameworks—and Amazon DynamoDB—a fully managed, serverless, key-value NoSQL database designed to run high-performance applications at any scale—to allow them to perform at speed. The tool also offers close-up vision images embedded into the ultrasound scans to give context to the features, including terrain and structures, such as bridges, curves, level crossings, and nearby buildings. “This means that, for the same resources, our analysis team can make more informed decisions and reduce the number of false positives by as much as 50 percent,” says Gilbert. “And we’ve eliminated that 8-hour queue. Data is processed in near-real time, as it is collected.”

With Elmer transforming the way that Sperry is doing business, the organization is re-imagining itself. It sees a transition from being a non-destructive testing business to being a data- and AI-driven software business. “Gathering and analyzing data is at the heart of what we do—its what we’ve done for a century,” says Gilbert. “AI has opened up possibilities of how that expertise can be applied. Using AWS and working with Steamhaus as a partner, we can innovate new products and features and take them to market more quickly.”

Sperry and Steamhaus continue to work together, building new products on AWS. At the core of this ongoing program of work is the creation of a big data lake (named Zula, after Elmer Sperry’s wife). Using this, Sperry will be able to mine its data for a wealth of insights that will improve the company’s business intelligence. Steamhaus is also working with internal Sperry teams to free them from routine cloud administration and maintenance tasks using automation and by upskilling them in cloud-native technologies and methodologies. “If I’d known about the benefits of working with Steamhaus, I would have started sooner,” says Gilbert. “For the future, it’s full steam ahead.”