Kainos and AWS Help DVSA Utilize Machine Learning to Minimize Fraud and Improve Road Safety

For more than 30 years, Kainos has created digital technology services and platforms for organizations worldwide across the public and private sectors. Kainos employs more than 1,400 people in offices across Europe and North America. The company is an Amazon Web Services (AWS) Advanced Consulting Partner in the AWS Partner Network (APN). Kainos is also a qualified AWS Public Sector Partner.

Kainos’s extensive experience partnering with UK government departments made it a natural choice for the Driver and Vehicle Standards Agency (DVSA), which oversees Ministry of Transport (MOT) tests, testers, and garages. Since 2013, Kainos has worked with DVSA on multiple digital transformation projects.

Early work with DVSA centered on replacing the mainframe-based MOT system with a cloud-based system. The new service, hosted on the AWS Cloud, is faster and provides a user-friendly interface on devices already accessible to testers. More projects followed, including the MOT History Check, which enables the public to check a car’s MOT test history, mileage records, and test due date. Another project, the MOT Reminders service, uses AWS Lambda serverless technology to send test date reminders to vehicle owners.

DVSA selected Kainos to develop a system using machine learning and predictive analytics to identify fraud and inadequate standards at MOT testing sites. With its in-depth understanding of machine learning, Kainos delivered an effective algorithm, demonstrating the company’s expertise in collating, understanding, and preparing data, and its experience validating predictive model output. The project particularly suited Kainos’s agile work style, with regular testing and feedback making the design, tuning, and validation process as efficient as possible.

DVSA relies on a team of 266 vehicle examiners (VEs) to carry out time-intensive spot-checks on 66,000 MOT testers at 24,000 garages. Deciding which garages to spot-check was a manual, paper-based operation. Based on a review of historic information and data gathered at a spot-check, the VE assigned a risk score that was subjective and based on performance witnessed on a single day.

Kainos worked with DVSA on a proof of concept for the new solution. Both teams reviewed MOT system data, and data scientists from Kainos determined what could be extracted for use in data models. “The Kainos team showed us the potential—where this project could go, and what we could use it for,” says Chris Price, MOT testing product manager at DVSA. “After the presentation from Kainos, we thought, ‘If we can accomplish what they’ve shown us, this will be a great tool.’”

The machine learning tool enables DVSA to identify abnormal MOT testing behavioral patterns, effectively target enforcement and inspection activities, and ultimately improve Britain’s road safety. The risk-rating algorithm focuses on individual MOT testers’ performance, considers changes to testing behavior over time, and refreshes regularly, enabling accurate, up-to-date risk scoring. It also incorporates additional helpful information, including test volumes, frequency, duration, and pass rates, as well as disciplinary history.

Early trials revealed several high-risk garages that had not been flagged by previous methods. VEs visited the garages to confirm the model’s findings, and some garages were sufficiently high-risk that the VEs stopped them from testing immediately. “Using data models on the rich data sets we had was hugely powerful. We could not have manually processed that data. Confirming the accuracy of the models gave us huge confidence in the risk-rating tool,” says Price.

For this tool, Amazon Elastic Compute Cloud (Amazon EC2) provides resizable compute capacity in the cloud using MOT data stored in Amazon Simple Storage Service (Amazon S3) and Amazon DynamoDB. “The ability to scale compute power far surpassed what we’d have been able to accomplish in a traditional, on-premises data center. The underlying AWS infrastructure was what enabled us to achieve success,” says Ricky Walker, principal architect for Kainos.

The modeling algorithm builds a picture of performance over time, detecting garages with poor practices. “VEs now benefit from all the relevant information at their fingertips on a risk-rating spreadsheet, telling them what garages to visit, improving their operational efficiency and decision making,” says Price. Assessments are now objective, and the time required by VEs to analyze data before a spot-check has been reduced by 50 percent. Garages embraced the change, pleased that the process is now objective and data-driven, and it more accurately identifies high-risk garages. Since implementation, the model has prompted 3,200 site reviews, 74 percent of which resulted in an “unsatisfactory” or “needs improvement” rating. Of those site reviews, 33 percent of visits to high-risk garages resulted in disciplinary action or warning letters.

The DVSA modeling tool created by Kainos was named AI & Machine Learning Project of the Year in 2018 by the UK IT Industry Awards. “The success of the MOT projects over the years inspires our ambitions as an agency. It gives us the confidence to keep trying new things,” says Alex Fiddes, head of digital operations for DVSA. The collaboration between Kainos and DVSA continues, as does the innovation. The team’s next project is to predict if a vehicle will pass or fail its MOT test and use those predictions to add more data to the risk-rating system. “Our long relationship with AWS paved the way for our success with DVSA. Thanks to AWS, we’ve discovered the art of the possible. Kainos has helped DVSA push its boundaries to find the best, most innovative ways to make UK roadways safer,” says Darren McIlveen, delivery manager for Kainos.