Luno Boosts Fraud Detection with AWS-Powered Machine Learning

As the sixth-largest cryptocurrency platform in the world, with eight million customers and 300 percent year-on-year growth, Luno is at the cutting edge of financial technology. But—like all consumer-facing fintech companies—it’s still subject to one of the finance sector’s oldest problems: fraud. With its business dependent on credit card payments and bank transfers, Luno must detect and deny fraud while protecting honest customers.

The company started out using a third-party fraud detection service that scores requests and raises risk alerts, allowing Luno to secure and investigate suspicious-looking accounts. But rapid growth and a desire to make better use of its customer data led Luno to make a change. It decided to replace the third-party service with an in-house system that uses Amazon Web Services (AWS) for machine learning (ML) technologies.

Luno had found its previous third-party system was too generic, with decisions based on data from a wide range of clients. In addition to enabling more focused, more accurate, and better performing fraud detection, Luno wanted a more cost-effective approach using AWS. 

Two in-house data scientists led all of the artificial intelligence (AI) and ML work, initially testing various models on their laptops. After they identified the most promising models, Luno built a processing pipeline and expanded the project by bringing in other parts of the organization and experts from AWS.

The six-person project team includes data engineers who handle input to the ML system, and fraud analysts who check and act on that system’s results. That led to some cost savings compared to the third-party system, which needed eight people to manage integration, data transfer, and upgrade tracking. 

Luno chose Amazon SageMaker for the core of the project. The company was already using AWS for service hosting and data storage, so that seemed like a natural decision. “At the time, productionizing ML systems was still widely seen as difficult,” says Richard Ball, Luno’s lead data scientist. “But we’d heard good things about Amazon SageMaker, and it turned out to be a great way to run open-source tools in production.” Luno found Amazon SageMaker’s versatility to be attractive: the service can handle ML frameworks like TensorFlow, PyTorch, and others, and it supports a flexible ecosystem of techniques and tools for training and tuning models.

Using ML to examine examples and identify patterns, the system learns to identify user behavior that indicates fraud. Luno generates tables of 47 user data points, including where users are logging in from, how they navigate, and what devices they’re using. This data is stored in Amazon Redshift. 

Once a day, a pipeline is triggered to process this data using AWS Glue and Amazon Simple Storage Service (Amazon S3), with the results then fed into the ML model in Amazon SageMaker. That output goes into Amazon S3, and then via AWS Glue to AWS Redshift, where it can be inspected by business intelligence tools from AWS Partner Looker. Overnight, the system refines its pattern matching by examining the decisions that human fraud analysts make about its previous conclusions. 

The pipeline is coordinated by AWS Step Functions and AWS Lambda, which also manages integration with Slack, the tool used for queries and error alerts and management. 

With AWS, Luno took just a year to go from its initial models to having a fully automated system that produces useful results. 

To qualify those results, Luno still runs and tests those against the third-party service. It is already comfortably exceeding its performance, and expects continuous improvement through the nightly retraining sessions and further manual tuning. Using an ML quality metric called AUC to measure performance, the third-party system achieves results of around 80 percent, while the in-house system using AWS currently scores 94 percent. 

Because Luno is still running both systems, it hasn’t yet realized the full cost benefits of the switch. However, it expects to achieve a positive return on investment (ROI) when they decide to fully migrate to the in-house solution. “We’re still tuning our model, which is doing much better at identifying fraud,” says Ball, adding that the company will keep fine-tuning to improve those results even further.

Luno expects further long-term benefits as the system helps it understand its customers better going forward. Verified customers will be able to unlock extra features that aren’t accessible to others, making the process more friction-free, and building customer satisfaction and growth. 

The project has already inspired innovation. 

“Now that we have a fully automated ML pipeline system running so well in Amazon SageMaker, we can build new services beyond fraud detection,” says Ball. “We already have a second model for automatic customer service emails running in parallel on the same framework, a third payment system in development, and plans for app optimization.” 

Ball says that the project has gone well. “I'm definitely happy we went down this route,” he says. “I don’t think it would have been possible without machine learning, and it was relatively easy to create the product. From a strategic perspective, our successful deployment of Amazon SageMaker gives us the confidence to integrate machine learning and automated processes in future projects. It’s been a very useful project for a lot of people in the business.” 

Ball advises anyone considering an AI and ML fraud detection system to first thoroughly define the problem being solved and think about how the system will sit within a company’s data and technical frameworks. “Be prepared for concerted effort, dedication, and focus,” he says. “You have to really understand the problem, and how the project will interact with the organization.” Ball adds, “Communicating with the rest of the organization will need work, as will getting buy-in. And you will need a data scientist or two. Powerful as Amazon SageMaker is, it’s not a one-click solution. You’ll need to talk to fraud analysts about what data will create insights to feed the model.” 

But get it right, he says, and you’ll have a powerful internal resource that can open up many new possibilities across your organization, and for your customers: “It’s so much more than just another project.”