HUMAN Security Accelerates ML Training and Time to Market Using Amazon SageMaker

HUMAN Security uses a modern defense strategy, which includes disruptions, network effect, and internet visibility, to disrupt the economics of cybercrime. It helps businesses across all sectors increase the security of their digital presence by offering a wide range of cybersecurity solutions that help companies protect their digital assets from fraud and online bots imitating humans. For digital advertisers, the company created MediaGuard, an ad tech solution that uses ML in the Human Defense Platform to predict the validity of online advertising impressions in near real time across all digital channels and formats.

Because online bots are becoming increasingly sophisticated, HUMAN Security maintains strict latency and accuracy requirements for MediaGuard, and its team of engineers continually iterates new ML models to enhance its performance. However, when HUMAN Security rolled out this solution, the process for training its ML models was entirely manual and involved running a number of scripts and copying and pasting different configurations. In many cases, it took HUMAN Security weeks to deploy new ML models. “We wanted to save human time,” says Austin Leirvik, staff data scientist at HUMAN Security. “We were looking to set up a complete data pipeline that would do the data preparation, data extraction, model training, and off-line model evaluation all at the push of a button.”

Since its founding in 2012, HUMAN Security has relied on AWS for cloud solutions, and in 2020, it engaged the AWS team to mature its ML capabilities. “We collaborated every 2 weeks,” says Leirvik. “We received lots of feedback on how we could automate our model training, and we saw SageMaker as a tool that we could use to solve the problems that we were facing.”

While HUMAN Security engaged the AWS team, it also participated in multiple training opportunities through AWS, including AWS Partner Network Immersion Days, which are customer workshops delivered by AWS Partners. These training opportunities helped HUMAN Security upskill its staff and gain a deeper understanding of the ML model lifecycle. HUMAN Security also adopted Snowflake Data Cloud, a solution for data warehousing, data lakes, data engineering, data science, data application development, and data sharing from Snowflake, an AWS Partner. The company uses this solution to process and store its data tables at scale. “For a typical model training run, we’re working with around 50 million data points,” says Leirvik. “Because we can do our off-line evaluation on a bigger dataset, we have a much bigger picture of the long tail, which is really nice.”

The company also began using AWS Glue, a simple, scalable, and serverless data integration service. HUMAN Security uses AWS Glue for its extraction jobs and to prepare its data for querying. After the data has been prepared, HUMAN Security uses SageMaker to build, train, and deploy new ML models. “By using Amazon SageMaker, we’ve substantially reduced the amount of time that is needed to train ML models,” says Leirvik. “We have complete traceability and reproducibility across all our models.” Previously, training a new ML model could take HUMAN Security several weeks. Now, the company can build, train, and deploy a new ML model within hours.

Moreover, HUMAN Security runs its workloads using Amazon Elastic Compute Cloud (Amazon EC2) M5 Instances, which offer balanced compute, memory, and networking resources for general-purpose workloads. Since switching to this Amazon EC2 instance type, the company has increased its cost savings by 15 percent, and it can quickly scale demand. This scalability helps HUMAN Security power its ML models to validate the humanity of 15 trillion online interactions every week. “We’ve been very happy with the scalability and reliability of Amazon EC2 M5 Instances,” says Leirvik. “We’ve been able to increase the amount of data that we’re working with by five times.”

To achieve full automation, HUMAN Security set up step functions across all its AWS solutions by defining a set of configuration files using Amazon States Language and adding those files to its repository with the rest of its ML codebase. Anytime a change is made to its codebase, the company automatically redeploys those step functions, which has reduced the complexity of its workflows. This automation has helped the company accelerate its time to market and enhance its business agility. With additional time savings, HUMAN Security has refocused its efforts on releasing new predictive features for MediaGuard. “Using AWS, we’ve tripled our number of deployments compared with our previous process,” says Leirvik. “Now we can react more quickly when we see a performance problem emerging.”

HUMAN Security plans to apply its learnings from this project to other ML models that it has in production. It will also continue using AWS services for a wide variety of use cases across the company. “Working alongside the AWS team has been a very positive experience,” says Leirvik. “The AWS team helped us look at the problem that we were facing in a new way and kept us on pace so we would succeed.”