Dataminr Achieves up to 9x Better Throughput per Dollar Using AWS Inferentia

Founded in 2009, Dataminr employs over 850 people across eight global offices. Dataminr’s AI platform detects early signs of high-impact events and emerging risks in near real time, from more than 500,000 publicly available data sources. The company’s alerts help customers to know critical information first, mobilize for quick response, and manage crises effectively. Speed and coverage are the key values that Dataminr strives to provide its customers. “We cover many types of events all over the world in many languages, in different formats (images, video, audio, text sensors, combinations of all these types) from hundreds of thousands of sources,” says Jaimes. “Optimizing for speed and cost given that scale is absolutely critical for our business.”

Due to the size and scope of Dataminr systems, the company strives to optimize everywhere that it can. However, it’s not enough to reduce costs. Each project that the company undertakes must help it increase scale, whether that be in the form of the speed of compute or number of data sources. Dataminr uses Amazon Elastic Compute Cloud (Amazon EC2), a broad and deep compute solution, to host its models at scale. “For any organization, time and money are constraints, but we wanted to continue efficiently scaling our coverage to generate additional types of alerts,” says Hill. The company started searching for a way to optimize for both speed and cost simultaneously to scale on Amazon EC2.

In 2021, the company started to experiment with AWS Inferentia to optimize its Amazon EC2 spend, while scaling its models. “We built on our early experiments to develop a pattern by which many common model types can be dropped into an optimization workflow,” says Hill. “Then, we used AWS Inferentia to produce and benchmark a compiled model so that we could select an optimal way to deploy it.”

The first models produced using AWS Inferentia were deployed in spring of 2022, and the implementation process went as smoothly as possible. When there was an issue, Dataminr reached out to AWS Inferentia experts who provided quick guidance to develop a solution. “We were able to call in an AWS expert to diagnose memory-usage patterns and optimize our approach,” says Hill. The early results were promising. “On one of our early efforts, we increased speed by five times compared to GPU-based instances on a natural-language processing task,” says Hill. “That translated into a nine-times improvement in throughput per dollar spent for our natural-language processing models.” Those initial results inspired Dataminr to move forward with the effort, which is delivering five times increased throughput per dollar or more across all the models that it optimized, including computer vision and natural-language processing.

Dataminr is realizing three distinct business benefits from the project: increased scale, increased speed, and lower costs. Moreover, Dataminr is seeing increased accuracy in cases where AWS Inferentia has facilitated the use of more complex models or covers more data sources, which are vital to effective crisis-response efforts.

Developers are also enthused. Dataminr emphasizes innovation, and the engineers are excited to have a new, cost-effective way to deploy AI models beyond CPUs and GPUs. The company’s commitment to innovation is now driving an internal optimization push to automate model compilation and benchmarking. “We really like working on AWS Inferentia,” says Jaimes. “We need only a few people to get this up and running, which is great.”