Lower Latency and Costs Using AWS Graviton2–Based Instances with Sprinklr

Sprinklr’s artificial intelligence creates insight-driven strategies that help companies improve their customer experience. The company ingests and analyzes end users’ data from digital channels to detect trends and problems before they materialize so Sprinklr’s customers can proactively address them. For example, Sprinklr Modern Care analyzes billions of end users’ conversations across various channels, identifies intent and sentiment, and routes end users to the right support, all in near real time.

Sprinklr serves 70 percent of Fortune 200 companies, such as Cisco, Procter & Gamble, and Disney, and has service-level agreements for throughput and performance speed. Yet processing a massive amount of data in near real time is challenging: the search workload is disk intensive and requires many input/output operations to transfer data between sources and servers, not to mention immense memory and processing power that’s expensive. Without enough compute, Sprinklr experienced bottlenecks at the level of its disks and central processing units. “Our main goal is to have better performance in terms of latency and the user experience while optimizing cost,” says Nitin Goyal, vice president of engineering at Sprinklr.

The company needed new instances to support its search infrastructure, which fuels search, aggregation, storage, and analytics capabilities. Sprinklr uses several storage engines, and Elasticsearch is the biggest one, comprising 40–50 percent of Sprinklr’s cloud spend. Knowing that Sprinklr is always searching for new compute, AWS reached out in early 2022 to suggest that the company try Amazon EC2 Im4gn and Is4gen Instances, next-generation, storage-optimized instances designed for running applications that require high throughput and low-latency access to large amounts of data on local solid state drive storage. Both instance types run on AWS Graviton2 processors.

Moving to AWS Graviton2–based instances took just 2 weeks. Sprinklr developed an internal testing tool on AWS to compare instance types and configurations; using it, the company ran key complex queries to test the performance of Amazon EC2 Im4gn and Is4gen Instances. Because Sprinklr was an early adopter of the instances, it ran into capacity limitations as it tried to migrate its workloads to those instances. However, AWS support delivered the capacity that the company needed a few days after Sprinklr raised the issue. “Our experience with AWS support in terms of escalation and issue resolutions, especially when we’re evaluating new instances, has been outstanding,” says Jamal Mazhar, vice president of infrastructure and DevOps at Sprinklr.

For Elasticsearch, Sprinklr is using only Amazon EC2 Im4gn Instances, which offer excellent price performance for storage-intensive workloads. Query performance has improved by 36 percent, resulting in a 40 percent price performance benefit. Sprinklr also uses Amazon Elastic Kubernetes Service (Amazon EKS), a managed Kubernetes service to run Kubernetes in the AWS Cloud and on-premises data centers. Amazon EKS pairs with AWS Graviton2 processors to power Arm-based Amazon EC2 Instances, delivering increased performance and cost savings.

Sprinklr, which processes more than one billion queries daily, reduced latency for queries by 36 percent, and its query time went from 700 to 450 milliseconds on read-heavy clusters—all by migrating to AWS Graviton2–based instances. The company has better disk performance and higher throughput on central processing units, which has reduced bottlenecks during input/output operations and accelerated recoveries. Now Sprinklr customers can perform analytics faster, improving their experience and increasing their usage of the solution. “On AWS Graviton2–based instances, we can build more features that before were either expensive or not feasible,” says Goyal. For example, Sprinklr now provides analytics on high-cardinality fields, which facilitates deeper analytics for customers.

Because Sprinklr sees a higher throughput on the same number of instances, it has a lower carbon footprint, appealing to environmentally conscious customers. The company’s report shows that its carbon footprint was reduced by 56 metric tons between January and June 2022. “AWS Graviton2–based instances run at less clock speed, thus using less heating and electricity,” explains Mazhar.

Switching to AWS Graviton2–based instances was quick and resulted in higher performance, reduced costs, improved customer experience, and a better carbon footprint for Sprinklr. The company plans to run other pieces of its analytics pipeline, including persistent storage NoSQL databases and data streaming services, on Amazon EC2 Im4gn Instances. It will continue to seek new compute from AWS that will improve the customer experience. Goyal says, “We are constantly innovating for customer needs and for the company.”