Partner Success with AWS / Financial Services / India
Muthoot Fincorp Reduces Platform Costs by 61% with Ganit and AWS
Muthoot Fincorp engaged AWS Partner Ganit to optimize its data platform, improving performance, reducing costs, and enabling personalized experiences for customers.
61%
reduction in platform costs
76%
reduction in data processing time
99%
reduction in daily data writes to data storage layer
Overview
Muthoot Fincorp, one of India’s leading non-bank financial companies, provides a wide array of financial services to its customers. It was transforming, reading, and storing the data related to these services in a way that was inefficient and costly, causing a strain on the company’s resources. With a solution from AWS Partner Ganit, Muthoot Fincorp was able to optimize its data storage setup and workflows to improve processing speeds and significantly lower costs.
Opportunity | Seeking a Cost-Effective, Scalable Solution for an Abundance of Data
Muthoot Fincorp provides customers with more than 20 different financial services, and the data behind these offerings lives on Amazon Web Services (AWS). In Muthoot Fincorp’s previous infrastructure, data was pushed to Amazon Simple Storage Service (Amazon S3) from an on-premises SQL Server database using AWS Database Migration Service (AWS DMS). Multiple AWS Glue ETL jobs processed the data to read it in Amazon Athena, write it to Amazon S3, then push it to Amazon QuickSight dashboards for data analysis.
However, with this workflow, 25 years’ worth of data was being overwritten on a daily basis, which took hours to refresh. AWS Glue jobs were rewriting the entire dataset during processing cycles. This led to high spikes in Amazon Athena and Amazon S3 costs each time queries scanned the dataset during a refresh cycle. As data, usage, and users increased, so did costs, and Muthoot Fincorp realized it needed a way to scale operations to manage data more efficiently and cost-effectively.
“The SPICE functionality in Amazon QuickSight was a gamechanger. The service scales up or down as needed to accommodate fluctuating volumes of new data, and removed the need for our engineering teams to manually refresh and load data.”
Sony Soman Pillai
Head of Analytics, Muthoot Fincorp
Solution | A New Approach to Boost Data Workflow Efficiency
Muthoot Fincorp reached out to AWS Advanced Services Partner Ganit to review its data architecture and recommend opportunities for optimization. A thorough evaluation from Ganit determined that appropriate AWS resources were being leveraged, but processes could be better optimized to take advantage of cloud capabilities for scalability, storage, and automation.
Ganit discovered the existing Amazon QuickSight configuration for data retrieval was causing latency when loading reports. The dashboards were reading data from Amazon S3 through Amazon Athena, and large datasets were causing huge read volumes. The previous setup with AWS DMS allowed for only a full load of data, and this data was then processed using AWS Glue. This often caused bottlenecks and outages due to the large data processing jobs timing out. Instead of moving all its on-premises data to the cloud on a daily basis, Ganit configured AWS Glue workflows to load incremental data only. This dramatically reduced the overall load. The new AWS Glue jobs minimized the amount of manual data transformation, reducing processing resources and the potential for data load failures. The incremental data stored in Amazon S3 was then partitioned and saved by date into subfolders, making it easier to quickly find and read.
A key piece of Ganit’s solution was leveraging Amazon QuickSight’s Super-fast, Parallel, In-memory Calculation Engine (SPICE). Muthoot Fincorp utilizes nearly 400 dashboards in Amazon QuickSight to view activities such as total business volume and services used by customers. By summarizing and moving Muthoot Fincorp’s datasets into SPICE, report queries processed faster, and data could be reused without incurring additional costs. Previously, complex joins from reports were accessing tables in Amazon Athena, which took about 10 minutes to load. Once the datasets were summarized in SPICE, loading reports took only a few seconds. SPICE also enabled automated scanning whenever new data was populated or refreshed. Sony Soman Pillai, head of analytics at Muthoot Fincorp, noted, “The SPICE functionality in Amazon QuickSight was a gamechanger. The service scales up or down as needed to accommodate fluctuating volumes of new data, and removed the need for our engineering teams to manually refresh and load data.”
Outcome | Optimizing Data Processing, Reducing Platform Costs, and Enhancing Scalability
By changing the way data was loaded into AWS, Muthoot Fincorp saw a major reduction in processing times. Amazon Athena no longer reads Muthoot Fincorp’s entire historical data, reducing data processing time by 76%. By loading only incremental data, workflows are completed in much less time. Previously, data engineers spent seven hours loading all new data each day. After implementing Ganit’s solution, the same process takes just one and a half hours.
Utilizing SPICE for Amazon QuickSight not only helped process data faster, but also dramatically reduced platform costs. Because data was being processed more efficiently and at much lower volumes via SPICE, Muthoot Fincorp saw platform costs drop by 61%. The company was no longer constantly overwriting its entire stored dataset, and processing only a single day’s worth of data reduced daily data writes to the data storage layer by 99%.
The new architecture enabled faster reporting so that Muthoot Fincorp executives could develop additional reports, such as comprehensive views of a customer’s business data. Employees can use these reports during customer interactions to provide a personalized view of customer activity, informing conversations and improving customer experiences. Faster reporting also facilitated an increase in the volume and frequency of marketing campaigns.
Muthoot Fincorp continues to work with Ganit to further the evolution of its data processing capabilities. It’s currently working on rearchitecting its data pipeline with AWS services. The company continues to explore alternative database options to enhance speed and scalability and incorporate advanced analytics. As Pillai concludes, “Working with Ganit feels like a truly collaborative effort. They’re so engaged and have helped us achieve an optimized experience without compromising performance.”
About Muthoot Fincorp
Muthoot Fincorp is one of India’s leading non-bank financial companies, with over 3,600 branches across India, serving over 100,000 customers a day. It offers an extensive range of products and services, such as gold loans, small business loans, affordable housing loans, insurance products, and wealth management services.
About AWS Partner Ganit
Ganit is an AWS Advanced Services Partner that helps organizations manage data to maximize analytics and decision velocity and minimize decision risks. They help translate data into insights and actions to discover, frame, and solve descriptive, diagnostic, predictive, and prescriptive problems. Utilizing AI and machine learning solutions, Ganit helps businesses understand how to effectively leverage data to make thoughtful, informed decisions.
AWS Services Used
Amazon Amazon QuickSight
Amazon QuickSight powers data-driven organizations with unified business intelligence (BI) at hyperscale.
Learn more »
Amazon Athena
Amazon Athena is a serverless, interactive analytics service built on open-source frameworks, supporting open-table and file formats. Athena provides a simplified, flexible way to analyze petabytes of data where it lives.
Learn more »
Amazon S3
Amazon Simple Storage Service (Amazon S3) is an object storage service offering industry-leading scalability, data availability, security, and performance.
Learn more »
AWS Glue
Preparing your data to obtain quality results is the first step in an analytics or ML project. AWS Glue is a serverless data integration service that makes data preparation simpler, faster, and cheaper.
Get Started
Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today.