Profectus Capital Builds a Data Lake on AWS with Noventiq for Faster Insights

Profectus Capital Private Limited, a financial institution that caters to small-and medium-sized businesses (SMBs), partners with many fintech organizations to make the loan disbursement journey digital, seamless, and quick. With its entire technology stack on AWS, the next step for Profectus was to consolidate data from multiple systems for a comprehensive view of loan applicants, business profiles, and loan collaterals. Working with AWS Partner Noventiq, Profectus developed a Data Lake on AWS that uses AWS Glue, Amazon S3, and Amazon Redshift. As a result, Profectus now has an instant, 360-degree view of insights across multiple systems to help streamline the decision-making process around loan sanction.

Profectus was founded in 2017 to help the millions of SMBs in India access financial line of credit. Early on, Profectus executives knew they wanted a scalable IT infrastructure and decided to build their entire data estate on Amazon Web Services (AWS). The company has multiple systems running on AWS that capture loan applicants, borrower business profiles, and collaterals details. This data helps different departments make lending decisions. While this setup worked, Profectus wanted to consolidate data from various systems to leverage business intelligence (BI).

For each data request, data consolidation took approximately five hours, and the IT team was looking for a faster turnaround time. Along with a goal to modernize and improve the customer experience, Profectus also wanted a modeled data layer to drive a 360-degree view of each customer.

With references from its internal team who works with AWS, Profectus connected with AWS Partner Noventiq because of the team’s deep cloud capabilities. “When you interact with a potential technology partner, you get a sense of their knowledge,” said Vitthal Naik, chief technology officer at Profectus. “It was clear that Noventiq really knew AWS services in depth, and we would be able to deploy an efficient and performant solution together.”
 
Noventiq created a comprehensive plan to integrate the disparate systems at Profectus. Within just two-and-a-half months, Noventiq developed a data lake that uses Amazon Simple Storage Service (Amazon S3) to store the point-in-time view of all critical historic data and the operations performed on this data. It also uses Amazon Redshift as a data warehouse layer to handle BI queries, while AWS Glue takes care of data movement and extract, transform, and load (ETL) processes. AWS Glue brings data from disparate systems into Amazon S3, then performs ETL to generate key performance indicators (KPIs). In addition, Amazon Athena handles ad-hoc queries.

“Using open-source distributed processing frameworks like PySpark over AWS Glue and Amazon Redshift enables a completely fault-tolerant ETL solution and a highly available data warehouse at a fraction of the cost,” said Satheesh Nair, senior solutions architect at Noventiq. “These AWS services are perfect for managing a huge amount of data, and they’re much more performant and efficient to use than legacy technologies.”

With a single source of truth that links data from multiple systems, Profectus has accelerated its time to insights. For all its departments—from sales to credit underwriting—faster insights have enabled more streamlined decision-making. For instance, the creation of customer credit scorecards is much easier now. The scorecards consolidate KPIs from various systems to help the company determine if an applicant is qualified to receive a loan.

Because of the data lake architecture, Profectus plugs in additional datasets and external data about the business from various agencies like banking partners, tax agencies, credit bureaus, and payment gateways to gain a holistic view on the customer’s ability to repay a loan. “It used to take a few hours to stitch together data from individual systems previously,” Naik said. “With a Data Lake on AWS, generating a holistic customer view can be done with a click of a button.”

Building a data lake has also enabled Profectus to use unstructured data to generate leads. The company’s sales team has analyzed roughly one million annual reports from schools that are available in the public domain as PDFs. Profectus added the PDFs to the data lake and using Amazon Textract, extracted useful information from them so the sales team could search for schools in need of loans. “Amazon Textract allowed us to extract structured data from unstructured PDFs, which eventually helped our sales team with targeted leads,” Naik said.

With a solid data foundation, the next step for Profectus will be enhancing predictive analytics capabilities and developing generative artificial intelligence (AI) applications. The company currently works with a few fintech partners for various data analytics use cases. But with a data lake architecture, Profectus can implement these predictive capabilities through an in-house data science team. Using generative AI, internal teams can standardize their interactions with customers through bots and create a knowledge base that will help employees stay current on industry trends.