Discover Financial Services Builds a Generative AI Solution on AWS for Faster Decision-Making and Time to Market

Chicago-based Discover, a digital banking and payment services company, aims to help people spend smarter, manage debt better, and save more. In its various lines of business, including decision and credit and portfolio risk management, Discover faced the challenge of making sure that it could launch its services fast enough. “We wanted to make decisions quicker and get to insights faster so that we could respond faster to our customers,” says Rahul Gupta, AI/ML expert platform engineer at Discover.

The bank wanted to use generative AI and ML to analyze data and generate insights. It was also looking for ways to train large language models faster and use compute capacity optimally to reduce time to market. Discover decided to run its data science solution on Amazon Elastic Compute Cloud (Amazon EC2), which provides secure and resizable compute capacity for virtually any workload.

Amazon EC2 is part of AWS AI infrastructure, which helps accelerate AI innovation. Enterprises and developers can use the comprehensive, secure, and price-performant infrastructure to build AI applications with a broad and deep set of AI and ML capabilities across compute, networking, and storage.

Discover built an analytics workbench on Amazon EC2 and a unified data science workbench that its data scientists can use to run or process AI/ML applications, train models with large sample sizes—requiring up to 6 TB of memory—and deliver high-performance computing (HPC) in the cloud using core GPUs. “We have provided our scientists with a cloud-scale data warehouse with on-demand HPC scaling, and we have accelerated our analytics innovation,” says Gupta.

Discover uses Amazon EC2 P3 Instances to accelerate ML and HPC applications with powerful GPUs for tasks that require a multi-GPU setup. It also uses Amazon EC2 P4 Instances to get high performance for ML and HPC applications in the cloud. The team spent considerable time on optimizing the architecture and implementing best practices to help speed up analytics and deliver insights faster. “We wanted to make sure that we had optimized runtimes for our infrastructure, especially our compute,” says Will Hinton, director of data and AI platforms engineering at Discover.

For instance, the team ran benchmark tests using different models and codes to assess the speed of processing 20,000 rows of data, transcribed from recorded interactions between customer service agents and customers. Using 16 CPUs, the processing took 6.5–7 hours, while a single-GPU setup took 23 minutes. Using multiple GPUs brought the processing time down to 4 minutes but also increased costs. This helped the Discover team choose different GPU setups based on use case requirements and cost.

Discover uses Amazon Simple Storage Service (Amazon S3)—object storage built to retrieve any amount of data from anywhere—to store the model artifacts. To share these artifacts and data with different engineering teams and lines of business, Discover uses Amazon Elastic File System (Amazon EFS), which provides serverless, fully elastic storage. This solution is connected to Discover’s observability tool, data warehouse, and source code repositories.

The solution is straightforward to use and is designed to be self-serving for data scientists. “A data scientist can go to the solution automation tool, select the template, and select the compute and Amazon EC2 instance based on their requirement—multi-GPU, single GPU, or a memory-bound application,” says Gupta. “They just select, click, and they are done.”

The solution helped Discover reduce the time to insights. Using feature embedding, the team reduced the time to market from hours to minutes. The high compute power available was used for parallel model training, which reduced the processing time of 30 million records from days to hours. For sentiment analysis—for example, to analyze whether a customer was satisfied or dissatisfied after talking to a customer service agent—the solution helped reduce the processing time of a 57,000-record dataset from hours to minutes.

The Discover team put the solution to work for a use case to manage the bank’s “do not contact” model. For customers who did not want the bank’s representatives to contact them for marketing and similar purposes, the team created a model to classify such customers. The solution classified these customers in near real time and made the relevant data available to customer care agents. The agents could thus identify customers who should not be contacted, which helped increase customer satisfaction.

“The team did a great job adapting and matching the speed with the risk,” says Jason Strle, EVP CIO at Discover. “When generative AI is helping in a human-in-the-loop scenario, it reduces risk and allows for more speed in the delivery. This can be contrasted to scenarios where a generative AI solution autonomously interacts with a customer or otherwise makes a business decision. In these cases, there are more risk steps to get to production.”

The Discover team now wants to explore using Amazon S3 to add event-based activations for further automation. It currently uses a scheduler to schedule daily tasks, such as identifying and classifying customers. Additionally, Discover is looking to event-based activations with queueing mechanisms to repurpose the compute for other generative AI use cases. This will help further reduce the compute costs for its generative AI use cases.

“This solution, based on GPU-powered Amazon EC2 instances, is helping us reduce risk and improve the customer experience,” says Gupta.