AUDI AG Slashes Response Times with Generative AI Chatbot Powered by Storm Reply and AWS

To simplify and standardize cloud usage across the organization, German premium and luxury vehicle manufacturer AUDI AG created a Cloud Foundations Team. The team is charged with ensuring consistent security, compliance, and quality standards across AWS accounts companywide. Part of that mandate includes creating documentation internal teams can use to work more efficiently and make better-informed decisions.

Over time, the amount of documentation has grown significantly, making it difficult for people to find what they need quickly without having to wade through pages of irrelevant information. This growth also created more work for the Cloud Foundations Team, who were answering the same questions over and over.

To get internal customers accurate, relevant information fast—and to free up the Cloud Foundations Team for higher-value activities—Audi turned to AWS Premier Consulting Partner Storm Reply to build a secure generative artificial intelligence (AI) chatbot on AWS.

Part of the Reply group, Storm Reply specializes in the design and implementation of cloud-based solutions. Storm Reply has worked with AWS since 2008, completing hundreds of successful cloud projects across the entire AWS technology stack.

The generative AI chatbot developed for the Cloud Foundations Team uses Llama 2 large language models (LLMs) on Amazon SageMaker. The solution employs Retrieval Augmented Generation (RAG) technology, which combines vector databases and LLMs to make the search of large amounts of data more structured and efficient.

“Thanks to Storm Reply, we were able to move from idea to minimum viable product in only four weeks,” said Bernd Schuster, cloud solutions architect at Audi’s Cloud Foundations Team. “Reply is our partner of choice when it comes to building on AWS.”

To test the initial chatbot, Schuster’s team uploaded 80 gigabytes of internal documentation. “We looked at the most frequently asked questions, and we achieved nearly 90 percent accuracy from the very beginning,” Schuster said. Because the chatbot provides accurate answers tailored to the asker’s specific question, Schuster’s team has seen a decrease in the number of repeat requests for information.

Since implementing the chatbot, response times to FAQs have dropped to six seconds. “Our customers used to have to submit an internal ticket and then wait for us to respond. Now they ask the chatbot and get exactly what they need in 100 characters, without having to go through pages of documentation,” said Schuster.

Going forward, the Cloud Foundations Team will continue to tune the model with the help of Storm Reply’s data science experts. The team is also planning to see how fully managed Amazon Bedrock compares to Amazon SageMaker. “Thanks to Storm Reply and AWS, our Cloud Foundations Team can focus on what it does best,” Schuster concluded. “That’s coding and making great cars.”