AWS for Industries

Reinventing CPQ with Knowledge Base and Chatbot at Zilliant

Co-authored by Falk Brauer, SVP, Product & Technology – CPQ, Zilliant, and Joshua Tan, Account Manager, AWS

Excellent customer experience is a key reason why customers choose and continue to use a product. This is also true for software-based applications. An experience which allows productive use of the end users’ time is widely preferred. With the rise of generative AI, Zilliant saw potential to integrate it into its existing CPQ (Configure, Price, Quote) manufacturing solution, transforming the user experience and bringing additional value to customers.

Customers are using Zilliant’s CPQ manufacturing solution to manage their engagements with buyers. The buyers are looking for manufacturing-related products, such as forklifts, which often contain swathes of technical information. For example, a buyer may be looking for forklifts that satisfy requirements regarding rated capacity, fork dimensions, and lifting speed. With numerous models available, each with its own technical specifications, many sellers require manual searches across product manuals to provide answers. This leads to potential delays for the buyer in getting accurate information to configure, price, and quote. In critical decision-making situations, taking too long to provide a quote may result in a lost opportunity for the seller, on top of the time already invested in the engagement.

A knowledge base-augmented Chatbot system integrated into the CPQ’s user interface (UI) can help reduce the time and work needed to obtain information about the products a user needs to build a quote. Users can easily question the chatbot with natural language, obtain a fast and accurate answer, and access the source data hyperlink to the product manual where the information is obtained. Users can also leverage the chatbot’s capability to understand context from previous messages within the chat to increase productivity without having to repeat the context for every question. The potential time and effort saved for both buyers and sellers with this solution is very substantial.

In this blog, we will dive into how Zilliant incorporated Amazon Bedrock Knowledge Bases into the CPQ manufacturing solution and how they implemented the chatbot solution to improve customer experience.

Incorporating Amazon Bedrock into the CPQ solution

Zilliant initiated development of the knowledge base-augmented Chatbot system with a solution using Amazon Bedrock, a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies. It provides a serverless way of accessing the FMs, allowing Zilliant to use FMs without having to provision or maintain the underlying infrastructure. This gives Zilliant the option to only pay for what they use with Amazon Bedrock on demand pricing model.

Since Zilliant needed a retrieval augmented generation (RAG) solution, it decided to use Amazon Bedrock Knowledge Bases. With Amazon Bedrock Knowledge Bases, you can give FMs contextual information from your data. In the next sections, we will discuss how Zilliant created the knowledge base used in the system and how the Question Retrieval and Response Generation works.

Knowledge Base Creation

The image below shows the architecture of the data ingestion part of the knowledge base-augmented chatbot system that Zilliant adopted.

The product manuals in PDF format are uploaded to Amazon Simple Storage Service (Amazon S3) by users through the CPQ application. Any new documents the customer creates can be easily uploaded to the same Amazon S3 bucket through the same location any time. In addition to Amazon S3, there are other supported data sources that can be ingested into the knowledge base as listed in this AWS documentation, such as Confluence whose data source connector support is available in preview release. Data can be ingested into the knowledge base by users’ ad hoc requests or by scheduled jobs. The ingestion is programmatically done by the CPQ application that calls StartIngestionJob API. The ingestion can be performed multiple times to ingest newly added documents in the data source.

The Amazon Bedrock Knowledge Bases perform parsing of the ingested documents according to the configuration. There are some parsing options available, including those that can parse images, charts, diagrams, and tables. Zilliant uses the default parsing option since the data is in text form stored in PDF files.

The Amazon Bedrock Knowledge Bases also performs chunking according to the configuration. You can choose to use standard (with fixed-size and default options), semantic, or hierarchical chunking strategies. You can also opt for no chunking. Zilliant uses the default chunking configuration to split the product manuals into text chunks of approximately 300 tokens while honoring sentence boundaries. Proper chunking allows the data to be more specific when being represented in embedding form in the vector database, which can improve relevancy during retrieval.

Amazon Titan text embedding model performs the task to convert the text chunks into embedding representations that are stored in the vector database. The Amazon Bedrock Knowledge Bases invokes this model and store the resulting embeddings into an Amazon OpenSearch Serverless as the vector database.

Amazon OpenSearch Serverless allows customers to run petabyte-scale workloads without configuring, managing, and scaling OpenSearch clusters. This is ideal for Zilliant’s use case because the project started with a quick proof-of-concept and Zilliant needed to alleviate the heavy lifting of setting up infrastructure, allowing the engineers to focus on building the chatbot.

Amazon OpenSearch Serverless also supports hybrid search. The hybrid search combines keyword-based search and semantic search to enhance the relevance of returned search results. This works well for Zilliant because there can be specific product characteristics that cannot be differentiated using a semantic-only search. For example, a question about the temperature range of a particular forklift model may semantically be similar to a paragraph explaining the temperature range for another model type. This could potentially return irrelevant results in semantic-only search. A combination of semantic and keyword-based search can be more reliable in this scenario.

Question Retrieval and Response Generation

After the data is ingested, the chatbot is ready to answer questions. The process is shown in the diagram below. The numbering indicates the order of the processes.

When users have questions about products, they input the question into the chatbot in Zilliant’s CPQ application UI. The application then calls RetrieveAndGenerate, an API call on Amazon Bedrock, to process the information and return an answer.

The “Retrieve” part of the process starts by converting the question text into an embedding representation with Amazon Titan text embedding model to generate the embedding vector. Then, the embedding vector is sent into Amazon OpenSearch Serverless to perform a hybrid search in the vector database to find relevant information chunks. Amazon Bedrock Knowledge Bases retrieves the relevant text chunks from Amazon OpenSearch Serverless. The “Generate” part of the process combines conversation context with the relevant information chunks and sends it into an Anthropic Claude large language model to generate the final answer to the corresponding session.

In order to maintain conversation context and knowledge from previous interactions, Zilliant adds the sessionId attribute to the RetrieveAndGenerate API call. This is important for the chatbot application to provide a more natural experience from the chat responses.

From idea to solution

From ideation, the Zilliant team collaborated with the Amazon Web Services (AWS) account team to plan the solution and the architecture. Since Zilliant wanted a personalized chatbot solution, they chose a RAG architecture.

The Zilliant team performed a proof-of-concept as guided by the AWS team with a samples of forklift manuals. During the proof-of-concept, the AWS Console experience managed the Amazon Bedrock Knowledge Bases, including data ingestion and sync, configuration, and testing of the solution.

The proof-of-concept was then extended with the use of Amazon Bedrock Knowledge Bases API to integrate with the CPQ application. Zilliant’s engineering team, who built the solution, consisted of software engineers, a solutions architect, and a senior executive sponsor. The solution demo was subsequently showcased at AWS Summit Singapore in May 2024. The feature launched in beta by Zilliant in August 2024. The image below shows the chatbot experience in the CPQ application.

While feedback on the experience of the feature is still being collected, Zilliant expects improvements in self-service issue resolution rates, customer satisfaction scores, and a reduction in operational costs.

Conclusion

This new solution in the CPQ application allows Zilliant to transform and enhance their customer experience by leveraging generative AI on AWS. Going forward, Zilliant and AWS will continue to continuously improve the chatbot solution.

First, Zilliant can use a hierarchical chunking strategy to improve search relevancy. Hierarchical chunking allows the retrieval to include larger bodies of text for additional context when answering more granular questions about a particular product, such as the question about the operating voltage for a specific product’s model which could be similar with that of other models. Next, it can switch to using a newer-yet-lighter foundation model like Claude 3.5 Haiku model in Amazon Bedrock to improve latency while keeping performance. Finally, Zilliant can add an integration with Amazon Bedrock Agents to allow more complex CPQ tasks, such as accessing and updating real-time data, to be automated and served from the chatbot itself.

Learn more about how you can leverage generative AI on AWS to transform your customers’ experience.

Yudho Ahmad Diponegoro, MComp

Yudho Ahmad Diponegoro, MComp

Yudho is a Senior Solutions Architect at AWS. Having been part of Amazon for 8+ years, he has had various roles from software development to solutions architecture. He helps startups in Singapore when it comes to architecting in the cloud. While he keeps his breadth of knowledge across technologies and industries, he focuses in AI and machine learning where he has been guiding various startups in ASEAN to adopt machine learning at AWS.