QnABot on AWS

Step 9A
If pre-processing guardrails are enabled, they scan and block potentially harmful user inputs before they reach the QnABot application. This acts as the first line of defense to prevent malicious or inappropriate queries from being processed. 

Step 9B
If using Amazon Bedrock guardrails for LLMs or Amazon Bedrock Knowledge Bases, it can apply contextual guarding and safety controls during LLM inference to ensure appropriate answer generation. 

Step 9C
If post-processing guardrails are enabled, they scan, mask, or block potentially harmful content in the final responses before they are sent to the client through the fulfillment Lambda. This serves as the last line of defense to ensure that sensitive information, such as personally identifiable information (PII) is properly masked and inappropriate content is blocked.

Step 10
If no match is returned from the OpenSearch Service question bank or text passages, the Bot fulfillment Lambda function forwards the request as follows:

Step 10A
If an Amazon Kendra index is configured for fallback, then the Bot Fulfillment Lambda function forwards the request to Amazon Kendra if no match is returned from the OpenSearch Service question bank. Optionally, the text generation LLM can be used to create the search query and to synthesize a response from the returned document excerpts.

Step 10B
If an Amazon Bedrock Knowledge Base ID is configured, the Bot Fulfillment Lambda function forwards the request to the Amazon Bedrock Knowledge Base. The Bot Fulfillment Lambda function then uses the RetrieveAndGenerate or RetrieveAndGenerateStream APIs to fetch the relevant results for the user's query, augment the foundational model's prompt, and return the response.

Step 11
When streaming is enabled, the LLM responses from text passages or external data sources are enhanced by Retrieval-Augmented Generation (RAG). Responses are streamed through WebSocket connections using the same Amazon Lex sessionId, while the final response is processed through the fulfillment Lambda function.
 

Step 12
User interactions with the Bot Fulfillment function are logged, and the resulting metrics data is sent to Amazon Data Firehose, then forwarded to Amazon S3 for later data analysis.
 

Step 13
The OpenSearch Dashboards can be used to view a variety of analytics, including usage history, logged utterances, no-hit utterances, positive user feedback, negative user feedback, and the ability to create custom reports.

Step 14
Using Amazon CloudWatch, the admins can monitor service logs and use the CloudWatch dashboard created by QnABot to monitor deployment’s operational health.
 

Step 1
The admin deploys the solution into their AWS account, opens the content designer user interface (UI) or Amazon Lex web client, and uses Amazon Cognito to authenticate.

Step 2
After authentication, Amazon API Gateway and Amazon Simple Storage Service (Amazon S3) deliver the contents of the content designer UI.

Step 3
The admin configures questions and answers in the content designer and the UI sends requests to API Gateway to save the questions and answers.

Step 4
The Content designer AWS Lambda function saves the input in Amazon OpenSearch Service in a question bank index. If using text embeddings, these requests pass through LLMs hosted on Amazon SageMaker or Amazon Bedrock to generate embeddings before being saved into the question bank on the OpenSearch Service.

In addition, the Content designer saves the default and custom configuration settings in the Amazon Dynamo DB.

Step 5
Users of the assistant interact with Amazon Lex through the web client UI, Amazon Alexa, or Amazon Connect.

Step 6
Amazon Lex forwards requests to the Bot Fulfillment Lambda function. Users can also send requests to this Lambda function through Amazon Alexa devices.

NOTE: When streaming is enabled, the chat client uses the Amazon Lex session identifier (sessionId) to establish WebSocket connections through API Gateway V2.

Step 7
The user and chat information are stored in DynamoDB to disambiguate follow up questions from the previous question and answer context.

Step 8
The Bot fulfillment Lambda function uses Amazon Comprehend and, if necessary, Amazon Translate to translate non-native language requests into the native language selected by the user during deployment. The function then queries the OpenSearch Service to retrieve the appropriate answer.

Step 9
If using large language model (LLM) capabilities such as text generation and text embeddings, these requests will first pass through various foundational models hosted on Amazon Bedrock. This will generate the search query and embeddings, which will then be compared against those saved in the question bank on the OpenSearch Service.