AWS Machine Learning Blog

Create a Question and Answer Bot with Amazon Lex and Amazon Alexa

Update November 2019 (v2.4.0): See New features

Your users have questions and you have answers, but you need a better way for your users to ask their questions and get the right answers. They often call your help desk, or post to your support forum, but over time this adds stress and cost to your organization. Could a chat bot add value for your customers? Interestingly, a recent poll shows that 44% of people would rather talk to a chat bot than to a human!

In this post we provide a sample solution, called QnABot (pronounced “Q and A Bot”). The QnABot uses Amazon Lex and Amazon Alexa to provide a conversational interface for your “Questions and Answers.” This allows your users to ask their questions and get quick and relevant answers.

Amazon Lex lets you integrate both voice and text chat access into your existing applications. Amazon Alexa allows you to offer a hands-free voice interface to your users in their homes or workplaces using Amazon Echo or any Alexa-Voice-Service-enabled device. QnABot offers the best of both worlds.

QnABot uses the Amazon Elasticsearch Service (Amazon ES) to make your questions and answers searchable. When a user asks a question, the Amazon ES powerful full-text search engine is used behind the scenes to find the answer that is the best match for that question.

In the sections that follow we’ll show you how to do the following:

  • Deploy QnABot to your AWS account. This blog assumes that you are already an AWS customer. If you want to open an account, choose the Create an AWS Account button on the AWS home page.
  • Populate the QnABot with your questions and answers using the Content Designer UI.
  • Ask questions using voice or chat with the Web Client UI.
  • Ask questions hands-free using the latest Amazon Echo devices.
  • Enrich your answers with images, rich text, and SSML.
  • Support follow-up questions.
  • Support variables and conditional content in your answers.
  • Extend QnABot to integrate with other services and deliver dynamic answers.
  • Test, tune, and troubleshoot QnABot content to minimize the chances of getting wrong answers.
  • Modify QnABot configuration settings.
  • Monitor QnABot Usage and User Feedback.

We’ll also take a look under the hood to show you how QnABot works, and give you some ideas for how you can enhance it to suit your needs.


Contents


 

Deploying QnABot

Use this AWS CloudFormation button to launch QnABot in AWS Region us-east-1:

Note: The source code is available in our GitHub repository. Follow the directions in the README to deploy QnABot to additional AWS regions supported by Amazon Lex – see Region Table.

Open the AWS CloudFormation console, and on the Parameters page, enter a valid Email and Username for the QnABot Content Designer admin user.

Choose Next twice to display the Review page.

Select the acknowledgement checkboxes, and choose Create stack to deploy QnABot.

The master CloudFormation stack uses nested stacks to create the following resources in your AWS account:

When the deployment is complete (after about 30 minutes), the master stack Output tab shows the following:

  • ContentDesignerURL: URL to launch the Content Designer UI.
  • ClientURL: URL to launch the user client webpage.
  • DashboardUrl: URL to launch the CloudWatch dashboard for monitoring.

You will receive an email at the email address you provided with the subject “QnABot Signup Verification Code.” This email contains a generated temporary password that you can use to log in and create your own password. Your new password must have a length of at least 8 characters, and contain upper-case and lower-case characters, plus numbers and special characters.

Creating initial content in QnABot

You need to load some question and answer data using the Content Designer, before turning QnABot over to your users. Your data is stored in Amazon ES behind the scenes. This way, the data can be searched later when users ask questions using either an Amazon Lex client UI or an Alexa hands-free device. The steps that follow show you how to load sample questions used to illustrate the features of the QnABot. You can easily substitute your own content later.

  1. In the CloudFormation console, launch the Content Designer webpage using the ContentDesignerURL link from the Outputs tab of the master CloudFormation stack.
  2. Log in with the administrator username you provided when you launched the stack (default: ‘Admin’), and your new password.
  3. Choose Add.
  4. Enter the id: QnaBot.001
    NOTE:
    Use a naming convention to identify your items within categories.
  5. Enter the question: What is Q and A Bot
  6. Enter the answer: The Q and A Bot uses Amazon Lex and Alexa to provide a natural language interface for your FAQ knowledge base, so your users can just ask a question and get a quick and relevant answer.
  7. Choose the CREATE button (bottom right of the card).
  8. Repeat steps 3-7, entering the items from the table below.
    Alternatively, to avoid the copy/paste process, you can import the items directly from a file. Choose Import from the top left tools menu ( ), then enter the URL below, and choose IMPORT: https://raw.githubusercontent.com/aws-samples/aws-ai-qna-bot/master/assets/examples/documents/blog-samples.json
  9. When the import is complete, choose Edit from the top left tools menu ( ), and then choose LEX REBUILD from the top right edit card menu ( ⋮ ). NOTE: We will describe the purpose of the LEX REBUILD later in this post.
Id Question Answer
QnABot.002 How do I use Q and A bot Create and administer your questions and answers using the Q and A Bot Content Designer UI. End users ask questions using the Amazon Lex web UI that supports voice or chat, or using Alexa devices for hands free voice interaction.
Admin.001 How do I modify Q and A Bot content Use the Content Designer Question and Test tools to find your existing documents and edit them directly in the console. You can also export existing documents as a JSON file, make changes to the file, and re-import.
Admin.002 Can I back up Q and A Bot content Yes. Use the Content Designer to export your content as a JSON file. Maintain this file in your version control system or in an S3 bucket. Use the Designer UI Import feature to restore content from the JSON file.
Admin.003 Can I import Q and A Bot content from a file Yes, the Content Designer has an import function that lets you load items from a formatted JSON file. You can create JSON files using the Export feature, or you can write your own tools to create JSON files from existing content such as a website FAQ page.
Admin.004 How do I troubleshoot and fix problems with Q and A Bot. Use the Content Designer test tool to test a question, and check what items are returned, ranked in order of score. If the desired item does not have the highest score, then add the question to the item and run the test again. The desired item should now have the highest score. Take care not to create items with duplicate questions, to avoid unpredictable responses.
Admin.005 How can I find specific Q and A items in the Designer UI Use the Filter feature in the Questions tab to filter the items list based on the ID field. Or use the Test tab to list all the items that match a question.
Media.001 How can I include pictures in Q and A Bot answers Add an image attachment to the item using the Content Designer.

Getting answers using a Lex Web Client UI

You can launch QnABot from a Chrome, Firefox, or Microsoft Edge browser on your PC, Mac, Chromebook, or Android tablet.

  1. On the CloudFormation console, select the main QnABot stack, choose Output, and then choose the link to the ClientURL. Alternatively, launch the client by choosing QnABot Client from the Content Designer tools menu ( ).
  2. When your browser requests access to the microphone on behalf of the web application, allow it. You’ll see the QnABot chat window:

    You can interact with QnABot using either text chat or voice.
  3. Try it! Choose the microphone icon (bottom right) and say, “What is Q and A Bot?”
    The bot will respond with the answer you previously entered for this question.

 

Getting answers using Alexa

QnABot also works with Alexa, allowing your users to get answers from your content via any Alexa enabled device, including Amazon FireTV, Amazon Dash Wand, and any of the Amazon Echo family of devices.

To enable Alexa, first create an Alexa skill for QnABot using the Amazon Developer Console. QnABot can’t automatically create the Alexa skill at this time, so we’ve provided instructions. Log in to the QnABot Content Designer (as explained earlier), and choose Alexa from the tools menu ( ):

Follow the displayed instructions, step by step, to create an Alexa skill in your Amazon account. It will take you just a few minutes. You can test your new skill in the Amazon Developer Console, even if you don’t have an Alexa enabled device handy.

If you want to publish your QnA skill to the Alexa skills store so other users can access it, see Submitting an Alexa Skill for Certification. Unpublished skills are accessible only to Alexa devices registered to your Amazon account, while published skills are available to anyone.

Adding images to your answers

You can easily augment your answers with image attachments that can be displayed on a user’s Lex Web Client UI, Amazon Alexa smartphone app, or new Amazon Echo Show device touch screen. Use images to display maps, diagrams, or photographs to depict places and products relevant to the question.

  1. Log in to the Content Designer, and choose Add
  2. Enter ID: Alexa.001
  3. Enter question: What is an Amazon Echo Show
  4. Enter answer: Echo Show brings you everything you love about Alexa, and now she can show you things. She is the perfect companion for Q and A Bot.
  5. Choos Advanced.
  6. Enter Response card:
    1. Card Title: Echo Show
    2. Card ImageUrl: https://images-na.ssl-images-amazon.com/images/I/61OddH8ddDL._SL1000_.jpg
  7. Choose CREATE to save the new item.
  8. Use the Web UI to ask: “What is an Echo Show?”
    You’ll see the photograph displayed in the web UI chat.
  9. Or, use an Amazon Echo or Echo Dot to say: “Ask Q and A, What is an Echo Show?” The card shown in the Alexa smartphone app shows the photo attachment.
  10. Or, best yet, use an Amazon Echo Show to say: “Ask Q and A, What is an Echo Show?”
    The photo attachment is displayed on the Echo Show’s touch screen.

Use Amazon S3 or a photo sharing service to host your images (see hosting images).

Displaying rich text answers on the web UI

The latest version of QnABot supports Markdown, allowing you to create beautiful rich text versions of your answers for displaying on the Web UI client. To use this feature, populate the ‘Alternate Markdown answer’ field in Content Designer.

  1. In Content Designer, edit item QnABot001 (“What is Q and A Bot”). Open the Advanced section and enter the following text in the Markdown Answer field:
    # QnaBot
    The Q and A Bot uses [Amazon Lex](https://aws.amazon.com/lex) and [Alexa](https://developer.amazon.com/alexa) to provide a natural language interface for your FAQ knowledge base.
    Now your users can just ask a *question* and get a quick and relevant *answer*.
  2. Choose UPDATE to save the modification:
  3. Use the Web UI to ask: “What is Q and A bot?”.
    You will see that the answer now displays the heading, links, and emphasis specified in your Markdown text.
  4. QnABot also supports inline HTML in the Markdown field. This is very powerful, and allows you to do cool things, like including video clips in your answers. Let’s try it – choose ADD to create a new item:
    1. Enter ID: FireTV.001
    2. Enter question: What is Amazon Fire TV?
    3. Enter answer: Fire TV brings all the live TV and streaming content you love, and Alexa, onto the big screen. Use Alexa on the Fire TV to bring QnABot into your living room!
    4. Enter Markdown answer:
      **Fire TV** brings all the live TV and streaming content you love, and Alexa, onto the big screen. Use Alexa on the Fire TV to bring QnABot into your living room!
      <iframe src="https://www.youtube.com/embed/OE4MrFx2XCs"></iframe>
    5. Choose CREATE to save the item
    6. Use the Web UI to ask: “What is Amazon Fire TV?”.

Markdown support currently works only with the latest version of our chatbot web UI.

Using SSML to control speech synthesis

The latest version of QnABot supports SSML, giving you additional control over the speech generation for your response. To use this feature, populate the ‘SSML answer’ field in Content Designer.

  1. In Content Designer, edit item QnABot001 (“What is Q and A Bot”). Open the Advanced section and enter the following text in the SSML Answer field:
    <speak>QnABot is <emphasis level="strong">great</emphasis>. But I want to tell you a secret. <amazon:effect name="whispered">I am not a real human.</amazon:effect>. Can you believe it? </speak>

    Choose UPDATE to save the modification:

  2. Use the Web UI to ask, using voice: “What is Q and A bot?”, and listen to the whispered response.

Using SSML answers gives you control over voice responses for both Lex and Alexa.

When you are using the web UI, you can use both Markdown and SSML to get QnABot to say one thing (SSML) while displaying another (Markdown). Use this feature to deliver short punchy verbal answers, while taking advantage of the web UI to display rich, illustrated answers.

Using Topics to support follow-up questions

QnABot remembers the topic from the last question you asked, allowing you to ask follow-up questions, for example: “How much does it cost?” The correct answer depends on the context set by the previous question. To use this feature, you assign a value to the Topic field in Content Designer.

  1. In Content Designer, edit item Alexa.001 (“What is an Amazon Echo Show”), and enter “EchoShow” as the Topic in the Advanced section. Choose UPDATE (bottom right) to save the item.
  2. Edit item QnABot.001 (“What is Q and A bot”), and enter “QnABot” as the Topic. Choose UPDATE to save the item.
  3. Choose ADD to create a new item for our first follow-up question:
    • Enter ID: Alexa.Cost
    • Enter question: How much does it cost?
    • Enter answer: For latest prices on the Echo Show, see the Amazon retail site or shopping app.
    • Enter topic: EchoShow
    • Choose CREATE to save the item
  4. Choose ADD to create a new item for our next follow-up question. Enter the following values:
    • Enter ID: QnABot.Cost
    • Enter question: How much does it cost?
    • Enter answer: Q and A Bot is priceless
    • Enter topic: QnABot
    • Choose CREATE to save the item
  5. Use the Web UI to ask the following questions and observe the context appropriate answers:
    1. “What is an Echo Show?”
      • The answer to this question now sets the conversation topic to ‘EchoShow’.
    2. “How much does it cost?”
      • The topic disambiguates this question, so it responds with the answer for the Echo Show.
    3. “What is the Q and A bot?”
      • This question changes the Topic to ‘QnABot’.
    4. “How much does it cost?”
      • The new topic allows the QnABot to respond with the correct answer.

Displaying buttons on the web UI

Add buttons to your answers to help guide your user, by suggesting what they might want to do next. You define and attach buttons to response cards. Let’s try it:

  1. In Content Designer, edit item Alexa.001 (“What is an Amazon Echo Show”)
  2. Find Lex Buttons under Response card in the Advanced section
  3. Enter Display Text: QnABot
  4. Enter Button Value: What is Q and A bot?
  5. Choose ADD LEX BUTTON to add another button
  6. Enter Display Text: FireTV
  7. Enter Button Value: What is Amazon Fire TV?
  8. Choose UPDATE to save the item with your new buttons
  9. Use the Web UI to ask: “What is an Echo Show?
  10. Choose one of the new buttons to automatically send the next question to QnABot:

Adding style to QnABot

As you define your content, be sure to make your answers light and engaging. Try to adapt a consistent style in your answers. Use images to augment verbal answers. Add flourish to make your bot friendly and stylish!

For example, teach it how to respond to chit-chat like ‘Hello,’ Bye-bye,’ ‘Thanks’ and more.

This is easy to try:

  1. Log in to the Content Designer, and choose Add.
  2. Enter ID: BotStyle.001
  3. Enter question: Hello
  4. Enter answer: Greetings friendly human! Ask me a question. Try to stump me.
  5. Save the new item.
  6. Use the Web UI, or any Alexa device to say “Hello” to your bot, and listen to it respond.

You should also use this technique to teach QnABot to respond with some brief instructions when your users say “Help.”

Using variables and conditional content with Handlebars

QnABot now lets you use Handlebars templates in your answers, including the Markdown and SSML fields, so you can include variable substitution and conditional elements. Let’s try a simple example to illustrate the concept:

  1. Log in to the Content Designer, and choose Add.
  2. Enter ID: Handlebars.001
  3. Enter question: What is my interaction count?
  4. Enter answer: So far, you have interacted with me {{UserInfo.InteractionCount}} times.
  5. Save the new item.
  6. Use the Web UI, or any Alexa device to say “What is my interaction count?” to your bot, and listen to it respond.
  7. Ask a few more questions, and then ask “What is my interaction count?” again. Notice that the value has increased.
  8. In Content designer, edit item Handlebars.001
  9. Modify answer to:

    So far, you have interacted with me {{UserInfo.InteractionCount}} times.
    {{#ifCond UserInfo.TimeSinceLastInteraction ‘>’ 60}}
    It’s over a minute since I heard from you last.. I almost fell asleep!
    {{else}}
    Keep those questions coming fast.. It’s been {{UserInfo.TimeSinceLastInteraction}} seconds since your last interaction.
    {{/ifCond}}

  10. Use the Web UI, or Alexa, to interact with the bot again. Wait over a minute between interactions and observe the conditional answer in action.

Use the Chatbot Web UI integrated Cognito log in feature to authenticate your QnABot users. When users are authenticated, the chatbot UI attaches a signed token as a session attribute. QnABot uses the token to make additional user attributes available, including fields such as Given Name, Family Name, and Email address.

There’s a lot more that you can do with handlebars, such as randomly selecting content from a list, and accessing session attributes. For more information see HANDLEBARS_README.md in the GitHub repository.

Extending QnABot with Lambda hook functions

Content Designer now gives you the ability to dynamically generate answers by letting you specify your own Lambda ‘hook’ function for any item. When you specify the name, or ARN, of a Lambda function in the Lambda Hook field for an item, QnABot will call your function any time that item is matched to a user’s question. Your Lambda function can run code to integrate with other services, perform actions, and generate dynamic answers.

QnABot comes with a simple Lamdba Hook function example that you can explore:

  1. Log in to the Content Designer, and choose Import from the tools menu ( ☰ ).
  2. Open Examples/Extensions, and choose LOAD from the ‘GreetingHook’ example.
  3. When the import job has completed, return to the edit page, and examine the item “GreetingHookExample.” Note that the Lambda Hook field is populated with a Lambda function name.
  4. Use the Web UI to say “What are lambda hooks?”. Note that the answer is prepended with a dynamic greeting based on the current time of day – in this case ‘good afternoon’:
  5. Inspect the example function (ExampleJSLambdahook) using the AWS Lambda console.

Choose Lambda Hooks from the Content Designer tools menu ( ) to display additional information to help you create your own Lambda hook functions.

See Extensions README for more information on how you can package lambda hooks in your own copy of the QnABot GitHub repository.

Keyword filters and custom “Don’t Know” answers

Keyword filters

The new keyword filter feature helps QnABot to be more accurate when answering questions, and to admit more readily when it doesn’t know the answer.

The keyword filter feature works by using Amazon Comprehend to determine the ‘part of speech’ that applies to each word you say to QnABot. By default, nouns (including proper nouns), verbs, and interjections are used as ‘keywords’. Any answer returned by QnABot must have at least one question that matches these keywords, using the following (default) rule:

  • if there are 1 or 2 keywords, then all keywords must match.
  • if there are 3 or more keywords, then 75% of the keywords must match.

If QnABot can’t find any answers that match these keyword filter rules, then it will admit that it doesn’t know the answer rather than guessing an answer that doesn’t match the keywords. QnABot logs every question that it can’t answer so you can see them in the Kibana Dashboard which we’ll show you a little later.

The keyword filters feature is enabled by default, but if it causes problems for you, you can customize its settings, or disable it altogether – see Modifying Configuration Settings.

Custom “Don’t Know” answers

When QnABot can’t find an answer, by default you’ll see or hear the response, “ You stumped me! Sadly I don’t know how to answer your question“. You can easily customize this answer by creating a new item in Content Designer, called the ‘no_hits’ item. Let’s try it. In Content Designer, choose ADD to create a new item:
  1. Enter ID: CustomNoMatches
  2. Enter question: no_hits
  3. Enter answer: Terribly sorry, but I don’t know that one. Ask me another.
  4. Optionally enter provide Markdown and SSML alternatives for your answer, as described earlier.
  5. Choose CREATE to save the item.
  6. Use the Web UI to ask: “What are Echo Buds?”
  7. For bonus points, teach QnABot to answer the question about the Echo Buds, so it can be smarter next time!

Tuning, testing, and troubleshooting

Tuning answers using the Content Designer

QnABot attempts to match a user’s question to the list of questions and answers stored in Amazon Elasticsearch service by the Content Designer. QnABot uses full text search to find the item that is the best match for the question asked. Words that are used infrequently score higher than words that are used often, so sentence constructs such as prepositions have lower weighting than unique keywords. The closer the alignment between a question associated with an item and a question asked by the user, the greater the probability that QnABot will choose that item as the most relevant answer.

QnABot tries to find the best answer to questions by applying the keyword filters discussed above and also by matching the user’s question to both the question and the answer fields of the stored items, giving more weight to any matches found in the question field.

Scores for matches across multiple questions in an item are additive. To avoid undesirable skew in the scoring, make your questions as unique as possible and limit repeating words or phrases.

You might find that users ask questions in ways that you haven’t anticipated, resulting in unexpected answers being returned by QnABot. When this happens, use the Content Designer to troubleshoot and fix the problem.

To illustrate an example troubleshooting / tuning scenario, let’s use the item we entered in the previous section – Admin.001 “How do I modify Q and A Bot content”:

  1. Use the Lex Web UI (voice or text chat) to ask the question in a different way: How can I edit items?
  2. Ideally you want the answer for item Admin.001, but in this case, since the question is phrased very differently than the question associated with this item, you don’t get the desired answer.
  3. Log in to the Content Designer, and choose TEST
  4. Enter the same question you asked, “How can I edit items”, into the question field, and choose SEARCH. There are no results, because none of the questions pass the keyword filter.
  5. In the QUESTIONS tab, find and choose Admin.001. This is our preferred answer.
  6. Let’s tune this item, so that next time you ask the same question this item will have the top score. Choose the item to expand it, choose the edit tool (right), and then ADD QUESTION:
  7. Enter the new question: How can I edit items, and choose the UPDATE button (bottom right of card).
  8. In the TEST tab try the same question again.
  9. Success! This time Admin.001 is a match, and has the highest score.
    Try asking the question again in the Lex Web UI. This time you’ll get the desired answer.

Testing all your questions

QnABot now helps you to quickly test all your questions. Let’s try it:

  1. Log in to the Content Designer, and choose TEST ALL.
  2. Use the default Filename, or enter your own.
  3. If you want to test only a subset of questions, you can optionally filter by qid prefix. Leave this field blank to test all the questions.
  4. Choose the TEST ALL button, and wait for the tests to complete.
  5. Choose the view results icon () on the far right to view the test results.

Behind the scenes, a test function sends every question for every item to QnABot via Amazon Lex, and checks that QnABot matched the question to the expected item. Any incorrect matches are highlighted in red. Test results can be viewed in the browser, or downloaded to your computer as a CSV file.

Tuning the Bot’s Natural Language Processing

When you ask QnABot a question, it’s processed and transcribed by either Amazon Lex or Alexa using a Natural Language Processing (NLP) engine. QnABot initially trains the NLP to match a wide variety of possible questions and statements, so that our Lex bot and Alexa skill will accept just about any question a user might ask. (You can see the default training questions listed in the file default-utterances.js.)

In earlier versions of QnABot you might have found that sometimes you wanted to say things to QnABot that it didn’t understand and would ask you to repeat. In the latest (November 2019) version, this problem goes away! Now QnABot supports the new AMAZON.FallbackIntent in both Lex and Alexa, which allows it to process anything you might say without needing to retrain and rebuild the Lex bot or Alexa skill.

Occasionally you might notice that the transcription shown in the web client or the Alexa app isn’t accurate. This sometimes happens with unusual words that can be confused for other more common words or phrases. You have a couple of options for dealing with this. Either:

  • Use Content Designer to add additional question variants that match the actual transcription shown in the web client or in the Alexa app; this allows QnABot to anticipate the transcription accuracy problem, and respond anyway.
  • Or retrain the Lex and Alexa NLP with examples to influence the transcription to more closely match what you want – see below.

Retrain Amazon Lex

QnABot can automatically generate additional NLP training data for Lex using questions from all the items you have added.

Log in to the Content Designer. Choose LEX REBUILD from the top right edit card menu ( ⋮ ). Wait for the rebuild to complete.

Retrain Alexa

QnABot can generate additional NLP training data for Alexa using questions from all the items you have added.

  • Log in to the Content Designer.
  • Choose LEX REBUILD from the top right edit card menu ( ⋮ ). Wait for the rebuild to complete.
  • Then choose ALEXA UPDATE from the same menu ( ⋮ ).
  • Choose COPY SCHEMA to copy the updated Alexa skill schema.
  • Log in to the Alexa Developer Console, open your QnABot skill, and use the JSON Editor to paste the new schema, replacing the existing one.
  • Save and then Build the updated model.

 

Monitoring QnABot usage and user feedback

QnABot logs what users are saying to your bot. Behind the scenes, we use Amazon Kinesis Data Firehose to store logged utterances to a new index in the Amazon Elasticsearch service.

You can also allow your users to provide feedback on QnABot’s answers. To enable the feedback feature:

  1. Choose Import from the top left tools menu ( ☰ )
  2. Open Examples/Extensions, and choose the LOAD button for the QnAUtility demo.
  3. Choose Edit from the top left tools menu ( ☰ ), and examine the newly imported items, Feedback.001 and Feedback.002; observe the list of default expressions that the user can input to invoke feedback. (The example QnAUtility demo package also loads items Help and CustomNoMatches.)

Now that you have enabled the feedback feature, try it:

  1. Use the Web UI to ask a question, such as: “What happens if I ask an unanticipated question?”. Since we have not entered a suitable answer for this question, QnABot responds with the newly imported CustomNoMatches response indicating that it doesn’t know the answer.
  2. In the Web UI, say or type “Thumbs down”, or just click the thumbs new down icon ()beside the answer.

Visualize the usage logs and feedback using the Kibana dashboard. Note that it can take up to 5 minutes for new utterances and user feedback to become visible in the dashboard.

  1. Choose Kibana Dashboard from the top left tools menu ( ☰ ). Kibana opens in a new browser tab, showing usage history at the top, then all logged utterances, no hits utterances, positive user feedback, and negative user feedback at the bottom).
  2. Use Kibana to change the time span, customize and build your own visualizations, or to run your own queries.

Using CloudWatch to monitor and troubleshoot AWS resources

QnABot metrics and logs are available using an Amazon CloudWatch dashboard.

  1. Launch the dashboard using the DashboardUrl on the Outputs tab of the master CloudFormation stack.
  2. When troubleshooting QnABot responses to your questions, trace the request and response using the logs created by the Fulfilment Lambda function.
    1. Choose the menu tool in the upper right of the Fulfilment Lambda widget, choose View logs, and choose the AWSLambda function.

    2. Inspect the log messages. Each interaction with QnABot is delimited by START and END messages. Between these messages you’ll be able to find insights into how QnABot processes the question.

Exporting and importing answers

The QnABot Content Designer allows you to export and import your content using JSON files.

Use the export feature to create backup versions of your content that you can use to restore if you accidentally delete items or need to go back to a previous version. You can also use exported JSON files to load content into another instance of your bot to help with testing or blue/green deployments.

Or, you might want to automatically create JSON content files using scripts which scrape and reformat content using existing sources like FAQ web pages or documents.

Try it! Let’s export all the items that are in the QnABot category (items whose ID starts with “QnABot”).

  1. Log in to the Content Designer, choose the tools menu ( ) and Export
  2. Type ‘QnABot’ in the optional filter field, and choose EXPORT to generate a JSON file containing the filtered items. When the export has completed, choose the download tool (bottom right) to download the exported file:
  3. Open the exported file in a text editor and inspect the JSON structure.
  4. Add a new item to the qna list, as shown in the following example, and save the file.
    {
            "qid": "QnABot.003",
            "q": ["What can Q and A bot do"],
            "a": "You can integrate it with your website to provide quick and easy access to frequently asked questions. Use it with Alexa to provide hands free answers in the kitchen, in the factory or in the car. Since it can display images too, use it to provide illustrations and photographs to enrich your answers.",
            "r": {
                "title": "",
                "imageUrl": ""
            }
    }
    
  5. In the Content Designer, choose Import/Export, and From File. Import your modified JSON file from your local computer. NOTE: Importing items with the same ID as an existing item will overwrite the existing item with the definition contained in the JSON file.
  6. In the Content Designer, enter QnABot in the filter field, and inspect the newly imported item, QnABot.003.

Modifying configuration settings

QnABot now uses AWS Systems Manager Parameter Store to hold default and custom configuration settings.
Use the Outputs tab of the master CloudFormation stack to get the names of these parameters from the output keys:
  • DefaultSettingsSSMParameterName: name of the parameter that stores default settings
  • CustomSettingsSSMParameterName: name of the parameter that stores customized settings
In the AWS console, open Systems Manager Parameter Store. Find and choose the default settings parameter.
The parameter contains a JSON value with all the default settings for QnABot. Copy the JSON to a web site like jsonlint.com to view the parameter values in a more readable layout.

Customizing QnABot Settings

You should not edit the default settings parameter because your edits may be overwritten if you upgrade QnABot later. Instead, you should use the customized settings parameter to override any of the default values that you want to customize.

In the AWS console, open Systems Manager Parameter Store. Find and choose the custom settings parameter. The custom parameter value is an empty JSON object.

Choose Edit and replace the empty object with the following and then choose Save Changes:

{
“ES_USE_KEYWORD_FILTERS”: “false”
}

Congratulations, you have just disabled the new keyword-filters feature that we discussed earlier! For proof, let’s repeat an exercise we did earlier:
  1. Use the Web UI to ask a question that the bot can’t answer, such as: “What happens if I ask an unanticipated question?”.
  2. Observe that now, instead admitting it doesn’t know, QnABot simply returns the best match, which in this case is clearly not useful. Give it a thumbs down!
Enable the keyword filters feature again by once more editing the custom settings parameter, and change the value to:
{
“ES_USE_KEYWORD_FILTERS”: “true”
}
Test it again to make sure you successfully re-enabled the feature. QnABot admits that it doesn’t know the answer.
You can also customize how the keyword filters feature works, by changing:
  • ES_KEYWORD_SYNTAX_TYPES: A list of tokens representing parts of speech identified by Amazon Comprehend.
  • ES_MINIMUM_SHOULD_MATCH: A query rule used to determine how many keywords must match an item question in a valid answer.

You can also override any default settings by using custom settings to specify new values. Custom settings will not be replaced when you upgrade QnABot. A future release of QnABot will include a Settings page in the Content Designer to make it much easier to inspect and change settings.

 

Looking under the hood

Let’s take a quick look at how QnABot works. Here’s a diagram of the main architectural components and how they fit together at a high level.

The following sections explain how all these architectural components work together for different modes of interacting with QnABot. For additional detail, see the source code and READMEs in the GitHub repository, and use the AWS console to explore the configuration of each of the resources.

Amazon Lex web client

Amazon Lex allows conversational interfaces to be integrated into applications like our Lex web client. A Lex bot uses intents to encapsulate the purpose of an interaction, and slots to capture elements of information from the interaction. Since QnABot has a single purpose, to answer a user’s question, it defines just one intent. This intent has a single slot which is trained to capture the text of the question. QnABot also uses the new AMAZON.FallBackIntent to ensure that all user input is processed (see the discussion of “Tuning the Bot’s Natural Language Processing” earlier). To learn more about how Amazon Lex bots work, and to understand the concepts of intents, slots, sample values, fulfillment functions, and more, see the Amazon Lex Developer Guide.

The QnABot Lex web client is deployed to an Amazon S3 bucket in your account, and accessed via Amazon API Gateway.

An Amazon API Gateway endpoint provides run time configuration. Using this configuration, the web client connects to Amazon Cognito to obtain temporary AWS credentials, and then connects with the Amazon Lex service.

When you use the Lex web client to ask QnABot a question, for example, “How can I include pictures in Q and A Bot answers?”, the web client interacts with the QnABot Lex bot. The Lex bot then invokes the Bot fulfillment Lambda function in your AWS account with the transcribed question captured as a slot.

The Bot fulfillment Lambda function generates an Amazon ES query containing the transcribed question. The following example shows the query. The query attempts to find the best match from all the questions and answers you’ve previously provided, filtering items to apply the keyword filters and using Amazon ES relevance scoring to rank the results. Matches found in the text of stored questions (nested field: questions.q) are summed and given greater weight than matches found in the text of the stored answers (field: a). The topic value assigned to the previous answer (if any) will increase the overall relevance score for matches on topic value (field t).

The Bot fulfillment Lambda function uses the answer field from the item with the highest score to create a response to send back to Lex. If the item also contains an image attachment, a response card is also sent back in the response. The Lex web client responds to the user with either voice or text depending on the mode of operation, and also displays any images attached to the answer.

Alexa-enabled devices

Alexa devices interact with QnABot using an Alexa skill. Like a Lex bot, an Alexa skill also uses intents to encapsulate the purpose of an interaction, and slots to capture elements of information from the interaction. The QnABot Alexa skill mirrors the setup of the Lex bot described above.

The Alexa QnA skill uses the same Bot Fulfillment Lambda function as the Lex bot. When you ask a question, for example, “Alexa, ask Q and A, How can I include pictures in Q and A Bot answers?”, your Alexa device interacts with the skill you created, which in turn invokes the Bot fulfillment Lambda function in your AWS account, passing the transcribed question as a parameter.

The Bot fulfillment Lambda function queries Amazon ES items as described earlier. The Bot fulfillment Lambda function uses the answer field from the item with the highest score to create a response to send back to Alexa. If the item also contains an image attachment, a card is also sent back in the response, causing the image to be displayed in the Alexa smartphone app. If you use an Echo Show or Echo Spot the image will be displayed on the touchscreen, or if you use Alexa on your Amazon FireTV the answers and images will be displayed on your big screen TV.

Content Designer UI

The QnABot Content Designer UI, like the Web Client, is also deployed to an Amazon S3 bucket and accessed via Amazon API Gateway, and it too retrieves configuration from an API Gateway endpoint. The Content Designer website requires the user to sign in with credentials defined in a Cognito user pool.

Using temporary AWS credentials from Cognito, the Content Designer UI interacts with secure API Gateway endpoints backed by the Content Designer Lambda functions. All interactions with Amazon ES and Amazon Lex are handled by these Lambda functions.

Customizing and enhancing QnABot

Fork the QnABot GitHub repository, enhance the code, and send us pull requests so we can incorporate and share your improvements!

Here are a few suggestions for features you might want to implement:

  • Multiple answers: Allow the user to request a different answer by saying “Try again,” or “Next” to instruct QnABot to reply with lower ranked answers. Useful when questions are ambiguous or when there is more than one correct answer.
  • Scrape existing FAQ pages or documents: Write scripts to capture content from existing suitable sources. Automate, or semi-automate the creation of the QnABot JSON format files for importing.
  • Extend QnABot to access public knowledge base services: Tap into Wikipedia, Evi, WolframAlpha, or other public knowledge servers by extending the Bot fulfillment Lambda function to access additional external information sources.
  • Integrate QnABot with Amazon Connect: Lower your support center call wait times by automating answers to frequently asked questions, using QnABot within an Amazon Connect contact flow. (See Amazon Connect and Amazon Lex Integration.) You can also use popular messaging platforms to interact with QnABot.

Quizzes and Guided tours

Did you know that you can also get QnABot to ask questions and grade your answers? See this companion blog post to find out how: Create a questionnaire bot with Amazon Lex and Amazon Alexa

As described earlier, you can now use buttons to guide the user through your content on the web UI. You can also implement a guided tour with full text and voice support, using Next and Previous commands to navigate. See Creating virtual guided navigation using a Question and Answer Bot with Amazon Lex and Amazon Alexa

About the web UI

The QnABot uses the chatbot web UI described in the blog post: Deploy a Web UI for Your Chatbot.

You can easily deploy this Chatbot UI separately to integrate QnABot as a component into your own website. The Chatbot UI offers many features, including optional integrated user authentication which you can use to create personalized responses from QnABot.

 

Upgrading QnABot

Option 1: Install new stack

If you have installed an earlier version of QnABot, you should use the Export feature to create a JSON file containing all your Q and A content. Install this latest version as a new stack (follow above instructions), and use the Import option to load your saved JSON file.

After importing your content, run tests to validate that your questions are answered correctly. You might find that you need to retune some existing questions for the new version.

After you have validated that the new QnABot is answering questions as expected, you can switch users over, and uninstall the old version.

Option 2: Upgrade existing stack

You can use CloudFormation to upgrade your existing QnABot stack. Before doing a production upgrade, be sure to use the Export feature to create a JSON file containing all your Q and A content. Create a test installation of the new version, import your content, and test all your questions. You might find that you need to retune some existing questions.
Upgrade your production QnABot stack only when you are sure that the latest version passes all your tests:
  1. On the CloudFormation console, select the main QnABot stack.
  2. Choose Update.
  3. Choose Replace current template.
  4. Enter Amazon S3 URL: http://s3.amazonaws.com/aws-bigdata-blog/artifacts/aws-ai-qna-bot/templates/public.json
  5. Choose Next three time to display the Review page.
  6. Select the acknowledgement checkboxes, and choose Update Stack to upgrade QnABot.

Uninstalling QnABot

When you’re done experimenting, in the CloudFormation console, choose and delete the master QnABot stack. This removes all of the AWS resources associated with the bot. Be sure to first export any QnA items that you want to reuse later.

Summary

This blog shares our sample Question and Answer bot, QnABot. We explain how you can use QnABot to create a natural language interface that works with both text and voice using Amazon Lex via your web browser or using any Alexa-enabled device.

Now you can try it with your own questions and answers! Please use the comments link to share your experiences with the QnABot – we’d love to hear from you!


New features log

  • November 2019 (v2.4.0) – Customizable configuration settings, minimized need to rebuild Lex bot when adding new content, improved accuracy using keyword filters, configurable response when answer isn’t found, improved handling of user feedback with integrated thumbs up/down buttons in web client, automated question testing in Content Designer, support variable substitution and conditional answers using ‘handlebars’ templates in Content Designer.
  • November 2018 – Buttons on response cards, eliminate need to retrain Alexa skills for new question, and minor UI improvements.
  • June 2018 – Markdown (rich text), SSML (Speech markup), usage and user feedback logging and dashboard.
  • May 2018 – New version of QnABot that contains bug fixes, a new look for the user interface, and additional features, including support for follow-up questions and extensibility with Lambda hook functions.

 


Additional Reading


About the Authors

Bob Strahan is a Principal Consultant for AWS Professional Services. Bob Potterveld is a Senior Consultant for AWS Professional Services. John Calhoun is an Associate Solutions Architect for the AWS Public Sector Partners team. They help our customers and partners on a variety of projects.