AWS Machine Learning Blog

Building a multilingual question and answer bot with Amazon Lex

Updated June 2021 – QnABot now supports voice interaction in multiple languages using Amazon LexV2.

You can use Amazon Lex to build a question and answer chatbot. However, if you live in a non-English-speaking country or your business has global reach, you will want a multilingual bot to cater to all your users. This post describes how to achieve that by using the multi-language functionality of your question and answer bot (QnABot).

The QnABot can detect the predominant language in an interaction by using Amazon Comprehend, a natural language processing (NLP) service that uses machine learning to find insights and relationships in text.

The bot then uses Amazon Translate, a neural machine translation service to convert questions and answers across languages from a single shared set of FAQs and documents.

In earlier releases, QnABot’s multilingual capabilities were limited to Amazon Lex bot text interactions. With QnABot v4.7.0 and later, that limitation is removed. Now QnABot supports voice interactions in multiple languages as well. This is an especially powerful feature when using QnABot in a contact center supporting callers with different language preferences.

The solution consists of these steps:

  1. Configure the multi-language functionality.
  2. (Optional) Set up the alternate curated answers in a different language.
  3. (Optional) Configure your Alexa skill with multiple languages.
  4. (Optional) Configure Amazon Connect to use QnABot with multiple languages

For instructions on creating and customizing your bot, see Create a Question and Answer Bot with Amazon Lex and Amazon Alexa. You can also see the following videos on YouTube:

Prerequisites

To implement this solution, you must have an AWS account. If you don’t have an account, the AWS Free Tier lets you gain experience with AWS products, and services.

You also need to deploy QnABot. If you have not already done so, launch the QnABot on an AWS CloudFormation stack in one of the available Regions.

When specifying your stack name, include QnA in the name, for example, MultiLangQnABot.

QnABot supports both Amazon Lex V1 and V2 APIs. Amazon Lex V2 APIs use an updated information architecture (IA) to deliver simplified versioning, support for multiple languages in a bot, and streaming capabilities.

When deploying the AWS QnABot solution (version 4.7.0 and higher) CloudFormation template, you will see two new parameters LexBotVersion and LexV2BotLocaleIds. These parameters are used to support users to interact with the bot using voice in the configured languages.

  • LexBotVersion — you can choose between two options: “LexV1 and LexV2” or “LexV2”.
  • LexV2BotLocaleIds — enter one or more of the language codes (with multiple values separated by comma such as: en_US, es_US, es_ES, fr_FR). For list of language codes refer to https://docs.aws.amazon.com/lexv2/latest/dg/how-languages.html

After you successfully deploy your CloudFormation stack, complete the following steps:

  1. Open the content designer page of your chatbot.

If this is the first time accessing the chatbot design console, you can find the correct URL on the AWS CloudFormation console on the Output tab of your bot stack. Look for the value for the ContentDesignerURL key. You should also receive an email with temporary credentials to access the QnABot designer.

  1. On the designer page, choose the menu icon.
  2. Under Tools, choose Import.
  3. Expand the Examples/Extensions
  4. Next to blog-samples, choose Load.

Configuring the multi-language functionality

You are now ready to configure the multi-language functionality. Complete the following steps:

  1. In the content designer page, under Tools, choose Settings.
  2. For the ENABLE_MULTI_LANGUAGE_SUPPORT parameter, change the value from false to true.
  3. Choose Save.
  4. To test the bot, open the client webpage.
  5. From the designer page, under Tools, choose QnABot Client.
  6. Enter the following questions (in English, Spanish, and French):
    • How do I modify Q and A Bot content ?
    • ¿Cómo modifico el contenido de Q y A Bot ?
    • Comment modifier le contenu Q et A Bot ?

The chatbot answers each time in the language you used, as shown in the following animation.

If you are using Voice to interact with the bot, you can now choose the language you want to interact with. The language choices will be the ones you had initially provided in the CloudFormation template in the {LexV2BotLocaleIds} parameter. With this, you can easily test and tune voice interactions using different languages.

Click the Menu icon on the left and select the language you would like to use.

QnABot can automatically generate additional Automatic Speech Recognition (ASR) training data for Amazon Lex using questions from all the items you have added. Now QnABot automatically generates LexV2 ASR training data in multiple languages using Amazon Translate, for each LexV2 locale specified during installation.

In addition to this out-of-the-box capability, you can also further improve and fine-tune ASR accuracy, by creating your own language specific questions into the Content Designer.

Once you have added the language specific questions, choose LEX REBUILD from the top right edit card menu ( ⋮ ). This will re-train Amazon Lex using the newly added questions as training data.

Setting up alternate curated answers in a different language

You might need to provide a more natural experience and want to add a curated answer in the native language of your choice. To further customize the translation for each question, you can use the {{handlebar}} functionality. The QnABot provides the {{handlebar}} function ifLang, which takes the locale as a quoted parameter. You can use any of the languages that Amazon Translate supports. For more information, see What Is Amazon Translate?

For example, to customize the translation in Spanish, the ifLang function uses es as the locale parameter. See the following code:

{{#ifLang 'es'}}
          Su traducción al español
{{/ifLang}}

Additionally, if an unknown language is detected, you can support that with a default response by using the defaultLang function. See the following code:

{{#defaultLang}}
          Your default language answer
{{/defaultLang}}

As an example, modify the question you used earlier. Go back to the content designer and complete the following steps:

  1. Under Tools, choose Edit.
  2. Select 001 and choose the pencil icon on the right.
  3. Replace the text in the answer with the following code: 
    {{#ifLang 'es'}}
Use las herramientas de 'Question and Test' de 'Content Designer' para encontrar sus documentos existentes y editarlos directamente en la consola. También puede exportar documentos existentes como un archivo JSON, realizar cambios en el archivo y volver a importar.
{{/ifLang}}
{{#ifLang 'fr'}}
Utilisez les outils 'Question and Test' de 'Content Designer' pour trouver vos documents existants et les modifier directement dans la console. Vous pouvez également exporter des documents existants sous forme de fichier JSON, apporter des modifications au fichier et réimporter.
{{/ifLang}}
{{#defaultLang}} 
Use the Content Designer Question and Test tools to find your existing documents and edit them directly in the console. You can also export existing documents as a JSON file, make changes to the file, and re-import.
{{/defaultLang}}

    Multi-language and handlebars, in general, also support markdown answers. For example, you could modify the preceding code to highlight the name of the interface that isn’t translated. See the following code:

    {{#ifLang 'es'}}
Use las herramientas de ***'Question and Test'*** de ***'Content Designer'*** para encontrar sus documentos existentes y editarlos directamente en la consola. También puede exportar documentos existentes como un archivo JSON, realizar cambios en el archivo y volver a importar.
{{/ifLang}}
{{#ifLang 'fr'}}
Utilisez les outils ***'Question and Test'*** de ***'Content Designer'*** pour trouver vos documents existants et les modifier directement dans la console. Vous pouvez également exporter des documents existants sous forme de fichier JSON, apporter des modifications au fichier et réimporter.
{{/ifLang}}
{{#defaultLang}} 
Use the ***Content Designer Question and Test*** tools to find your existing documents and edit them directly in the console. You can also export existing documents as a JSON file, make changes to the file, and re-import.
{{/defaultLang}}
  4. Choose Advanced and enter the new code in the Markdown Answer box.

  5. Choose Update.

If you try to ask your questions again, the answers are different because the chatbot is using your curated version.

You can also import the sample or extension named Language / Multiple Language Support.

This adds two questions to the system: Language.000 and Language.001. The first question allows the end user to set their preferred language explicitly; the latter resets the preferred language and allow the QnABot to choose the locale based on the automatically detected predominant language.

Debugging the answers in a different language

You can use the ENABLE_DEBUG_RESPONSES setting to see how local language questions are translated to English by QnABot, and to tune the content as needed to ensure QnABot finds the best answer to a non-English question.

Complete the following steps to set up and test:

  1. In the content designer page, under Tools, choose Settings.
  2. For the ENABLE_DEBUG_RESPONSES parameter, change the value from false to true.
  3. Choose Save.
  4. To test the bot, open the client webpage.
  5. From the designer page, under Tools, choose QnABot Client.
  6. Try one of the question we used before, you can read the translation and use this information to tune your answer.

Configuring your Alexa skill with multiple languages

You first need to create your Alexa skill. For instructions, see Create a Question and Answer Bot with Amazon Lex and Amazon Alexa.

When your Alexa skill is working, add the additional languages by completing the following steps:

  1. On the Alexa developer console, open your skill.
  2. From the drop-down menu with your default language, choose Language settings.
  3. Add all the languages you want to support and choose Save.
  4. Under CUSTOM, choose JSON Editor.
  5. Copy the JSON from the editor, switch to the other language you want to support, and enter it in the editor pane (this overwrites the default).
  6. Choose Save Model.
  7. Choose Invocation and change the invocation name.
  8. Choose Save Model.
  9. Repeat these steps for any language you want to support.
  10. Build the model.

Testing your Alexa skill

You can now test your Alexa skill in other languages.

  1. On the Alexa developer console, select your skill.
  2. Choose Test.
  3. Change the language and type the invocation name of your skill for that language.
  4. After Alexa gives her initial greeting, ask the question you used before or any other question you added in the content designer.

Alexa answers you in the selected language.

Now your multilingual chatbot can be published on your website or as an Alexa skill. To integrate the QnABot in your website, you can use lex-web-ui. For instructions, see Deploy a Web UI for your Chatbot.

Configure Amazon Connect to use QnABot with multiple languages

With the QnABot version 4.7.0 and higher, you can now integrate language support to your Amazon Connect contact flows. Users will be able to interact with the bot over the phone (via voice) using one of the languages you configured in the QnABot LexV2BotLocaleIds CloudFormation parameter.

Note: this support is currently available in the Canada (Central) ca-central-1 AWS Region using Amazon LexV2. When deploying the QnABot solution, you should also configure the LexBotVersion CloudFormation parameter to “LexV2 Only”. We are continuing to work through our Region expansion plans for this capability and will update this blog post with more details.

QnABot can export a Contact Flow to use in your Amazon Connect instance. Once you have deployed QnABot in the Canada (Central) ca-central-1 AWS Region, go to the QnABot Designer console, select the Tools menu (located in the top-left corner) and choose Connect from the menu options.

Follow the wizard steps to configure your Amazon Connect instance. Once the Amazon Connect instance is configured, download the contact flow from the Import Contact Flow.

This contact flow includes an IVR flow where callers can choose the language they would like to interact with. For example: “Press 1 for English”, “Press 2 for Spanish” etc.

Complete the QnABot wizard steps. Once the steps are completed, you can now call the Amazon Connect instance phone number and interact with the Bot in any of the supported languages.

Conclusion

This post shows you how to configure and use the out-of-the-box feature of your QnABot to support multiple languages. It is an inexpensive way to deploy a multi-language chatbot, and you can also extend the solution to be accessible via Amazon Alexa, and Amazon Connect.

For more information about experimenting with the capabilities of QnABot, see the new QnABot Workshop.

About the Authors

Fabrizio Napolitano is a Specialist Solutions Architect for Database and Analytics in the AWS Canada Public Sector team. He has worked in the analytics field for the last 20 years, and has recently, and quite by surprise, become a Hockey Dad after moving to Canada.

 

 

 

As a Solutions Architect at AWS supporting our Public Sector customers, Raj Chary excites customers by showing them the art of the possible of what they can build on AWS and helps accelerate their innovation. Raj loves solving puzzles, mentoring, and supporting hackathons and seeing amazing ideas come to life.