AWS Machine Learning Blog

Build an Amazon Lex Chatbot with Microsoft Excel

This is a guest post by AWS Community Hero Cyrus Wong.

Our institution (IVE) here in Hong Kong has begun experimenting with Amazon Lex in teaching, research, and healthcare. We have many non-technical employees, such as English teachers in IVE and therapists from IVE Childcare, Elderly and Community Services Discipline, who don’t have the technical know-how to build natural language conversational bots (chatbots) in the Amazon Lex console. We completed a few trial projects to build Amazon Lex chatbots for non-technical users. The non-technical users filled in an Excel spreadsheet with their questions, and then developers copied their questions to the Amazon Lex console. Any time a user wanted to change anything in their chatbot they had to go through the same copy and paste process.

We then thought to ourselves: why don’t we use Excel directly to build a chatbot and stop the repeated copy and paste work?

All our employees know how to use Microsoft Excel, so we decided to build a project called “ExcelLexBot” which lets them use Excel to create chatbots with minimum programming skills.

ExcelLexBot is a serverless application that converts an Excel file (xlsx) in predefined format into an Amazon Lex chatbot. The ExcelLexBot we’ve built can be used in many ways, all of which are designed to be built quickly and with minimal development skills. Teachers, such as myself, can use ExcelLexBot to create interactive chat-like tests and assignments. Students use it to add chatbot functionality into their final year-end projects, and they can do it fast: 80 students from the Higher Diploma in Cloud and Data Centre Administration created 80 chatbots within one hour.

In this post, I’ll explain how ExcelLexBot works and how to deploy and use it.

Amazon Lex concepts

Amazon Lex is a service that allows developers to build conversational interfaces for voice and text into applications. With Amazon Lex, the same deep learning technologies that power Amazon Alexa are now available to any developer, so you can quickly and easily build sophisticated chatbots.

An Amazon Lex chatbot consists of three types of components:

  1. Bot – A bot contains all of the components of a conversation.
  2. Intent – An intent represents an action that the bot’s user wants to achieve (scheduling an appointment, ordering flowers, and so forth).
  3. Slot Type – a slot type defines how data in the slot is recognized and handled. Amazon Lex supports built-in slot types from the Alexa Skills Kit. You can create custom slot types in your intents.

For more details, please refer to Amazon Lex – Build Conversational Voice & Text Interfaces in Amazon Lex Concepts.

Architecture and design

ExcelLexBot uses the AWS Serverless Application Model (AWS SAM) to create an AWS SAM Amazon Lex application connecting to Excel. AWS SAM is a specification for describing Lambda-based applications. It offers a syntax designed specifically for expressing serverless resources.

ExcelLexBot builder stack

It consists of an Amazon S3 bucket and an ExcelLexBuilder Lambda event hook function. When a user uploads an Excel file, the ExcelLexBuilder Lambda function performs the following actions:

  1. Parses the Excel file.
  2. Generates an ExcelLexBot SAM template.
  3. Creates a Lambda code package.
  4. Uploads the Lambda code package to S3
  5. Creates changeset from the ExcelLexBot SAM template.
  6. Deploys the chatbot stack.

ExcelLexBuilder reads the Excel file and generates the ExcelLex SAM Template and Deployment Package. It creates one JSON file per worksheet as currently, and I use AWS Lambda-backed custom resources to execute the JSON file to create the chatbot. You can check the details of using JSON to create the Amazon Lex Bot with Exercise 1: Create an Amazon Lex Bot (AWS CLI).

Chatbot stack

For the basic Lex terminology (for example, what is Intent), please refer to Amazon Lex – Build Conversational Voice & Text Interfaces on the AWS blog.

Lex Dispatcher is the default code hook that captures both the dialog and fulfillment event. It performs the following tasks:

  1. Forwards requests to other Lambda functions based on naming convention. There are three naming rules.
    1. The first rule “Capture All,” forwards both Dialog and Fulfillment event, if there is a Lambda function with the same name as the intent, for example MakeAppointmentIntent.
    2. The second rule, “Capture Dialog,” forwards the Dialog event to a Lambda function with name end with “_dialogCodeHook” for example, “MakeAppointmentIntent_dialogCodeHook”.
    3. The third rule, “Capture Fulfillment,” forwards the Dialog event to a Lambda function with a name ending with “_fulfillmentActivity” for example, “MakeAppointmentIntent_fulfillmentActivity”.

    You can use Lambda functions to perform initialization and validation, fulfillment, or both in your intent configuration.

  2. If you defined “Capture All”, then you should not define “Capture Dialog” or “Capture Fulfillment”. If you haven’t defined “Capture All”, you must define both “Capture Dialog” and “Capture Fulfillment” at the same time.
  3. When the fulfillmentActivity event happens, it saves the intent information into an Amazon DynamoDB table with the same name of the intent. And, all tables have Auto Scaling for Amazon DynamoDB enabled with throughput range from 1 to 50 capacity units.
  4. When the fulfillmentActivity event happens and the user provides email addresses in Excel, it will publish the intent information to Amazon SNS and let Amazon SNS send emails to those email addresses.

This design can handle the common tasks such as saving the intent information and emailing it to the user, and decoupling the custom code hook from chatbot stack.

If you want to migrate your existing Amazon Lex chatbot to ExcelLexBot, you need to fill in the Excel worksheet and redeploy all Lambda function code hooks according to the naming convention. There isn’t any special magic behind the Lex Dispatcher function. It just gets the current intent name, appends a proper suffix, and calls another Lambda function.

def call_lambda(event, function_name, function_set):
    if function_name in function_set:
        response = lambda_client.invoke(
            FunctionName=function_name,
            InvocationType='RequestResponse',
            Payload=json.dumps(event)
        )
        data = json.loads(response['Payload'].read().decode("utf-8"))
        print(data)
        return data
    else:
        return None

When the “FulfillmentCodeHook” event happens, it saves and sends email before calling another Lambda function.

if source == 'FulfillmentCodeHook':
    save_fulfillment(intent_name, event)
    publish_to_sns(intent_name, event, aws_account_id)

With an ExcelLexBot you define the codehook as usual. You just need to define the code hook as usual. Amazon Lex Chatbot as Lex Dispatcher does not modify anything and it  just forwards the event to the original code hook. You can check out Amazon Lex and AWS Lambda Blueprints to learn how to write code hooks.

Excel format

The Excel format follows the Convention Over Configuration Principle:

  1. The worksheet name will be the Amazon Lex component name.
  2. The type of component maps to the suffix of the worksheet name and there are three types of components: Bot, Intent, and Type. The following diagram for ExcelLexBot illustrates this, and it can map to the original Amazon Lex as well.

    A Bot can have 1 or more Intent. Each Intent can have 1 or more Question (Slot). Each Question has to set a Type (SlotType). Type can be Lex Built-in or Custom Slot Type (user defines in worksheet). For Custom Slot Type, there are one or more enumeration value (Possible Value). Each enumeration value can have synonyms.

I have reformatted the JSON data into Excel data so you can refer to the AWS blueprints for the detailed meaning of most fields.

A few things to note:

  1. The Excel file name only contains English characters. For example, ChatBot- Copy.xlsx or ChatBot2.xlsx are not valid.
  2. We renamed “slot” into “question”, “slotType” into “type”, and “slotConstraint” into “Constraint” to use more familiar terms.
  3. Sample Utterances, Receiver Email, Question and Possible Value allow multiple line inputs. For example:

    That represents the JSON Array in the original Lex JSON Receiver Email is the only additional field that does not exist in the Lex JSON schema.
  4. Max Attempts in the intent worksheet will populate to the “maxAttempts” of the slot (question) as users don’t care about the number of retry per question.
  5. The content cell value only supports ASCII characters, and it will remove non-ASCII

Bot worksheet

  1. Description – A description of the bot.
  2. Abort Statement – If Amazon Lex can’t understand the user’s input in context, it tries to elicit the information a few times through user prompts. After that, Amazon Lex sends the message defined in Abort Statement to the user, and then terminates the conversation.
  3. Clarification Prompt – When Amazon Lex doesn’t understand the user’s intent, it uses this message to get clarification.
  4. The name of the bot is the name of this worksheet, that is, ExcelLexBot and it will include all intents defined in this Excel workbook. These are OrderFlowersIntent and MakeAppointmentIntent.

For more details, please refer to the put-bot cli command.

Output ChatBot_ExcelLexBot.json
{
    "name": "ChatBot_ExcelLexBot",
    "description": "Bot to Demo Excel Lex Bot",
    "intents": [
        {
            "intentName": "ChatBot_OrderFlowersIntent",
            "intentVersion": "$LATEST"
        },
        {
            "intentName": "ChatBot_MakeAppointmentIntent",
            "intentVersion": "$LATEST"
        }
    ],
    "locale": "en-US",
    "abortStatement": {
        "messages": [
            {
                "content": "Sorry, I'm not able to assist at this time",
                "contentType": "PlainText"
            }
        ]
    },
    "clarificationPrompt": {
        "maxAttempts": 2,
        "messages": [
            {
                "content": "I didn't understand you, what would you like to do?",
                "contentType": "PlainText"
            }
        ]
    },
    "voiceId": "Salli",
    "childDirected": false,
    "idleSessionTTLInSeconds": 600
}

Intent worksheet

This snippet is the top section of MakeAppointmentIntent worksheet.

  1. Description – A description of the intent.
  2. Max Attempts – The number of times to prompt the user for information.
  3. Confirmation Prompt – Prompts the user to confirm the intent before fulfilling it.
  4. Sample Utterances – Each line is a sample utterance, and it converts to an array of sample utterances that are configured for the intent.
  5. Receiver Email – Each line is an email address. . A record indicating intent fulfillment will be sent to this email address via SNS..
  6. The name of the intent is the name of this worksheet.

This snippet is the question section of MakeAppointmentIntent worksheet.

I learned that users find it easier to understand making a questionnaire than the idea of Intent. For developers or IT students, the idea is just to create an online form for a set of variables in strong type language, for example, AMAZON.DATE Date = 10/5/2017. Amazon Lex just helps you to replace any complicated web form with data type validation. My students feel that using ExcelLexBot to build a chatbot is easier than creating a web form.

  1. Name – The name of the question (slot). It will be the attribute name in Amazon DynamoDB and JSON data for code hook.
  2. Description – A description of the question (slot).
  3. Question – Each line is a question text. The original AWS Lex JSON uses valueElicitationPrompt with the list of messages in string. All of our users don’t know SSML so ExcelLexBot does not support SSML.
  4. Type – The type of the question (slot), either a custom slot type that you defined or one of the built-in slot types.
  5. Constraint – Specifies whether the slot is required or optional.
  6. Priority – Directs Lex to follow the order in which to elicit this slot value from the user. For example, if the intent has two questions (slots) with priorities 1 and 2, Amazon Lex first elicits a value for the question (slot) with priority 1. If multiple questions (slots) share the same priority, the order in which Lex elicits values is arbitrary.
  7. Sample Utterances – If you know a specific pattern with which users might respond to an Amazon Lex request for a question (slot) value, you can provide those utterances to improve accuracy. This is optional. In most cases, Amazon Lex is capable of understanding user utterances.

For more details, please refer to the put-intent cli command.

Output is at ChatBot_ExcelLexBot.json, and here is the partial output.

{
	    "name": "ChatBot_MakeAppointmentIntent",
	    "description": "Intent to make appointment",
	    "slots": [
	        {
	            "slotType": "ChatBot_AppointmentType",
	            "name": "AppointmentType",
	            "slotConstraint": "Required",
	            "valueElicitationPrompt": {
	                "maxAttempts": 2,
	                "messages": [
	                    {
	                        "content": "What type of appointment would you like to schedule?",
	                        "contentType": "PlainText"
	                    }
	                ]
	            },
	            "priority": 1,
	            "description": "The type of flowers to pick up",
	            "sampleUtterances": [
	                "I would like a {AppointmentType}"
	            ],
	            "slotTypeVersion": "$LATEST"
	        },
…

Type worksheet

  1. Description – A description of the Type (slot type).
  2. Value Selection Strategy – Determines the slot resolution strategy that Amazon Lex uses to return slot type values. The field can be set to one of the following values:
    ORIGINAL_VALUE – Returns the value entered by the user, if the user value is similar to the slot value.
    TOP_RESOLUTION – If there is a resolution list for the slot, return the first value in the resolution list as the slot type value. If there is no resolution list, null is returned.
  3. Possible Values – A list of Possible Values (EnumerationValue) string that defines the values that the slot type can take. You can create synonyms by newline.

For more details, please refer to the put-slot-type cli command.

Output ChatBot_AppointmentType.json
{
    "name": "ChatBot_AppointmentType",
    "description": "Type of dentist appointment to schedule",
    "valueSelectionStrategy": "TOP_RESOLUTION",
    "enumerationValues": [
        {
            "value": "cleaning"
        },
        {
            "value": "whitening"
        },
        {
            "value": "root canal"
        }
    ]
}

Deploying ExcelLexBot

To deploy the application, perform the following steps:

  1. Create an Amazon S3 source bucket in the us-east-1 AWS Region.
  2. Download the two files in the Deployment folder on GitHub: excellexbot.yaml and lex_builder_function.zip.
  3. Upload the source packages that you get from the Deployment folder, lex_builder_function.zip, into the S3 source bucket. If you are not familiar with the Amazon S3 console, please follow this tutorial: How Do I Upload Files and Folders to an S3 Bucket?
  4. In the AWS CloudFormation console, choose Create Stack.
  5. Select Upload a template to Amazon S3, choose yaml, then choose Next.
  6. Specify the following parameters, and choose Next.
    1. Stack name: excellexbot (A unique name for the stack in your AWS region).
    2. SourceBucket: The name of the S3 source bucket that you created in Step 1.
    3. ExcelBucketName: The name of the S3 bucket that you want to create. The bucket name must be unique. When you delete the AWS CloudFormation stack, this bucket remains.
  7. On the Options page, choose Next.
  8. Select all acknowledgment boxes, and choose Create Change Set.
  9. When the change set has been created, choose Execute.
  10. Wait while the AWS CloudFormation stack is created.

You can reference “Analyze Emotion in Video Frame Samples Using Amazon Rekognition on AWS”  Deploying the “HowWhoFeelInVideo” section and those deployment steps are similar.

Try your deployment

  1. Download the sample excel file ChatBot.xlsx from GitHub.
  2. In the S3 console, upload a ChatBot.xlsx into the bucket for ExcelBucketName.
  3. Open the AWS CloudFormation console.

    Wait until the chatbot stack status changes into CREATE_COMPLETE. The stack name is the lowercase version of the excel file name.
  4. (Optional) If you input email for the Intent, you will receive an email to confirm your subscription and click on the Confirm subscription
  5. Open the Amazon Lex console, and choose ExcelLexBot
  6. Choose OK. Amazon Lex will update the resource permission to the Lex Dispatcher Lambda function. AWS CloudFormation already granted the permission to Amazon Lex, but if you don’t click on add the permission here, it will pop up every time when you are testing the chatbot.
  7. Choose the Test Chatbot bar on the left side.
  8. You can test the Chatbot from the Amazon Lex console.
  9. This deployment contains Example Bot: ScheduleAppointment and you can test by following that example.
  10. Open the Amazon DynamoDB console and the table ChatBotMakeAppointmentIntent will contain a new item.
  11. (Optional) If you input email for the Intent, you will receive an email with the intent information.

Add a code hook

In this section, you update the configuration of the MakeAppointment intent to use the Lambda function as a code hook for validation and fulfillment activities. For the details about this code hook, read Step 3: Update the Intent: Configure a Code Hook.

  1. Download the two files in the deployment folder on GitHub: codehook.yaml and codehook.zip.
  2. Repeat the deployment step in the previous Deploying ExcelLexBot section.
  3. Specify stack name and parameter:
    1. Stack name: codehook (A unique name for the stack in your AWS Region.)
    2. SourceBucket: The name of the S3 source bucket that you created in Deploying ExcelLexBot Step 1.
  4. Wait for the code hook step status changes to CREATE_COMPLETE.
  5. You can test the chatbot again and it will have the same features as Example Bot: ScheduleAppointment.

Update and delete a chatbot

If you delete the Excel file from the S3 bucket, it will delete the chatbot stack. Warning! If you delete the chatbot stack, it will also delete the DynamoDB table, so you have to back up the data before you delete the stack.

To make updates, you need to delete the Excel and wait for the chatbot stack completely delete. Then, you upload the updated Excel into Amazon S3.

Limitations

  1. The AWS Management Console can validate input immediately, but ExcelLexBot cannot validate the input.
  2. Lex allows reusing intents, slot-types across bots – but the Excel based solution assumes independence. Basically, downplay the limitation.
  3. Amazon Lex protecting itself by throttling aggressive calls, so you cannot delete multiple Excel files or chatbot stacks at the same time as Lex will return “There is a conflicting operation in progress” if there are too many “delete Bot API calls” at the same time. However, you can create multiple chatbots at the same time by uploading several Excel files to Amazon S3.

Conclusion

Non-technical users are sometimes hesitant to learn new tools. However, if you send them a few example Excel files, and ask them to fill in their questions in a spreadsheet, most of them can do it without hesitation.

ExcelLexBot can help build chatbots through Excel easily. IVE teachers can just copy assignments or assessments from Word to Excel, then we can adopt Amazon AI services to enhance the IVE teaching experience.

This project has been developed in collaboration with four of my students from the IT114115 Higher Diploma in Cloud and Data Centre Administration: LI, Ka Wing, LAU, Tik Haang, HON, Ho Yuk, and MA, Shi Sum. Also, thanks to the AWS Academy curriculum, which helps my students learn how to use AWS services in depth!

Additional Reading

Learn how to build better bots with Amazon Lex.

About the Author

Cyrus Wong is a Data Scientist at the Hong Kong Institute of Vocational Education (Lee Wai Lee) Cloud Innovation Centre and an AWS Community Hero. He has achieved all 7 AWS Certifications and enjoys sharing his AWS knowledge with others through open-source projects, blog posts, and events.