Predict customer contact intent using AI and Amazon Connect

Customers engage with businesses using a multitude of contact channels like voice, messaging, web, and social media. Each interaction includes customer identification, verification, and understanding the intent. This is followed by servicing the contact intent using either automated or live assistance. The primary customer need is expedited assistance to resolve the reason for the contact. It is therefore important to quickly understand the contact intent.

Legacy technologies limited businesses to forcing customers through menu trees to clarify their intent. The menu maze often was a reflection of the organizational structure of the business. Natural language understanding technologies (Amazon Lex) simplify this experience by enabling the customers to express the contact intent in a few words.

Another option to simplify intent capture is to use data. The contact intent is often correlated with a customer’s ongoing, or most recent set of interactions or events. For example, a retail banking customer experiencing a failed transaction at an ATM is likely to call up the contact center.

Another retail banking customer with an ongoing home loan request is likely to call about the loan status. However, it takes an ongoing domain and technical heavy-lifting to build, maintain, and adapt these patterns using rule-based systems. Rule-based approaches lack agility, and cannot scale across multiple customer interaction channels used by digital businesses. With pay-as-you-go AI/ML capabilities in the cloud, businesses can now use historical data to learn, generalize, and predict caller intents for future interactions.

In this blog post, we walk you through the steps to build an intent prediction model using Amazon Machine Learning capabilities. Customers can expect an intelligent, personalized, and faster resolution experience using this approach. Businesses can reduce technology maintenance costs, improve customer satisfaction scores, improve automation rates, and reduce agent transfer costs. We use Amazon Personalize from Amazon Connect to create this solution.

Pre-requisites

To follow along with the solution presented in this blog post, you must understand the following AWS services and features:

• Amazon Connect
• Amazon Lex
• Amazon Personalize
• AWS Lambda
• Amazon DynamoDB
• Amazon SageMaker
• AWS Identity and Access Management (IAM)
• You also need an active AWS account with the permission to create and modify Lambda functions, Amazon Personalize, Amazon S3 objects, and IAM roles.

To begin, you need an Amazon Connect instance configured for inbound and outbound calls. Claim a phone number after you create your instance. The Get Started with Amazon Connect and Create Amazon Connect contact flows documentation provide valuable background knowledge for this process.

Solution overview

In the preceding architecture, the following sequence of steps takes place for an incoming contact:

1. A customer calls into a toll Free/DID number, which invokes an Amazon Connect contact flow.
2. Amazon Connect invokes a Lambda function by passing the customer identification information within the contact flow.
3. The Lambda function invokes Amazon Personalize API to retrieve the recommendation for the customer contact intent. Amazon Personalize uses a trained model using the historical contact activity data from a representative sample of callers.
4. If the intent is predicted with a high confidence score, the Lambda function returns a predicted intent to Amazon Connect. The score threshold value is configurable. The contact flow in Amazon Connect prompts the caller to confirm the predicted intent.
5. Otherwise, the Lambda function returns a value indicating that the model could not predict an intent for this interaction. The contact flow in Amazon Connect proceeds with the intent capture experience using either Amazon Lex (NLU), or a menu.
6. Before the end of the call, the actual confirmed intent of the customer is fed back into the Amazon Personalize API. This is used for future predictions for this caller.

Solution datasets

This blog post uses a sample dataset in the format as prescribed by Amazon Personalize. It uses two datasets (1) Users and (2) Interactions. The first step in the Amazon SageMaker notebook downloads user data formatted in the prescribed format from a publicly accessible web-location. Next, it creates the interactions data to be used for initial training.

1. The Users dataset contains a required attribute USER_ID (string). It can have additional optional metadata attributes. The sample contact center dataset contains the user identifier and optional profile metadata.
2. The Interactions dataset has three required attributes USER_ID (string), ITEM_ID (string), and TIMESTAMP (long). The sample contact center dataset contains historical contact activity data of the users. ITEM_ID contains the caller intent for the contact, represented as a single word string. For example, a string AC_PA encodes a contact intent related to debit card and PIN activation related for retail accounts product.

You can explore the entire dataset in the Amazon SageMaker notebook deployed in the following section. If you want to use a different dataset, the dataset schema documentation explains the format requirements for Amazon Personalize.

Setting up the Solution

Step 1: Deploy Amazon SageMaker notebook

The AWS CloudFormation template launches a SageMaker notebook instance with required IAM roles and permissions. This notebook is used to download sample data, and to train an Amazon Personalize model for the solution.

1. Log in to the AWS Management Console.
2. Choose the following button to launch the stack in AWS CloudFormation. Note that, the template launches resources in us-west-2 (Oregon) Region.
   
   
   
3. Select “I acknowledge that AWS CloudFormation might create IAM resources.”
4. Choose Create stack.
   
5. Wait until the CloudFormation stack moves from CREATE_IN_PROGRESS state to CREATE_COMPLETE state (~5 minutes).
   

Step 2: Train and deploy Amazon Personalize campaign

1. From the AWS Management Console, open the Amazon SageMaker console.
   
2. On the navigation bar, choose Notebook instances.
3. Choose Open Jupyter to open the Jupyter web interface.
   
4. On the Jupyter web interface, choose sourcecode.
   
5. Click on train_personalize_with_customer_and_contact_records.ipynb to launch the Python notebook.
   
6. On the top menu bar, choose Cells, Run All. (Note: If you click on the Run option available on the toolbar, you need to run each cell separately.)
   
7. The notebook takes ~70 minutes to run. There are three cells that take longer than others to finish running. For these steps, you see progress indicator status (CREATE IN_PROGRESS) every minute as a console message until it reaches the final status (ACTIVE). Do not close the browser window with the notebook instance until all cells finish running.
8. After the cells have finished running, copy the values of campaign_arn and tracking_id variable outputs at the bottom of the notebook.
   

Step 3: Deploy Lambda functions

The AWS CloudFormation template following creates two Lambda functions with required IAM roles and permissions. These functions perform Amazon Personalize API operations and expose the necessary outcomes to Amazon Connect.

1. Log in to your AWS Management Console.
2. Choose the following button to launch the stack in AWS CloudFormation.
   
   
   
3. In the Parameters section, enter values for the two parameters PersonalizeCampaignARN and PersonalizeModelTrackingID copied from the notebook console earlier.
   
4. Check all acknowledgement boxes about IAM resources and capabilities.
5. Choose Create stack.
   
6. Wait until the CloudFormation stack moves from CREATE_IN_PROGRESS state to CREATE_COMPLETE state (~5 minutes). This CloudFormation stack launches two AWS Lambda functions with required IAM roles and permissions.
   
7. From the AWS Management Console, open the AWS Lambda console.
   
8. Verify that you are able to see two recently deployed AWS Lambda functions named predict-ci-li-update-real-time-customer-intent and predict-ci-li-get-personalized-intent.
   

Step 4: Grant Amazon Connect permission to run your Lambda function

Verify that the Amazon Connect instance has permissions to access this newly created AWS Lambda function by following these steps.

1. 1. 1. From the AWS Management Console, open the Amazon Connect console.
      2. Select your Amazon Connect virtual contact center instance.
      3. Choose Contact flows and scroll down to the AWS Lambda section.
      4. On the Function drop-down menu, select predict-ci-lf-get-personalized-intent function and choose +Add Lambda Function.

   

   1. Again, on the Function drop-down menu, select predict-ci-lf-update-real-time-customer-intent function and choose +Add Lambda Function.

   Step 5: Import and configure the Amazon Connect contact flow

   1. 1. Download the pre-built contact flow.
      2. From the AWS Management Console, open the Amazon Connect console.
      3. Select your Amazon Connect virtual contact center instance and log in.
      4. On the navigation bar of the Amazon Connect console, choose Routing, Contact flows
      5. Choose Create contact flow button at the top-right.
      6. On the drop-down at the top-right, select Import flow.
      7. Choose Select and select the PredictCustomerIntentFlow file downloaded in step 1.
      8. On the contact flow, find the Invoke AWS Lambda function block. Open the settings for the block by selecting the header.
      9. Select the Lambda function predict-ci-lf-get-personalized-intent that you granted Amazon Connect permissions in Step 4.

   

   1. 1. Choose Save.
      2. Choose the header for the Invoke AWS Lambda function block (second row) and select the Lambda function predict-ci-lf-update-real-time-customer-intent.

   

   1. Choose Save.
   2. Save and Publish your contact flow.

   The pre-built contact flow PredictCustomerIntentFlow is made up of the following key steps:

   1. Set the basics – logging, recording, and default voice using Amazon Polly (US-English).
   2. Greet the caller and prompt for identification. In this blog post, we use a 5-digit account number as an identifier.
   3. Invoke the predict-ci-lf-get-personalized-intent function to retrieve recommended intent prediction for this caller.
   4. Prompt the caller to confirm the predicted intent (high confidence score), or ask the caller to describe the contact intent.
   5. Invoke the predict-ci-lf-update-real-time-customer-intent function to provide a real-time update on the intent that the customer either confirmed or entered.

   

   Step 6: Try it out

   1. In your Amazon Connect instance, choose the Routing icon from the navigation bar, and choose Phone numbers.
   2. Choose the phone number you want to associate with your new contact flow to edit.
   3. Choose the name of your contact flow from the Contact flow/ IVR drop-down menu, and choose Save.
   4. To make a test call, call the phone number you associated to the flow.
      1. High-Confidence prediction test call: Use a customer identifier with a recent contact history record in the dataset results in a prediction with high confidence score. Enter the 5-digit customer identifier 35739 to be presented with an intent prediction, based on a prior transaction for this user identifier in the dataset.
      2. Low-Confidence prediction test call: A customer identifier that is new to the dataset, or with no recent contact history results in a prediction with a low confidence score. Enter the 5-digit customer identifier 89789 to be presented with the IVR main menu. The prediction score is not sufficient to present a personalized prediction. The sample main menu provides two options (home loans, and credit cards) to bootstrap new customer interactions.
   5. In both scenarios, the IVR confirms and captures the actual intent of the customer. It uses this data to make predictions for subsequent calls. If you call again with the same customer identifier, you may be presented with a different intent prediction result dynamically. Amazon Personalize updates the dataset in real time, and is able to adjust the recommendations accordingly.

   Considerations

   1. This blog post uses a sample dataset injected with some activity patterns. For enterprise use cases, we recommend using customer interaction data from all available channels such as online, email, mobile in addition to the contact center. The prediction results are more accurate with interactions and event data from all contact channels consolidated together.
   2. Programming rules and logic can be used along with predictions. If there are strong correlations between intents on prior call and next, these can be implemented using rules. These rules if applicable for an interaction, can be prioritized before a prediction.

   Clean up

   To avoid ongoing charges, follow these steps to clean up this deployment.

   1. Open the clean_up_personalize_resources.ipynb notebook provided in the sourcecode directory in the SageMaker notebook.
   2. On the top menu bar, choose Cells, Run All to delete the personalize artifacts, Amazon S3 objects, and buckets created by the SageMaker notebook. (Note: If you click on the Run option available on the toolbar, you need to run each cell separately.)
      
   3. To remove the associated AWS resources for this solution, delete the two CloudFormation stacks one at a time.
      

   Conclusion

   Predicting customer contact intent simplifies and improves customer experience. Automating the predicted intents in Amazon Connect can also result in higher self-service rates. For calls that are routed straight to an agent, it improves routing accuracy and reduces agent handoffs. There are multiple approaches to predicting customer contact intents. In this blog post, we implemented an intent prediction solution based on machine learning, using Amazon Personalize and Amazon Connect. The objective is to build a solution that can learn and generalize without hardcoding a fixed set of rules. The Amazon Personalize based approach can reduce the time it takes to implement a machine learning model from scratch.