AWS Contact Center

Analyze Amazon Connect Chat sentiments using Contact Trace Record, Amazon Athena and Amazon QuickSight–Part 3

Note: This is the third blog in the Amazon Connect reporting blog series. If you haven’t read “Analyze Amazon Connect Contact Trace Record with Amazon Athena and Amazon QuickSight–Part 1”, we strongly recommend you do before proceeding further.

Introduction

Organization that offer web chat as a communication channel strive to get insights into their customers’ sentiments, trends, and compliance risk of these conversations. These organizations use these insights to replicate successful interactions, identify important customer feedback and train their agents on these findings. These insights are generated using Amazon Comprehend for sentiment analysis, Amazon Connect Contact Trace Record (CTR), and the customer-agent chat.

In this blog, you will learn how Amazon Athena enables contact center analysts to run detailed SQL queries to get the aforementioned insights from the customer-agent web chat interactions on the Amazon Connect platform.

Architecture diagram for the solution.

Fig 1–High-level architecture diagram

In the preceding architecture diagram, Amazon Connect stores the chat transcript from the interaction in Amazon Simple Storage Service (S3). Amazon EventBridge tracks the chat transcript creation and streams the data to Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose invokes an AWS Lambda function to perform the following action:

  1. Reads the chat record from S3
  2. Parses the chat transcript in the JSON format to extract the transcript per speaker
  3. Invokes Amazon Comprehend for the sentiment of the chat transcript per speaker
  4. Enriches the chat record with the sentiment analysis

Amazon Kinesis Data Firehose stores the final form of the chat record to S3 with date-based partitioning in Parquet file format.

AWS Glue Catalog has table definitions pointing to the S3 bucket for the CTR and Chat transcript output files. Users can cross reference CTR, transcripts, and sentiment data by Contact Id and query the data using Amazon Athena. Amazon QuickSight is used for visualizations.

This solution uses a parameter called “Project Name” that defines the solution domain. You can deploy this solution with different project names to maintain isolation. Since the project name will name fundamental resources, such as an S3 bucket, the project name must be globally unique.

Prerequisites

To follow along with the solution presented in this blog post, you should be familiar with the following AWS services and features:

Walkthrough

There are two options to deploy based on the level of analyses:

  • Option 1—detailed sentiment analyses: This solution allows you to query chat transcripts along with its sentiment data and CTR entities, such as queues and agents.
  • Option 2—base sentiment analyses: This solution allows you to query chat transcripts only along with its sentiment.

Option 1 deployment steps

Follow the blog “Analyze Amazon Connect Contact Trace Record with Amazon Athena and Amazon QuickSight–Part 1” to deploy and validate the solution until Athena. For data visualization in QuickSight, follow the steps in this blog

Note the “ProjectName” used while deploying the CloudFormation template in the first blog, as the same name must be used in this step to continue to build on top of the same deployment. You can use a project name like “acathenaquicksight” but the name must be unique or CloudFormation will return an error while creating the S3 bucket. If an error happens, you can redeploy with a different project name.

  1. Sign in to the AWS Management Console.
  2. Download the CloudFormation template from here into your local machine.
  3. In your preferred Region, create a CloudFormation stack using the template file downloaded in step 2
  4. The deployment requires the following parameters:
    1. Under “CreateANewGlueDatabaseAndS3BucketForTheProject” select the “Use existing…” option
    2. Use the same project name as given while deploying “Analyze Amazon Connect Contact Trace Record with Amazon Athena and Amazon QuickSight-Part 1” blog.

Screen shot of cloud formation template for input parameters. 1. Stack name 2. Connect Bucket 3. Create a new Glue Database and S3 bucket for the this project from the drop down 4. Project name

Option 2 deployment steps

  1. Sign in to the AWS Management Console.
  2. Download the CloudFormation template from here into your local machine.
  3. In your preferred Region, create a CloudFormation stack using the template file downloaded in step 2
  4. The deployment requires the following parameters:
    1. Under “CreateANewGlueDatabaseAndS3BucketForTheProject” select the “Create a new…” option
    2. Provide a project name that will be used to create resources within AWS

Screen shot of cloud formation template for input parameters. 1. Stack name 2. Connect Bucket 3. Create a new Glue Database and S3 bucket for the this project from the drop down 4. Project name

Keep the defaults on the remaining options page and select Create Stack

Steps common to “Option 1” and “Option 2”

Configure S3 to stream data to Amazon EventBridge

  1. Navigate to S3 bucket, where Amazon Connect stores Contact Lens data (ConnectBucket), select Properties and scroll to Amazon EventBridge.
  2. Configure the “Send notifications to Amazon EventBridge for all events in this bucket” to be “On” as shown below.

Validation

Generate data by initiating a chat request using Amazon Connect. You can go to Amazon Connect Cloud Control Panel (CCP) to receive the chat request. Within a few minutes, the chat transcript and sentiment data will appear in the bucket named as your project name under the folder “chat-base” as a “.parquet” file. Generate data by initiating a chat request using Amazon Connect. You can go to Amazon Connect Cloud Control Panel (CCP) to receive the chat request. Within a few minutes, the chat transcript and sentiment data will appear in the bucket named as your project name under the folder “chat-base” as a “.parquet” file.

  1. Navigate to Amazon Athena, select Tables, you should see “connect_chat” table and under Views “avg_chat_sentiment_per_agent”.
  2. To query the data with Athena, click on three dots besides the ‘connect_chat’ table, then select “Preview Table”.Screenshot showing how to preview Connect_Chat table options with Preview table option
  3. Athena displays the query results when you run it.Screenshot showing query gives the results in the result section

Visualize in Amazon QuickSight

Using Athena to query the data allows you to visualize the results directly with Amazon QuickSight. Before connecting Amazon QuickSight to Athena, you must grant Amazon QuickSight access to Athena and its associated S3 bucket. For more information, see managing Amazon QuickSight permissions to AWS resources. After granting the access permissions, you can create a new dataset in Amazon QuickSight based on the Athena table that was created.

  1. Navigate to QuickSight and select ‘Analyses’.
  2. Click on “New Dataset, then select “Athena” and give the data source a name.

In the example below, the name chosen is ‘connect-ctr-cl-visualization’. Afterwards, click on Create data source.screenshot for Pop up to enter the Data source name and create data source button

    1. Choose the name of the database and the source view/table based on the following deployment option, then click select:
      • If you are deploying with option 1, select view “avg_chat_sentiment_per_agent”
      • If you are deploying with option 2, select the table name “connect_chat”
      • Note: Create views within Amazon Athena to build enhanced visualization. For more details refer to this blog post.Screenshot for Pop up to choose database and the table from the dropdown and select button
    2. Finally, click ‘Visualize’ to complete the data-set creation.Screenshot for Pop up to select the data creation method and button to vizualize
    3. In the next screen, you should see the chat data like below

You can use Amazon QuickSight to add other chat records visualizations. Build enhanced visualizations, by creating views within Amazon Athena, check this blog post for more details. In addition, you can build dashboards that continuously monitor your connect instances and share those dashboards.

Clean up

In order to remove the resources created by the stack, perform the following:

  1. Delete the CloudFormation stacks created while implementing this blog.
  2. Delete the object and the S3 bucket created from the CloudFormation template. The Bucket name should contain the project name.
  3. Delete the Glue database created from the CloudFormation template.

Conclusion

In this blog, you learned how to use AWS Lambda, AWS Glue, Amazon Athena, Amazon Comprehend to get insights from Amazon Connect CTRs, Chat transcripts, and its associated sentiments. This blog also shows how Amazon Athena lets you run complex SQL queries on logs. Last, you learned how to design event driven architecture by using Amazon S3 with Amazon EventBridge. To build enhanced visualizations, by creating views within Amazon Athena, check this blog post for more details. To visualize your data in Tableau and Power BI, follow Building AWS data lake visualizations with Amazon Athena and Tableau and Using the Amazon Athena Power BI connector blog, respectively.

For more analytics capabilities on Amazon Connect data sources, refer the following blogs in the Amazon Connect reporting blog series:

Author Bio

Mehmet Demir is a Senior Solutions Architect at Amazon Web Services (AWS) based in Toronto, Canada. He helps customers in building well-architected solutions that support business innovation.
Ankur Taunk is a Senior Specialist Solutions Architect at AWS. He helps customer achieve their desired business outcomes in the Contact Center space leveraging Amazon Connect.