AWS Contact Center

Analyze Amazon Connect Contact Lens using Contact Trace Record, Amazon Athena, and Amazon QuickSight–Part 2

Note: This is the second blog in the Amazon Connect reporting blog series. If you haven’t read “Analyze Amazon Connect Contact Trace Record with Amazon Athena and Amazon QuickSight–Part 1”, we strongly recommend you do before proceeding further.

Introduction

Many organizations want the capability to improve the efficiency of their contact center’s quality analysis process. Contact Lens for Amazon Connect, a feature of Amazon Connect, helps organizations understand customers’ sentiments, conversation characteristics, and agent compliance risks. Contact Lens achieves this by using natural language processing (NLP) and ML-powered speech-to-text-analytics to analyze call transcripts. In addition, organizations can stream Contact Lens data along with Contact Trace Records (CTR) to their data lake to perform advanced analytics.
While CTR captures transactional metrics, such as hold time, wait time, and agent interaction time, and more, Contact Lens output captures call transcript, sentiment analysis, non-talk time, categorization labels, talk speed, and interruptions, and others.
In this blog you will learn how to stream CTR from Amazon Connect, relate it to the Contact Lens output, and visualize the results using Amazon QuickSight.

Overview

Architecture diagram for the solution

Fig 1–High-level architecture diagram

In the above architecture, Amazon Kinesis Streams Amazon Connect CTRs as raw data to an Amazon Simple Storage Service (S3) bucket. The CTR captures the events associated with a contact call from your contact center. Amazon Connect delivers contact records at least once. It may deliver more CTR records for the same call, such as new information arriving after initial delivery.
Amazon Connect delivers the Contact Lens output file directly to Amazon S3 bucket when the Contact Lens feature is activated. The Contact Lens output file is delivered once at the end of the call. In this architecture, Amazon EventBridge tracks the Contact Lens output file creation and streams the data to Amazon Kinesis Data Firehose. Kinesis Data Firehose collects these events for a defined interval and uses AWS Lambda to retrieve the content of each file. The Lambda compresses the records retrieved from these files to a parquet file using the schema defined in the AWS Glue Catalog, and stores this parquet file in the S3 bucket.
AWS Glue Catalog has the table definitions for the CTR and Contact Lens output files. You can relate CTR and Contact Lens data by Contact Id and run queries using Amazon Athena. For visualization, Amazon QuickSight is used.
This solution uses a parameter named “Project Name” that defines the solution domain. You can deploy this solution multiple times with different project names to maintain isolation. Since it will be used to name fundamental components such as an S3 bucket, project name must be unique globally.

Prerequisites

To follow along with the solution presented in this blog post, you should be familiar with the following AWS services and features:

Walkthrough

There are two deployments based on the analysis’ details:

  • Option 1-detailed sentiment analysis: This solution allows you to query Contact Lens along with CTR entities, such as queues and agents.
  • Option 2-base Contact Lens analysis: This solution allows you to query Contact Lens entities only.

Option 1 deployment steps

Follow the blog “Analyze Amazon Connect Contact Trace Record with Amazon Athena and Amazon QuickSight–Part 1” to deploy and validate the solution until Athena (Ignore this step, if already deployed). For data visualization in QuickSight, follow the steps in this blog

Note the “ProjectName” used while deploying the CloudFormation template in the first blog, as the same name must be used in this step to continue to build on top of the same deployment. You can use a project name like “acathenaquicksight” but the name must be unique or CloudFormation will return an error while creating the S3 bucket. If an error happens, you can redeploy with a different project name.

  1. Sign in to the AWS Management Console.
  2. Download the CloudFormation template from here into your local machine.
  3. In your preferred Region, create a CloudFormation stack using the template file downloaded in step 2
  4. The deployment requires the following parameters:
    1. Under “CreateANewGlueDatabaseAndS3BucketForTheProject” select the “Use existing…” option
    2. Use the same project name as given while deploying “Analyze Amazon Connect Contact Trace Record with Amazon Athena and Amazon QuickSight” blog.

Screen shot of cloud formation template for input parameters. 1. Stack name 2. Connect Bucket 3. Create a new Glue Database and S3 bucket for the this project from the drop down 4. Project name

Option 2 deployment steps

  1. Sign in to the AWS Management Console.
  2. Download the CloudFormation template from here into your local machine.
  3. In your preferred Region, create a CloudFormation stack using the template file downloaded in step 2
  4. The deployment requires the following parameters:
    1. Under “CreateANewGlueDatabaseAndS3BucketForTheProject” select the “Create a new…” option
    2. Provide a project name that will be used to create resources within AWS

Screen shot of cloud formation template for input parameters. 1. Stack name 2. Connect Bucket 3. Create a new Glue Database and S3 bucket for the this project from the drop down 4. Project name

Keep the defaults on the remaining options page and select Create Stack

Steps common to “Option 1” and “Option 2”

Configure S3 to stream data to Amazon EventBridge

  1. Navigate to S3 bucket, where Amazon Connect stores Contact Lens data (ConnectBucket), select Properties and scroll to Amazon EventBridge.
    Screenshot for Amazon S3's Amazon EventBridge configuration. By default it is off
  2. Configure the “Send notifications to Amazon EventBridge for all events in this bucket” to be “On” as shown below.
    Screenshot for Amazon S3's Amazon EventBridge configuration after it is turned on.

Validation

Generate data by making or receiving calls using Amazon Connect. Go to Amazon Connect Contact Control Panel (CCP) to make or receive calls using a software phone or desktop phone. Within few minutes, Contact Lens data will appear in the bucket named as your project name under the folder “contactlens-base” as a “.parquet” file.

  1. Navigate to Amazon Athena table, you should see “connect_contactlens” table
    Screenshot showing query gives the results in the result section
  2. To query the data with Athena, click on three dots besides the ‘connect_contactlens’ table, then select “Preview Table”
    Screenshot showing query gives the results in the result section
  3. Athena displays the query results when you run it.
    Screenshot showing query gives the results in the result section
  4. If you are deploying with option 1, create Amazon Athena view by executing the query under the CloudFormation output.

Screenshot of CloudFormation output

Note: If you run Contact Lens to generate both Raw and Redacted version, add appropriate filter in your query to avoid double counting of data.

Visualize in Amazon QuickSight

To automate Amazon QuickSight dashboard creation for Contact Lens, and to deploy available dashboards please follow “Automating Amazon QuickSight dashboard creation for analyzing Amazon Connect data” blog. To create a custom dashboard manually, follow below steps.

Using Athena to query the data allows you to visualize the results directly with Amazon QuickSight. Before connecting Amazon QuickSight to Athena, you must grant Amazon QuickSight access to Athena and its associated S3 bucket. For more information, see managing Amazon QuickSight permissions to AWS resources. After granting the access permissions, you can create a new dataset in Amazon QuickSight based on the Athena table that was created.

Next create a new analysis in Amazon QuickSight.

  1. Navigate to QuickSight and select ‘Analyses’.
  2. Click on “New Dataset, then select “Athena” and give the data source a name.

In the example below, the name chosen is ‘connect-ctr-cl-visualization’. Afterwards, click on Create data source.

Screenshot for Pop up to choose database and the table from the dropdown and select button

  1. Choose the name of the database and the source view/table based on the following deployment option, then click select:
    • If you are deploying with option 1, select view “{ProjectName}_AverageContactLensSentimentAndWpmPerAgentPerDate”
    • If you are deploying with option 2, select table connect_contactlens
    • Note: Create views within Amazon Athena to build enhanced visualization. For more details refer to this blog post.Screenshot of QuickSight, selecting view
  2. Finally, click Visualize to complete the dataset creation.
    Screenshot for Average sentiment in the QuickSight panel in the bar graph
  3. In the next screen you should see the CTR and Contact Lens data like below (if you are deploying with option 1)
    Screenshot of QuickSight showing Customer sentiment score by agent

After following the steps above, you can use Amazon QuickSight to add different columns from the call records and perform visualizations. Build enhanced visualizations, by creating views within Amazon Athena, check this blog post for more details. In addition, you can build dashboards that continuously monitor your connect instance and share those dashboards with others in your organization.

Clean up

In order to remove the resources created by the stack, perform the following:

  1. Delete the CloudFormation template for both CTR and Contact Lens solution.
  2. Delete the object and the S3 bucket created from the CloudFormation template. The Bucket name should contain the project name.
  3. Delete the Glue database created from the CloudFormation template.

Conclusion

In this post, you learned how to use services like AWS Lambda, AWS Glue, and Amazon Athena to process Amazon Connect CTR and Contact Lens data. The post also demonstrates how CTR and Contact Lens data is streamed via S3 Events, Kinesis Streams and queried by Athena leveraging AWS Glue. Finally, the post shows how to use Amazon QuickSight to perform visualizations. To build enhanced visualizations, by creating views within Amazon Athena, check this blog post for more details. To visualize your data in Tableau and Power BI, follow Building AWS data lake visualizations with Amazon Athena and Tableau and Using the Amazon Athena Power BI connector blog, respectively.

For more analytics capabilities on Amazon Connect data sources, refer the following blogs in the Amazon Connect reporting blog series:

Author bio

Mehmet Demir is a Senior Solutions Architect at Amazon Web Services (AWS) based in Toronto, Canada. He helps customers in building well-architected solutions that support business innovation
Ankur Taunk is a Senior Specialist Solutions Architect at AWS. He helps customer achieve their desired business outcomes in the Contact Center space leveraging Amazon Connect.