Measure latency for validation testing and troubleshooting in Amazon Connect

Setting up a contact center for your business is an important step in the direction of offering an enhanced customer experience to your end users. In such a scenario, it is vital that one should follow best practices as a part of the planning phase for your contact center to ensure smooth execution of deployment. An end-to-end testing of your Amazon Connect set up is essential to a successful production release and technical operation of your contact center instance.

In the AWS Well-Architected Framework, Operational Excellence is the first pillar to consider when evaluating, migrating, and implementing workloads on AWS. When preparing for operational excellence with Amazon Connect, you will need to determine which Amazon Connect Region to select according to your use case, telephony costs in each Region, data governance requirements, and latency in relation to your agents, contacts, and external transfer endpoint geography. In global deployments or situations where your Amazon Connect instance, agents, or contacts are in geographically distant locations, you should measure latency for quality of experience as part of validation testing prior to launching your workload.

While there are three stages to Operational Excellence – Prepare, Operate, and Evolve, let us look at a few recommendations we have in the ‘Prepare’ stage necessary for Amazon Connect:-

1. Checking agent-side network latency to AWS resources and ensuring necessary ports are open for softphone using the Amazon Connect Call Control Panel (CCP) -Connectivity tool
2. Set up your network
3. Troubleshooting common issues
4. Measuring latency and Connect instance region selection for global deployments.

This blog post will focus on diving deep into the fourth point of how you can measure latency between agents and callers especially in scenarios where your Amazon Connect instance is geographically distant from both. For example, when your instance is in US-East (North Virginia) but callers and agents are based out of Tokyo, Japan. You could also use these results to evaluate the need for adopting different methods to mitigate effects of latency using the recommendations provided in the ‘PSTN and Agent Connection Latency’ section of Amazon Connect Administrator Guide.

Solution Overview:

Latency/cross-talk—in a voice connection manifests as a delay between when something is said and when the person on the other end hears it. In some use cases that requires a lot of conversation, high latency can create situations in which both parties are talking over each other. The PSTN and agent latency need to be calculated in this scenario to identify contributing factors and take action to reduce PSTN latency, agent latency, or both. This becomes particularly important for global footprint Amazon Connect instances that service agents and callers from geographically distant locations. Overall Latency becomes noticeable at 300ms and may affect quality of experience above 500ms as also mentioned in the Interpreting the Test Results section of the Amazon Connect Administrator Guide.

Before we begin, it would be worthwhile to spend some time on understanding a few basic terminologies that we would use often in the blog post going forward.

1. Overall Latency: This is the total latency experienced between a caller and an agent. This is the measure of how long it takes the audio to travel from the caller to the agent.
2. PSTN Latency: This is the latency between the caller and the Amazon Connect endpoint. This is the measure of how long it takes the audio to travel from the caller to Amazon Connect.
3. Agent Latency: This is the latency between the Amazon Connect endpoint and the agent. This is the measure of how long it takes the audio to travel from Amazon Connect to the agent.

All of these can be summarized in one equation:-

Overall Latency = PSTN Latency + Agent Latency

A diagrammatic representation of what I described is shown ahead:-

Prerequisites:

This blog post assumes that you know how to set up an Amazon Connect instance, enable call recording, access the call recordings and create a basic Amazon Connect Contact flow that can reach to an agent. Make sure you are following the best practices by having appropriate agent workstation requirements in place and have set up your network as mentioned in the Amazon Connect Administrator Guide.
Most importantly, the post also assumes that you have read the steps of latency validation testing.

The following links can help you get started:-

1. Get Started with Amazon Connect
2. Create Amazon Connect Contact Flows
3. Set up recording behavior with Amazon Connect
4. Troubleshooting and Best Practices
5. Validation Testing

Steps to measure latency

The steps to measure latency are given under the title Test Inbound Calls using Softphone in the Validation Testing section of the Amazon Connect Administrator Guide. Once you have run through the steps you should have captured two recording files- one using the audio analysis tool of your choice running on the agent desktop and the other using the native call recording capability within Amazon Connect. You will use these recording files to analyze your results and calculate latency.

In this post, I will use sample test recordings and walk through the procedure to analyze them and calculate the results using a guided visual example.

Guided example overview

In this example, the Amazon Connect instance has been set up in Sydney, Australia and the agents as well as callers are based out of Seattle, US. I will also be using Audacity as the audio analysis tool. This tool will run on the agent desktop to capture sound recordings locally as received by the agent. (Please note: You can use any tool of your preference or choice)

After running through the steps given in Validation Testing section of the Amazon Connect Administrator Guide, for the above described scenario, I obtained two set of recording files-

1. Audacity’s recording -running locally on agent desktop
2. Amazon Connect’s native call recording

A copy of each of these files and the analysis of latency results for each of these tests has been shared ahead.

Please note, the examples below have a separate screenshot for every tap, however all the three taps were a part of a single recording file and obtained as a part of a single sequential test.

Audacity recording analysis

1. You can download the Audacity recording file of our test call for reference.
2. Latency measurement for the first tap is as below. The audio recording will show a series of taps, the first one being the initial sound made from the tap on customer’s desk, and the recurring one is the sound heard from the agent’s speakers. In this example, we have not separated the two sounds as different tracks and you can see them in the same time series.

Latency for this given peak is recorded as the difference between the timestamp for the two taps = T2-T1 = 5.779 s – 5.306 s = 473 ms (L1)

     3. Similarly, we can calculate the latency for the second tap

Latency for this given peak is recorded as the difference between the timestamp for the two taps = T2-T1= 11.24 s – 10.774 s = 470 ms (L2)

    4. And finally we can measure the latency for the third tap as well

Latency for this given peak is recorded as the difference between the timestamp for the two taps = T2-T1 = 16.985 s – 16.512 s = 473 ms (L3)

    5. Now that we have all the three latencies, we can calculate the average of them to get the Overall Latency

Overall Latency = (L1+L2+L3)/3 = 473 + 470 + 473 = 472 ms

Amazon Connect native recording analysis

1. You can download the recording file  obtained from the native recording of Amazon Connect, of our test call for reference. Open the file in Audacity ( or your audio analysis tool)
2. Latency measurement for the first tap is as below. In this example, the recording file has two tracks. The lower one showcases the sound of the tap made from the customer’s desk. The upper track shows the tap sounds received at the agent end. The results for the first tap are as below:-

Latency for this given peak is recorded as the difference between the timestamp for the two taps  T2-T1 = 17.658 s – 17.528 s = 130 ms (L1)

     3. Similarly, we can calculate the latency for the second tap

Latency for this given peak is recorded as the difference between the timestamp for the two taps = T2-T1 = 23.138 s – 23.007 s = 131 ms (L2)

    4. And finally we can measure the latency for the third tap as well

Latency for this given peak is recorded as the difference between the timestamp for the two taps = T2-T1 = 28.866 s – 23.750 s = 116 ms (L3)

    5. Now that we have all three latencies, we can calculate the average to get the PSTN Latency

PSTN Latency = (L1+L2+L3)/3 = 130 + 131 + 116 = 125.66 ms

Calculating agent latency 

As per the formula : Overall Latency = PSTN Latency + Agent Latency,

we can determine the value for agent latency where,

Agent latency= Overall Latency – PSTN Latency = 472-125.66 = 346.43 ms

Summary

Using the above steps, you can measure the audio latency for your call center configuration. It is important to recreate the actual use case as closely as possible for the most accurate results. For example, if you have a call forwarding set up with your Amazon Connect instance; make sure you test in an environment where the call forwarding is recreated as this may add to the PSTN latency. While your geographic location, agent set up or use case may vary (transfer, conference, etc.), it is important to note the concept remains the same and you can apply the same logic to test your individual scenarios.