AWS Public Sector Blog

How to create a task-generating voicemail solution with Amazon Connect

It’s time consuming for public sector organizations to sort through packed voicemail inboxes, figure out how to respond, return calls, and take detailed records on each call. Consider a common issue for healthcare systems: How does a hospital employee report that they are sick, especially during a pandemic? A common approach for healthcare organizations is to set up a voicemail box for the sick employee to leave a message and for a team of nurses to review and return the employee’s call, but this approach quickly does not scale. Voicemail boxes get full, it’s difficult to keep up with return calls in a busy hospital environment, and records about calls vary.

Working backwards from this common problem, Amazon Web Services (AWS) developed a solution that scales voice interactions by automating the tasks of listening, capturing, tabulating, and distributing the content from voicemail messages. Organizations can now quickly provision an artificial intelligence (AI) and machine learning (ML)-powered call centers in the AWS Cloud using Amazon Connect. In our hospital example, this lets sick staff to quickly leave a message and get back to bed. Amazon Connect generates tasks for the nurses, and they receive a transcription, a recording, and caller details of the call. Additionally, natural language processing (NLP) is performed on the transcription of the call to extract symptoms the caller has reported using Amazon Comprehend Medical. Responding nurses can focus on the medical needs of the employee and spend less time playing phone operator. All these attributes are stored in an Amazon DynamoDB table. From this table, a business workflow or dashboard reporting of calls can be made available to organization medical staff and leadership.

Using Amazon Connect, nurses can dial patients directly using the Contact Control Panel and leave a common number to which sick employees can return the call. Sick employees receive calls from the same number every time, and if they miss it, can call the number back to reach the hotline.

This blog post guides you through how to create this task-generating cloud call center solution with Amazon Connect:

Prerequisites

To follow along with the solution presented in this blog post, you need the following AWS services and features:

  1. Amazon Connect
  2. Amazon Lex
  3. AWS Lambda
  4. Amazon DynamoDB
  5. Amazon Kinesis Video Streams
  6. Amazon Transcribe
  7. Amazon Comprehend Medical
  8. Amazon Simple Storage Service (Amazon S3)
  9. AWS CloudFormation
  10. AWS Identity and Access Management (IAM)
  11. An active AWS account with the permission to create and modify IAM

Solution overview

Figure 1. Amazon Connect Task with call transcription architecture.

In the architecture shown in Figure 1, the following sequence of steps takes place for an incoming contact:

1. Start Media Streaming block helps stream audio from a customer, to the customer, or both.

a. Once the Start Media Streaming block is triggered, a Kinesis Video Stream is assigned and Amazon Connect begins to stream the audio.

b. Amazon Connect continues to stream the customer audio for the duration of this call until a Stop media streaming block is executed, or the call is disconnected.

2. In the Amazon Connect Contact Flow, Lambda Function automatically passes the Kinesis Video Stream details and the ContactId.

a. Tip: Set the following Contact Attributes prior to invoking the trigger lambda function:

    • key: transcribeCall, value: true or false
    • key: saveCallRecording, value: true or false
    • key: languageCode, value: en-US or es-US
    • key: streamAudioFromCustomer, value true or false
    • key: streamAudioToCustomer, value true or false

3. Amazon Connect Contact Flow invokes the kvsConsumerTrigger function with all the Contact Attributes set in the previous step.

4. The kvsConsumerTrigger function writes the ContactID, CustomerPhoneNumber, Date, and Time to the DynamoDB table ContactDetails.

5. The kvsConsumerTrigger function invokes the kvsTranscriber function passing it all the attributes needed to start consuming the Kinesis Video Stream (call audio). The Contact Flow continues to execute while the kvsTranscriber function processes the audio stream. The function processes audio for up to 15 minutes or until the call is disconnected.

6. The kvsTranscriber function sends the audio to Amazon Transcribe to transcribe the audio from speech to text. The Kinesis Video Stream has two channels: one for audio to customer and one for audio from customer. Each audio channel is transcribed separately.

7. The kvsTranscriber function writes the transcribed text to two different DynamoDB tables one for audio from customer, transcriptSegments, and one for audio to customer, transcriptSegmentsToCustomer.

8. The kvsTranscriber function writes two audio files AUDIO_FROM_CUSTOMER.wav and AUDIO_TO_CUSTOMER.wav to the designated Amazon S3 bucket in the CloudFormation template with ContactID, date, and time prepended to the filenames.

9. The S3 bucket has a trigger that looks for files to be written with a .wav extension and invokes the Lambda processContactSummary.

10. The processContactSummary queries the contactTranscriptSegments tables to retrieve the transcript.

11. The processContactSummary then updates the contactDetails table with the contacts transcripts using the ContactID as the key.

12. The processContactSummary invokes the overlay-audio Lambda that takes the two audio file names, FROM_CUSTOMER and TO_CUSTOMER, and combines them.

13. The processContactSummary writes the combined audio file to the combined audio S3 bucket.

14. The processContactSummary writes the combined audio file to the combined audio S3 bucket.

15. The DynamoDB stream triggers Lambda function, passing the transcription and audio recording URL as part of event data.

16. The Lambda function passes the transcription to Amazon Comprehend and Amazon Comprehend Medical to get the key phrases and symptoms respectively.

17. The Lambda function generates the pre-signed URL to the audio recording.

18. The Lambda function invokes Amazon Connect Task API to create a task and passes the transcription, pre-signed audio recording URL, key-phrases, symptoms, and customer captured information within the call flow.

Procedure

Before you begin:

Before deploying the solution mentioned in this blog, you must deploy the AWS CloudFormation template described in Amazon Connect real time transcription post. After the successful completion of the CloudFormation template, complete the steps described in this guide, to follow.

This blog post guide is implemented using python. The deployment is done using AWS Cloud Development Kit (AWS CDK). Make sure that you have installed:

  1. aws-sdk with pip install boto3 
  2. aws-cdk with npm install -g aws-cdk

For more information about AWS CDK, please refer to the Getting Started the AWS CDK Guide.

Deploy and configure

1. Clone the solution to your computer:

a. git clone https://github.com/aws-samples/connect-task-creation-for-voicemail

2. Follow the instructions from the README.md file to prepare the environment.

3. Make sure that you set the proper AWS_PROFILE in your environment (as this set of account/region/credentials will be used by AWS CDK).

a. On Mac: ‘export AWS_PROFILE=user1‘ where user1 is the name of your AWS profile in the $HOME/.aws folder.

b. On Windows: ‘setx AWS_PROFILE user1‘ where user1 is the name of your AWS profile in the $HOME/.aws folder.

4. Bootstrap your account and region: cdk bootstrap

5. Go to the AWS DynamoDB Console, select table contactDetails

a. Select Manage DynamoDB Stream, select New and old images, and Enable the stream.

b. Note the Latest stream ARN

Figure 2. The DynamoDB stream details.

4. Update the cdk.json file with appropriate context parameters (instanceId, contactFlowId, and eventSourceArn).

a. eventSourceArn is the Latest stream ARN value in Step 2.

5. Deploy the solution cdk deploy, and wait for it to finish.

6. Check the deployed resource by going to the AWS CloudFormation Dashboard, select that stack that has just been deployed (record-comprend-task), and select the Resources tab. Note the following lambdas, whose Amazon Resource Names (ARNs) will be used in the Contact Flow in Step 9:

i. Logical ID starts with comprehendtasklambda (every deployment will be different, but will begin with this notation).

ii. Logical ID starts with trackingtasklambda

Figure 3. Deployed resources by the Cloud Formation Template.

7. Import the Amazon Lex bot:

a. Go to the Amazon Lex Console.

b. On the Actions dropdown menu, choose Import.

c. Browse to assets folder and select the RecordComprehendTask.zip file to import.

d. Create and publish the Lex bot.

8. Enable the Lex bot for use in your Contact Flow.

a. Go to the AWS Connect Console, select your instance, then select Contact Flows on the left panel.

b. Add the Lex bot that you created in Step 4 for use in your Contact Flows.

Figure 4. Select the Bot in Amazon Lex Console.

9. Enable the Lambdas for use in your Contact Flow in the Amazon Connect Console (see Figure 5).

Figure 5. Adding the lambdas to be used in Connect Contact Flows.

10. Import the Contact Flow.

a. Open the kvsStreamingFlow CF deployed in the prerequisite Amazon Connect real time transcription solution.

b. Replace it by importing thekvsStreamingFlow.json CF in the assets folder.

11. Update the kvsStreamingFlow CF with appropriate Lex bot and lambdas.

a. Update the Invoke AWS Lambda function block with the kvsConsumerTrigger lambda deployed in the prerequisite Amazon Connect real time transcription solution.

b. Update the Get customer input block with the Lex bot deployed in Step 4.

c. Update the Invoke AWS Lambda function block with the trackingtasklambda function deployed in Step 3.

d. Go to the AWS Connect Console, select your instance, then select Contact Flows on the left panel.

e. Add the two lambdas you noted in Step 3 for use in your Contact Flows.

f. Note that the lambda that starts with kvsConsumerTrigger should’ve already been added when deploying the pre-requisite Amazon Connect real time transcription.

Figure 6. Imported Contact Flow for the solution.

Testing

1. In your Amazon Connect instance, choose the Routing icon from the navigation bar, and choose phone numbers.

2. Choose the phone number you want to associate with your new contact flow to edit.

3. Choose the name of your contact flow from the Contact flow/IVR drop-down menu, and choose Save.

4. In your Amazon Connect instance, choose the phone icon in the top right corner to launch Contact Control Panel (CCP) to take call.

5. To make a test call, call the phone number you associated to the flow.

Clean up

To remove (or undeploy the solution), perform the following steps:

1. Run cdk destroy on the stack deployed in this solution.

2. Go to the Amazon Lex Dashboard and delete the Lex bot created in Step 6.

3. Run cdk destroy on the stack deployed in the prerequisite Amazon Connect real time transcription.

Conclusion and learn more

Public sector organizations can immediately provision an AI-powered call center on-demand to rapidly respond to events. Amazon Connect scales to handle surges in inbound call capacity, outbound calling, and augment operators with powerful AI tooling such as conversational bots, call recording, transcription, translation, and entity extraction to gather critical information from callers. With the implementation capturing key inputs, public sector organizations can collect the data they need to make informed decisions about their response. Provision Amazon Connect for your organization in minutes today; contact us to learn more.

Read more stories about Amazon Connect in the public sector.

Subscribe to the AWS Public Sector Blog newsletter to get the latest in AWS tools, solutions, and innovations from the public sector delivered to your inbox, or contact us.

TAGS: , , , , , , , , ,
Ankur Taunk

Ankur Taunk

Ankur Taunk is a specialist solutions architect at AWS supporting public sector customers. He works with public sectors customers to better understand their business, technical challenges, and how to achieve their desired business outcomes in the contact center space leveraging Amazon Connect.

Andrew Lieberthal

Andrew Lieberthal

Andrew Lieberthal is a principal solutions architect focused on bringing innovative solutions using cloud technologies to public sector and national security organizations.

Tu Do

Tu Do

Tu Do is a principal solutions architect at Amazon Web Services (AWS). Previously, he worked in Amazon Alexa leading Alexa Communications development. Prior to that, he worked at BlackBerry, Avaya, Nimcat Networks, and Nortel Networks on various telecom and real-time products and services. He is passionate about architecting and building innovative solutions to bring business success to customers, especially using cloud platforms and technologies.