Building a survey IVR system with Amazon Connect

Contact centers commonly employ phone-based survey applications. These applications use interactive voice response (IVR) to obtain feedback on agent performance, the ease of conducting a transaction, or both. This post describes how to build a survey IVR using Amazon Connect. Although typical survey accept rates can vary depending on your workflow, coding a survey application using legacy systems often requires a full software development cycle.
This post demonstrates how to implement and deploy a phone-based survey using Amazon Connect.

Prerequisites

To follow along with the solution presented in this post, you need the following AWS services and features:

AWS Lambda
Amazon DynamoDB
Amazon Connect
Amazon Kinesis Data Streams
Amazon CloudWatch
AWS Identity and Access Management (IAM)
You also need an active AWS account with the permission to create and modify Lambda functions and DynamoDB tables.

To begin, you need an Amazon Connect instance configured for inbound and outbound calls. Claim a phone number after you create your instance. The Getting Started with Amazon Connect and Understanding Contact Flows documentation, and Getting started with Amazon Connect YouTube tutorial all provide valuable background knowledge for this process.

Solution overview

Contact centers commonly use two types of surveys:

In-call survey: The customer opts into the survey in the IVR, then speaks to an agent and ends the call. The system calls the customer back for the survey.
External survey: The customer opts into the survey while talking with the agent. After the customer hangs up with the agent, the system calls the customer back for the survey.

The agents’ interaction with the customer may be biased during an external survey, as they know whether the interaction may be used to gauge their performance. An in-call survey is less biased as the agent remains unaware if the customer opted into the survey. This blog post presents an option to implement an in-call survey.

The survey sequence consists of the following steps:

The customer calls into the IVR. The IVR offers a survey option after the call. The customer opts in for a survey, and the system transfers the call to an agent.
The call routes to the most available agent via their Contact Control Panel (CCP).
The customer and agent talk. The customer ends the call.
Amazon Connect automatically moves the agent to the “after contact work” state. The agent manually changes to the “available” state.
Amazon Connect updates the contact trace record (CTR). This setup requires configuring CTR streaming via Amazon Kinesis Data Streams.
Once Amazon Connect finishes updating the CTR, the Kinesis data stream triggers an AWS Lambda function.
The Lambda function starts the survey IVR.
The survey IVR calls the customer and presents the survey question.
The survey IVR stores the customer’s survey response in a DynamoDB table for future analytics.

Walkthrough

To build the survey IVR, this post includes the following steps:

Preparing the environment
Setting up DynamoDB tables
Setting up and configuring Kinesis Data Streams in Amazon Connect
Setting up the Lambda functions
Logging in to the CCP
Configuring the Amazon Connect contact flow

Preparing the environment

You must implement several IAM policies to grant Lambda access to prepare the environment. The Lambda handler requires the following identity-based IAM policies.

Lambda #1 (ProcessCCPEventStream): Kinesis Data Streams triggers these policies when a CTR is ready.

Kinesis read-only access: [CF1] The policy name used for this is = AmazonKinesisReadOnly
[CF1]Please include a sentence explaining what this policy does.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kinesis:Get*",
                "kinesis:List*",
                "kinesis:Describe*"
            ],
            "Resource": "*"
        }
    ]
}

DynamoDB read/write access to the SurveyCustomerInfo table: This policy grants read/write access to the customer survey table. To create a policy that has access to read and write to a specific table, see Amazon DynamoDB: Allows Access to a Specific Table. Replace “MyTable with SurveyCustomerInfo. Policy Name = DDBSurveyCustInfo

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ListAndDescribe",
            "Effect": "Allow",
            "Action": [
                "dynamodb:List*",
                "dynamodb:DescribeReservedCapacity*",
                "dynamodb:DescribeLimits",
                "dynamodb:DescribeTimeToLive"
            ],
            "Resource": "*"
        },
        {
            "Sid": "SpecificTable",
            "Effect": "Allow",
            "Action": [
                "dynamodb:BatchGet*",
                "dynamodb:DescribeStream",
                "dynamodb:DescribeTable",
                "dynamodb:Get*",
                "dynamodb:Query",
                "dynamodb:Scan",
                "dynamodb:BatchWrite*",
                "dynamodb:CreateTable",
                "dynamodb:Delete*",
                "dynamodb:Update*",
                "dynamodb:PutItem"
            ],
            "Resource": "arn:aws:dynamodb:*:*:table/ SurveyCustomerInfo"
        }
    ]
}

Amazon Connect: This policy places the outbound call. Policy Name = AWSConnect-GrantOutboundPermission.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "connect:StartOutboundVoiceContact",
            "Resource": "arn:aws:connect::*:instance/<InsertAmazonConnectInstanceName>/contact/*"
        }
    ]
}

NOTE: Change <InsertAmazonConnectInstanceName> with your Amazon Connect instance name. 

Write to CloudWatch Logs. Policy name = WriteCloudWatchLogs. JSON noted below. 
{
"Version": "2012-10-17",
"Statement": [{
            "Action": ["logs:*"],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
 }

Lambda #2 (UpdateSurveyResult): This handler writes the survey results to the DynamoDB table labeled SurveyCustomerInfo. This is the same policy created for the previous DynamoDB read/write access to the SurveyCustomerInfo table policy.

Setting up the DynamoDB table

Create the table SurveyCustomerInfo. This table serves as a lookup table to retrieve the customer’s name. It also stores survey results. Configure the table with the following attributes:

Attribute: ‘PhoneNumber’
Key Type: Primary Key/Partition Key
Attribute Type: String
Table Region: US-east-1.

Setting up Amazon Connect to send CTRs to Kinesis Data Streams

Export CTRs via Kinesis Data Streams so that the Lambda function outlined in the next section can read it. After you set up your Kinesis data stream—named CCPAgentEvent in this example—navigate to the Amazon Connect Instance landing page. From here, following these steps to configure Amazon Connect to use that stream:

Navigate to the Amazon Connect instance setup screen. Choose Data streaming.
Choose Enable data streaming.
Under Contact Trace Records, select Kinesis Stream.
Type the name of your Kinesis data stream in the text box (CCPAgentEvent in this example).

Setting up the Lambda functions

This survey IVR system requires two new Lambda functions:

ProcessCCPEventStream
UpdateSurveyResult

After you set up the Kinesis data stream CCPAgentEvent, add it as a trigger to this Lambda function.

Lambda function: ProcessCCPEventStream

This Lambda function requires the following IAM policies:

WriteCloudWatchLogs
DDBSurveyCustInfo
AWSConnect-GrantOutboundPermission
AmazonKinesisReadOnlyAccess

import json
import base64
import boto3
from boto3 import resource
from boto3.dynamodb.conditions import Key

dynamodb_resource   = resource('dynamodb')
surveyCustomerTable = dynamodb_resource.Table('SurveyCustomerInfo')

CCPEventSource = 'arn:aws:kinesis:us-east-1:NNNNNNNNNN:stream/CCPAgentEvent'


#############################################################
## Connect Instance Details - edit to change 
## Contact Flow Name = OutboundLocationBasedCallback
#############################################################
theAmazonConnectPhoneNumber = '+19729999999'
theInstanceId='f633f7ae-84ad-4755-9fca-NNNNNNNNNNNN'
theContactFlowID = '351c39fb-273a-4570-a9bc-NNNNNNNNNNNN'

def lambda_handler(event, context):
#    print('lambda_handler(event) ==>', event)
    for aRecord in event['Records'] :
        eventSource = aRecord['eventSourceARN']
        if eventSource == CCPEventSource :
            payload = base64.b64decode(aRecord['kinesis']['data'])
            try:
                processEvent(payload)
            except Exception as ex:
                return {
                    'isBase64Encoded': 'False',
                    'statusCode': 999,
                    'Echo': 'Exception'
                    }
    return {
        'isBase64Encoded': 'False',
        'statusCode': 200,
        'Echo': 0
        }

def processEvent(payload) :
    jsonPayload = json.loads(payload.decode('utf-8'))
    isSurveyCandidate = ''
    print('processEvent(payload) ==> ', jsonPayload)
    
    try:
        isSurveyCandidate = jsonPayload['Attributes']['SurveyCandidate']
    except Exception as ex:
        raise ex
    
    if isSurveyCandidate == 'True' :
        phoneNumber = jsonPayload['CustomerEndpoint']['Address']
        print('Will call ', phoneNumber,' for survey')
        invokeOutboundIVR(phoneNumber, jsonPayload['ContactId'])



def invokeOutboundIVR(customerPhoneNumber, contactID) :
    theConnecClient = boto3.client('connect')
    try:
        response = theConnecClient.start_outbound_voice_contact(
	        DestinationPhoneNumber= customerPhoneNumber
	        ,ContactFlowId=theContactFlowID
	        ,InstanceId=theInstanceId
	        ,SourcePhoneNumber=theAmazonConnectPhoneNumber
	        ,Attributes={'OriginalContactID': contactID}
	    )
    except Exception as ex:
     raise ex
    
def retrieveCustomerName(aPhoneNumber):
    try:
        response = surveyCustomerTable.query(
                                    KeyConditionExpression=Key('PhoneNumber').eq(aPhoneNumber)
                                    ,ScanIndexForward=False
                                    ,Limit=1
                                    )
    except Exception as ex:
        raise ex
    else :
        if 1 == response['Count'] :
            return response['Items'][0]['CustomerName']
        else :
            return None

Update the following four fields after creating the Amazon Connect outbound survey call flow:

CCPEventSource: This is the Amazon record locator (ARN) for the data stream.
theAmazonConnectPhoneNumber: This is the phone number selected for the Amazon Connect call flow BasicCallFlow described later in the “Configuring the Amazon Connect Contact Flow” section.
theInstanceId: This is the instance ID of the Amazon Connect instance.
theContactFlowID: This is the contact flow ID of the OutboundSurvey contact flow (described below).

You can find both the instance ID and the contact flow ID listed in the ARN within the Additional flow information panel of the Contact flow designer.

Function #2: UpdateSurveyResult

This Lambda function requires the WriteCloudWatchLogs and DDBSurveyCustInfo IAM policies

import json
import boto3
import datetime
import time
from boto3 import resource
from boto3.dynamodb.conditions import Key

dynamodb_resource   = resource('dynamodb')
surveyCustomerTable = dynamodb_resource.Table('BlogTestSurveyCustomerInfo')

def lambda_handler(event, context):
    print('def lambda_handler(event, context): ==> ', event)
    aContactID  = event['Details']['ContactData']['Attributes']['OriginalContactID']
    aRating     = event['Details']['ContactData']['Attributes']['SurveyResponse']
    aPhoneNumber= event['Details']['ContactData']['CustomerEndpoint']['Address']
    
    updateSurveyResults(aContactID, aPhoneNumber, aRating)
    return {
        'statusCode': 200,
        'body': 'Ok'
    }


def queryPhoneNumber(aPhoneNumber):
    try:
        if aPhoneNumber:
            response = surveyCustomerTable.query(KeyConditionExpression=Key('PhoneNumber').eq(aPhoneNumber))
    except Exception as ex:
        return None
    else :
        return response


def updateSurveyResults(aContactID, aPhoneNumber, nRating) :
    if aPhoneNumber is not None and nRating is not None :
        surveyData = returnSurveyDataObject(aContactID, nRating)

        retVal = queryPhoneNumber(aPhoneNumber)
        if retVal['Count'] == 0 :
            response = surveyCustomerTable.put_item(Item={
                    'PhoneNumber': aPhoneNumber,
                    'SurveyData': [surveyData]
            })
        else :
            try:
                response = surveyCustomerTable.update_item(
                         Key={
                           'PhoneNumber': aPhoneNumber
                           },
                         UpdateExpression="set SurveyData = list_append(SurveyData, :obj)",
                         ExpressionAttributeValues={
                         ':obj': [surveyData],
                         },
                         ReturnValues="UPDATED_NEW"
                        )

                if 200 == response['ResponseMetadata']['HTTPStatusCode'] and 'Attributes' in response :
                    return response
            except Exception as ex:
                raise ex
            else :
                return response
    else :
        return None


def returnSurveyDataObject(aContactID, nRating) :
    
    ts = time.time()
    surveyData = {}
    surveyData['ContactID'] = str(aContactID)
    surveyData['TimeOfCall'] = str(datetime.datetime.fromtimestamp(ts).strftime('%Y-%m-%d %H:%M:%S'))
    surveyData['NPS_Rating'] = str(nRating)
    return json.dumps(surveyData)

Logging in to the CCP

After setting up your environment, creating your DynamoDB table, and completing your Lambda functions, start and log in to CCP using Amazon Connect.

Configuring the Amazon Connect contact flow

You must configure your Amazon Connect instance to use the Lambda function that you created earlier.

Basic call flow to route to an agent (Flow Name = BasicCallFlow): The basic call flow to send a call to an agent follows a simple process.

Set up the Play prompt block to speak the following text: “Thank you for calling. We will call you for a one question survey at the end of the call.”
Set up the Set contact attributes block as follows:
Use Text / Destination Key = SurveyCandidate / Value = True
Set up the Set working queue block to pick the Basic Queue. Make sure to map the agent’s routing profile to this queue.
Make sure that you select and assign a phone number to the call flow after publishing and saving it.

Survey IVR call flow (Flow Name = OutboundSurvey): The basic call flow to survey a customer follows a similarly simple process.

Set up the Store customer input box to play the following text and capture the customer’s response. “How likely are you to recommend us to a friend on a scale of 1 to 5. 1 being the least likely and 5 being most likely.”
Set up the Set contact attributes box to store values entered by the customer in a local variable called SurveyResponse. Use this configuration:
Use Attribute : Destination Key = SurveyResponse. Type = System. Attribute = Stored Customer Input
Set up the Play prompt block as output from the Set contact attributes block. Have it play the following: “Thank you for rating us $.Attributes.SurveyResponse.”
The variable $.Attributes.SurveyResponse allows playback of the rating selected by the customer.
Set up the Invoke AWS Lambda function box to store the attributes Survey Results in a DynamoDB table by calling the Lambda function UpdateSurveyResult.

Execution

If you followed the steps to build your survey IVR system, it fulfills the following steps.

Call the phone number assigned to the flow BasicCallFlow. The flow automatically enters the caller for a callback survey. (this example uses the caller ID +12549999999)
After the agent interaction completes and the agent sets themselves to “Available,” the Lambda Function ProcessCCPEventStream invokes the outbound call flow OutboundSurvey. This flow uses the same number assigned to the BasicCallFlow and calls the customer on their phone number (+12549999999) with a single survey question.
After the customer responds to the survey question, the system stores the response in DyanmoDB along with the ContactID for the call and timestamp, as shown in the following JSON:

{
  "PhoneNumber": "+12549999999",
  "SurveyData": [
    "{"ContactID": "c84b846e9d4b", "Date": "2019-04-24 17:36:31", "NPS_Rating": "5"}",
    "{"ContactID": "dfe518babceb", "Date": "2019-04-24 17:40:03", "NPS_Rating": "2"}",
    "{"ContactID": "739634da08c0", "Date": "2019-04-24 17:42:37", "NPS_Rating": "4"}",
    "{"ContactID": "d1b8fd7211fa", "Date": "2019-05-29 17:14:46", "NPS_Rating": "2"}",
    "{"ContactID": "99e242603dca", "Date": "2019-05-29 17:16:44", "NPS_Rating": "1"}",
    "{"ContactID": "3d1b47b81a23", "Date": "2019-05-29 17:22:03", "NPS_Rating": "1"}",
    "{"ContactID": "75a8eaa33b9c", "Date": "2019-05-29 17:24:04", "NPS_Rating": "2"}",
    "{"ContactID": "b07007156e49", "Date": "2019-05-29 17:25:56", "NPS_Rating": "1"}",
    "{"ContactID": "365a2c17cf99", "Date": "2019-05-30 18:07:09", "NPS_Rating": "2"}",
    "{"ContactID": "077f5577d09a", "Date": "2019-05-30 19:18:50", "NPS_Rating": "2"}",
    "{"ContactID": "7ef8ff792000", "Date": "2019-05-30 19:55:37", "NPS_Rating": "2"}"
  ]
}

Conclusion

In this post, you created a basic survey application using Amazon Connect. You can now extend this concept to create complex dynamic surveys, where the questions change based on the IVR transactions. Results can be used to provide insight into your IVR application.

The example in this post uses DynamoDB to store customer information. In actual scenarios, you could substitute a CRM database, either on-premises or in the cloud. In this case, you must modify the calls from Lambda.

This post uses a simple CCP. In actual scenarios, you can place the check box within a larger CRM application.

Although this post required the agent to mark a call for a survey, you can eliminate this step and have the customer opt in for the survey in the IVR itself. In this case, the Lambda function to call Amazon Connect and conduct a survey should call only if the customer first opted to participate in the survey.

AWS Contact Center