Redacting sensitive data in the chat message streaming API for Amazon Connect

Introduction

Organizations today strive to provide increasingly personalized customer journeys. For example, up-selling new products based on history of interest in related products. Data collection drives the ability for businesses to personalize – data is valuable and the more collected, the more impact a personalized experience can have.

Businesses must be able however to balance the collection of data with commitments to data privacy and managing PII (Personally Identifiable Information). Sensitive PII data can be redacted in real time from Amazon Connect chat messages, preventing sensitive data from being passed between customers and agents. Such data can often be innocently shared, such as over informal social media channels, but becomes difficult or impossible to identify and purge once uttered and recorded in chat transcripts. Redacting sensitive data encourages frank and open interactions, rich in customer insight and data, while reducing the risk of inadvertently capturing sensitive data.

The benefits of the described solution apply to customers, agents, and compliance stakeholders alike:

• Protects customers from accidentally revealing PII
• Protects agents from having to handle PII that is unintentionally shared
• Protects businesses from having to identify and purge PII from data stores such as chat histories
• Minimizes risk of PII compliance breaches e.g., financial account details being shared

Solution overview and architecture

The example project detailed here is available from this GitHub repository. This project includes end to end infrastructure to support the Facebook Messenger channel.

Figure 1: Solution architecture

1. Customer sends a message from the digital messaging channel to the webhook hosted on Amazon API Gateway.
2. API Gateway sends the message to AWS Lambda.
3. AWS Lambda writes the chat contact context to Amazon DynamoDB.
4. AWS Lambda invokes the Amazon Comprehend API to redact sensitive data.
5. If this is the first message for the contact, Lambda calls the API operations in the following order: StartChatContact, StartContactStreaming, and CreateParticipantConnection. If there is an existing chat, Lambda will send the message to Amazon Connect.
6. Amazon Connect streams Agent/System messages to Amazon SNS. Amazon Simple Notification Service (Amazon SNS) is a fully managed pub/sub messaging, SMS, email, and mobile push notifications.
7. Amazon SNS invokes AWS Lambda
8. Lambda queries DynamoDB for chat contact context.
9. Lambda delivers the reply message to the customer through APIs from the source channel.

Prerequisites

For this walkthrough, it is assumed that you understand and have access to the following resources:

• An AWS account with administrator access
• An Amazon Connect Instance
• An Amazon Connect Contact Flow setup for Amazon Connect (include Disconnect flow)
• AWS CLI setup in your local environment
• A Facebook developer account. For more details, visit the Facebook Developer console.
• Node.js installation on the developer machine with NPM. Click here for more details.

Deployment walkthrough

Facebook Developer console

1. Navigate to the Facebook Developer console.
2. Choose My Apps.
3. Select Create App (or use an existing App).

4. Choose an app type for your business. For this walk through choose Business, which supports messenger.
5. Choose Next.

6. Provide a display name, contact email, app purpose.
7. Choose whether to attach business manager account.
8. Choose Create app.
9. Navigate to Settings, Basic.
10. Next to App Secret select Show. Store this value for later use.

11. Complete settings including Privacy Policy URL, Terms of Service URL and Contact email as desired. Only Contact email is required for private testing purposes.
12. Navigate to Settings, Advanced, Security. Add relevant details.
13. Turn on Require App Secret.
14. Choose Save Changes.
15. Navigate to Dashboard, Messenger.
16. Choose Setup.

17. Under Access Tokens, add a page or create a new page
    1. This may prompt you for a login review. Follow the steps there before proceeding.

18. After the page has been added, choose Generate Token. Save this value on the side

Set up AWS Secrets Manager

1. Open the AWS Management Console and login with an AWS Account with admin access where Amazon Connect instance is set up.
2. Navigate to AWS Secrets Manager console, Store a new secret.
3. Choose Other type of secrets.
4. Under Secret key/value add APP_SECRET, PAGE_TOKEN and VERIFY_TOKEN
   1. APP_SECRET – App Secret generated previously.
      PAGE_TOKEN – Access Token generated previously
      VERIFY_TOKEN – For the verify token, use any random string. This will be added in Facebook later when the webhook Callback URL is created.
5. Select the default encryption key.
6. Choose Next.
   1. Note: feel free to add a new key, however, make sure to modify the CDK project to provide permissions to that encryption key.
7. Provide Secret name and Description.
8. Choose Next.
   1. Note: If you add any additional resource permissions or other settings, CDK stack resources must be given permissions to this secret.
9. Choose Next, and Store the secret.
   1. Note: you can configure automatic rotation setting based on your requirements, however, for our walkthrough we will keep the default values.
10. In the AWS Secrets Manager console, search for the secret you just created and note down the Secret ARN.

Amazon Connect instance details

1. Navigate to the Amazon Connect console.
2. Choose the Amazon Connect instance and note down the Instance ARN.
3. Log in to that instance’s Amazon Connect admin console.
4. Choose contact flows.
5. Choose the contact flow for Facebook messenger channel to start the chat contact. Note the Contact Flow ID.

Install AWS CDK and Bootstrap CDK environment (skip if you have CDK installed)

npm -g install typescript
npm -g install aws-cdk
cdk bootstrap aws://ACCOUNT_ID/AWS_REGION

Deploy the project

Before deploying, ensure you have the following variables from the previous steps before continuing:

• Amazon Connect Instance ARN
• Amazon Connect Contact flow ID
• AWS Secrets Manager secret ARNs where values for PAGE_TOKEN, APP_SECRET and VERIFY_TOKEN are stored

1. Using Git, clone the repository from GitHub.

git clone git@github.com:amazon-connect/amazon-connect-message-streaming-examples.git

2. In your terminal move into the root of the directory

cd amazon-connect-message-streaming-examples

3. Install the dependencies for the CDK project and AWS Lambda functions.

npm install
cd src/lambda/inboundMessageHandler
npm install
cd ../../..
cd src/lambda/outboundMessageHandler
npm install
cd ../../..
cd src/lambda/digitalChannelHealthCheck
npm install
cd ../../..

4. Deploy the CDK project using your AWS CLI profile. Pass in the context environment variables amazonConnectArn, contactFlowId and fbSecretArn, which are required for the CDK stack. Provide a list of Amazon Comprehend PII entity types in the piiRedactionTypes parameter, for example as “PIN, CREDIT_DEBIT_NUMBER”.
   1. For a full list of PII Redaction entity types, visit Detecting PII entities. Having piiRedactionTypes set enables the filtering of PII information entered by the customer.
   2. SMS and Facebook channels are part of the same CDK project. If you would like to deploy the SMS channel, you need to supply the pinpointAppId and smsNumber (the phone number). For more details, refer to the SMS blog.

cdk deploy \
--context fbSecretArn=<YOUR SECRET ARN> \
--context amazonConnectArn=<YOUR AMAZON CONNECT INSTANCE ARN> \
--context contactFlowId=<YOUR CONTACT FLOW ID> \
--context piiRedactionTypes="<CSV LIST OF AMAZON COMPREHEND PII ENTITY TYPES, EX: PIN, CREDIT_DEBIT_NUMBER>"

5. Once the CDK deployment has finished, note down the FacebookApiGatewayWebhook from the CDK output.

Note: This is a sample project designed to be easily deployable for experimentation. The IAM policy permissions use least privilege, however, the Amazon API Gateway deployed will be publicly accessible. Take the appropriate measures to secure your API Gateway by following the public documentation.

Facebook console

1. In the CDK output in the terminal, find the API Gateway invoke URL.
2. Navigate back to Facebook Developer console.
3. In the App you created in the first step, go to Messenger, Settings.
4. Under Webhook, add Callback URL
   1. Callback URL is your API Gateway invoke URL which was generated by CDK in step 5 of deploy the project section above.
   2. Verify token is the random string you created in the previous Secrets Manager section.
   3. Verify and save.
5. Choose Add subscription

6. Select messages

7. Choose Save

Congrats! You’ve successfully added Facebook Messenger as a digital channel to your Amazon Connect instance with PII redaction. To test this solution, send a message to your Facebook business page and you’ll be connected to your Amazon Connect instance!

PII Redaction User Experience

These two screenshots show a sample conversation between a customer and agent, where the customer is sharing their credit card number and PIN code with an agent. Both values are redacted by the solution and are replaced with placeholders.

Customer Experience

Agent Experience

Cleaning up

1. Delete Facebook business page.
2. Delete Facebook developer account.
3. Navigate to the AWS Secrets Manager Console and delete the secret.
4. Destroy the CDK stack

cdk destroy \
--context fbSecretArn=<YOUR SECRET ARN> \
--context amazonConnectArn=<YOUR AMAZON CONNECT INSTANCE ARN> \
--context contactFlowId=<YOUR CONTACT FLOW ID> \
--context piiRedactionTypes="<CSV LIST OF AMAZON COMPREHEND PII ENTITY TYPES, EX: PIN, CREDIT_DEBIT_NUMBER>"

Conclusion

In this blog post, we demonstrated an example of how to redact sensitive data to improve customer experience by protecting privacy of the customers, protecting agents from having to handle sensitive data, minimizing the risk of PII compliance breaches using Amazon Connect chat message streaming API. To get started, visit our GitHub repository and deploy the project!

Author Bio

	Abhishek Pandey is a Senior Solutions Architect with Amazon Web Services. With over 16 years of enterprise IT experience, Abhishek is passionate about diving deep with customers to architect creative solutions that support business innovation across different industries. Abhishek inspires builders to unlock the value of the AWS Cloud, using his secret blend of passion, enthusiasm, customer advocacy, curiosity, and creativity.
	Attila is an Amazon Connect Consultant with Amazon Web Services Professional Services group. Besides contact center experience, he has also had software development and enterprise networking background. Attila is always looking at innovative ways of enhancing product capabilities to deliver customer benefits.
	Jag Jhutty is a Senior Amazon Connect Consultant with Amazon Web Services Professional Services group. He has over 15 years of experience with various contact center technologies and infrastructure more generally, delivering solutions across different industries.
	Ernest Ozdoba is an Amazon Connect Consultant at Amazon Web Services Professional Services. He has been working with multiplicity of organizations of various size and industries, for over 16 years, creating customer and agent experiences. Ernest is dedicated to details when building solutions, making sure to meet all customer expectations and follow best practices.
	Michiel Buurman is an Amazon Connect Consultant with Amazon Web Services Professional Services. With more than 14 years of contact center experience, Michiel has helped many organizations deliver exceptional customer service to their clients.