AWS Contact Center

Ingesting content to power real-time recommendations and search with Amazon Connect Wisdom

Contact center agents often encounter questions from customers that require them to reference knowledge documents such as frequently asked questions (FAQs), wikis, product manuals, and how-to guides. Many contact centers use knowledge management systems and document repositories that are siloed and not well-integrated with the applications that contact center agents use when interacting with customers. As a result, agents spend valuable time manually searching across data sources for information to solve customer issues, leading to frustrating delays for customers.

With Amazon Connect Wisdom, agents can search across a connected repository and receive real-time recommendations from within their agent desktop to find answers to customer issues quickly. Wisdom provides built-in connectors to third-party applications including Salesforce and ServiceNow. Contact center managers can also use the Amazon Connect Wisdom API to ingest data, enabling them to build custom connectors to consume content from other enterprise applications and data sources, such as an Amazon S3 bucket.

You can view the help documentation to set up built-in connectors through the AWS Management Console. If you have content in sources not covered by built-in connectors, this blog will show you how to use Amazon Connect Wisdom APIs to build a custom connector and ingest data from other data stores. As an example of a production level implementation, customers could use the SDK versions of these API calls inside Lambda functions and trigger the Lambda function whenever information in their knowledge sources changes.

Solution Overview

AnyCompany has been using Amazon Connect with 1000+ agents for one year. Their agents regularly spend time looking for knowledge documents stored in the company’s S3 bucket. With Wisdom, agents will get real-time recommendations from the knowledge base and be able to search the knowledge base directly from their agent applications. This solution automatically ingests knowledge from S3 bucket into Wisdom. The rest of this blog post walks you through three key parts of the solution:

  1. Use the AWS CLI to setup Wisdom resources and Connect integrations
  2. Use a Lambda that triggers on any file create/update in an S3 bucket and uses Wisdom APIs through an AWS SDK to ingest the file.
  3. Use the AWS CLI to confirm that you can get query results from Wisdom based on the content you just uploaded

Wisdom’s data structure

The assistant is a top-level resource for Wisdom and is associated with a knowledge base whose contents are used to resolve searches and deliver real-time recommendations. A user can create multiple assistants, and an assistant can be associated with one or more Connect instances; however, a Connect instance can currently have only one assistant. For finer control over contacts to enable Wisdom real-time recommendations, assistants are associated with individual contact flows via contact blocks.

A knowledge base represents a collection of content. At launch, Wisdom will support two types: external and custom. The former is used for built-in integrations like Salesforce and ServiceNow, while the latter is used when customers want to upload content via the public APIs (see below). A knowledge base can be associated with multiple assistants.

Content resources represent individual documents or help articles. At launch, Wisdom will support two formats: HTML and plain text. When an external knowledge base is created, Wisdom will create content automatically when synchronizing with the source. For custom knowledge bases, customers create content manually.

Walkthrough

In this solution, we use an Amazon S3 bucket that contains content, in this case, HTML files. Set up the bucket to trigger Lambda executions on create and remove events. Please refer to this tutorial to set up the S3 bucket and Lambda triggers and remember to enable versioning on the bucket. Here is a reference about S3 versioning should you need it. The Lambda function ingests content into Wisdom.

We will set up the assistant, knowledge base, and various associations via the AWS CLI since they are one-off operations, then continue the rest of the walkthrough with JavaScript snippets that you can add to the aforementioned Lambda function.

Prerequisites

  • An AWS account
  • An Amazon Connect instance
  • An S3 bucket with versioning enabled
  • The latest AWS CLI released September 27, 2021 or later

Setting up Wisdom constructs

The following commands will be executed through the AWS CLI. We will create the Assistant, KnowledgeBase, AssistantAssociation, and IntegrationAssociation using the CLI, then we will move onto the Lambda code.

1. Create assistant

Use the Amazon Connect Wisdom API create-assistant to create a top level assistant resource

aws wisdom create-assistant \
  --name TestAssistant \
  --type AGENT

Running the command will return the assistantId and assistantArn. Please save this value for the following commands.

2. Create knowledge base (custom)

Use the create-knowledge-base CLI command to create a new knowledge base resource of type custom.

aws wisdom create-knowledge-base \
  --name TestKnowledgeBase \
  --knowledge-base-type CUSTOM

This command will return the knowledgeBaseId and the knowledgeBaseArn. Please save this value for the following commands.

3. Associate your knowledge base to your assistant

Use the create-assistant-association CLI command to link the assistant we created earlier with the knowledge base we created in the last step.

aws wisdom create-assistant-association \
  --assistant-id 00000000-0000-0000-0000-000000000000 \
  --association knowledgeBaseId=11111111-1111-1111-1111-111111111111 \
  --association-type KNOWLEDGE_BASE

4. Associate your assistant and knowledge base to a Connect instance

Use Amazon Connect’s create-integration-association CLI command to link both the assistant and knowledge base we created earlier with a Connect instance. This will enable you to add your assistant to Connect contact flows so agents can see automatic recommendations. Use the integrationArn and knowledgeBaseArn from when we created those artifacts.

aws connect create-integration-association \
  --instance-id aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa \
  --integration-type WISDOM_ASSISTANT \
  --integration-arn arn:aws:wisdom:us-east-1:123456789012:assistant/00000000-0000-0000-0000-000000000000
aws connect create-integration-association \
  --instance-id aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa \
  --integration-type WISDOM_KNOWLEDGE_BASE \
  --integration-arn arn:aws:wisdom:us-east-1:123456789012:knowledge-base/11111111-1111-1111-1111-111111111111

Ingesting Data

Now that we have finished setting up Wisdom assistant and knowledge base through the CLI resources, let’s set up the Lambda function function that ingests knowledge from S3 into Wisdom knowledge base. Remember that when you either add or remove an object from the S3 bucket, the JavaScript code will be executed in the Lambda function. You can find the complete code in the Amazon Connect Snippets repository. We will talk through the CreateContent, UpdateContent, and DeleteContent functionalities.

Using Node modules from a Lambda Layer

To execute the HTTP requests for uploading content, we like to use the promise-based HTTP library called axios. As Wisdom APIs are available only in the recent AWS SDK versions, we also need to ensure the Lambda function is using the latest AWS SDK. For those reasons, we will create a Lambda
layer and use it in the Lambda function.

Download and install the latest AWS SDK for Nodejs released on September 27, 2021 or later.

Create a directory for your Layer on your local machine and name it nodejs. Note: the nodejs directory name must be nodejs. Read more on environment paths here.

mkdir nodejs
cd nodejs
npm init
npm install --save axios aws-sdk
cd ..
zip -r lambda-layer.zip nodejs

Now zip up the nodejs directory (lambda-layer.zip) and and create a Lambda layer using the zip file through the Lambda console or the CLI.

Setup the Lambda Function

Next, we setup the Lambda function that ingests content into Wisdom, when you either add or remove an object from the S3 bucket. You can find the complete code in the Amazon Connect Snippets repository. Remember to add the Nodejs Layer to the Lambda function and to ensure that the IAM role assumed by the function has permissions to read the S3 bucket and to upload content to Wisdom.

The next few paragraphs walk you through key parts of the Lambda function code. We run the startContentUpload API in the Lambda function before any createContent or updateContent call. Invoking this API returns a pre-signed url and headers.

1. Start content upload

const uploadDetails = await client.startContentUpload({ 
  "knowledgeBaseId": knowledgeBaseId,
  "contentType": s3Object.contentType
}).promise();

We store the response–which contains a url and header values–in the variable uploadDetails.

2. HTTP Request

await axios.put(uploadDetails.url, s3Object.Body, {
  headers: uploadDetails.headersToInclude
});

Use the uploadDetails to make a PUT request. Make sure to include the required headers or your request will not be accepted.

Now we’ve sent Wisdom the content, we need to let Wisdom know if we’re creating new content or updating existing content. To decide which operation to use, we will query Wisdom for content that shares the same file name.

3. Search for existing content

const searchResponse = await client.searchContent({
  "knowledgeBaseId": knowledgeBaseId,
  "searchExpression": {
    "filters": [{ 
      "field": "name",
      "operator": "equals",
      "value": s3Object.key
    }]
  }
}).promise();

If SearchContent returns an empty contentSummaries array, then run the CreateContent API call. Note that we set the name equal to the S3 object key (represents the file name) which enables us to search for this content in the future. We also set a metadata key s3Version; this allows us to better manage lifecycle, as you’ll see later on.

4. Create content

await client.createContent({
  "knowledgeBaseId": knowledgeBaseId,
  "name": s3Object.key,
  "uploadId": uploadDetails.uploadId,
  "metadata": {
    "s3Version": s3Object.version
  }
}).promise();

If SearchContent instead returns an existing content resource, run the UpdateContent API call. Note that we include a revisionId argument. The UpdateContent operation uses optimistic locking to ensure the content hasn’t been modified since we read it using SearchContent.

5. Update content

await client.updateContent({
  "knowledgeBaseId": knowledgeBaseId,
  "contentId": searchResponse.contentSummaries[0].contentId,
  "revisionId": searchResponse.contentSummaries[0].revisionId,
  "uploadId": uploadDetails.uploadId,
  "metadata": {
    "s3Version": s3Object.version
  }
}).promise();

Test the Lambda Function

You can test the Lambda function either by configuring a Test Event in the Lambda console or uploading a file to the S3 bucket. Now let’s test the previous two functionalities. You can use our sample content file here to upload it to the S3 bucket. On upload, the S3 event will trigger the Lambda function to run and execute the CreateContent logic branch. Use CloudWatch Logs to confirm successful execution or to debug any errors. After we verify that the content got created by checking the CloudWatch logs, lets upload content.html again and it should trigger the UpdateContent logic branch.

Query Wisdom to validate content ingestion

Now that the content has been uploaded to S3, let’s run the following CLI command to validate that the uploaded knowledge is ingested in Wisdom.

aws wisdom query-assistant \
  --assistant-id 00000000-0000-0000-0000-000000000000 \
  --query-text "dog"

This will return query results from the Wisdom engine.

Deleting content

Now that we have ingested content, let’s delete it. Delete the content.html file from the S3 bucket and the event will trigger the Lambda function, calling the DeleteContent API call.

await client.deleteContent({
  "knowledgeBaseId": knowledgeBaseId,
  "contentId": existingContent.contentId
}).promise();

Conclusion

In this post, we walked through how to use Amazon Connect Wisdom to give agents easier access to the information they need to solve customer issues. Specifically, we learned how to use APIs to ingest, manage and query content. With Amazon Connect Wisdom, contact center agents have the information they need to deliver faster customer service thus generating higher customer satisfaction. Get started with using Amazon Connect Wisdom and ingest knowledge content with the Wisdom APIs to provide real-time recommendations and search functionality to your agents today.