Build a video contact center with Amazon Connect and Amazon Chime SDK

Businesses are investing in video-enabled experiences. They’ve been clear they want to embed more video-enabled functionality in the customer support products and services that they already offer to their customers. Businesses are prioritizing video for the processes used to connect their customers to support agents, or route other inbound inquiries. They have a variety of use cases for enabling their customers to initiate real-time video with a pool of available agents, rather than to a direct recipient. The Amazon Chime SDK can be plugged in to most call center applications that incorporate call routing and queuing features to deliver an integrated web-based video call center experience.

In this blog post, you will learn how to add video capabilities to a sample contact center using the pre-built Amazon Connect chat widgets. You can download the Amazon Chime SDK and Amazon Connect Integration demo on GitHub to see how a video enabled cloud contact center can be built with the using Amazon Chime SDK and Amazon Connect.

Note: Setting up and running the Amazon Chime SDK and the Amazon Connect integration demo may incur extra charges in your AWS account.

Amazon Connect is a straight forward to use omnichannel cloud contact center that helps businesses provide superior customer service at a lower cost. Amazon Connect has a single UI across both voice and chat for contact routing, queuing, analytics, and management. This omnichannel experience means that call center agents don’t have to learn and work across multiple tools. Using the Amazon Chime SDK, we can enhance the current support experience provided by Amazon Connect by enabling real-time video between the customer service agent and the customer. Amazon Connect provides a pre-built chat widget to be used on the customer’s end, and a customer service agent chat interface called Contact Control Panel (CCP) on the agent end. Amazon Connect also provides the Amazon Connect Streams API to customize the agent CCP portal.

The Amazon Chime SDK and Amazon Connect Integration demo has a customer application that uses the pre-built customer chat widget provided by Amazon Connect. The demo also has a customized agent CCP application that uses the Streams API to subscribe to the events that provide the agent and customer contact state. The following diagram explains the call flow between different components:

There are a few components at work here:

1.     Customer – The customer arriving at your company website and seeking support.

2.     Agent – The customer service agent who is helping the customer.  In this example, we assume they have the equipment to be video-enabled, such as a webcam and a WebRTC enabled browser.

3.     Company Website – Your customer can access support through your company website or mobile app that embeds the Amazon Connect chat widget. In this example, the widget allows your customer to choose between text, audio, and video support.

4.     Agent Portal – The customer service agent’s UI that helps them assist your customers. Amazon Connect default portal is called the Contact Control Panel (CCP).

5.     Meeting Lifecycle AWS Lambda – An AWS Lambda function responsible for creating the Amazon Chime SDK meeting and adding the agent and end user as attendees.

6.     Amazon Connect Contact Flow – An Amazon Connect Contact Flow that defines how a customer experiences your contact center from start to finish. For example, you can give customers options and route customers to agents from their modality preferences, whether it’s chat, audio, or audio and video. The Contact Flow can also create dynamic, personalized flows that interact with other AWS services such as AWS Lambda.

In this example, the purpose of the Contact Flow is to prompt the customer for their preferred support method: chat or video. If the customer chooses video, they are connected to a video-enabled agent. The Contact Flow is flexible, can be fit to the customer’s preferred support method, and places them in the correct Amazon Connect queue. The agent then receives an inbound call in their portal. The agent can accept this video call to connect with the customer. Next, they both see a “join video” button that can be clicked to join the video meeting.

Now we review how the video call center demo application is set up to provide this experience. The Amazon Connect Contact Flow used in this demo is shown in the following diagram:

When the customer elects to start a video call in step (3), step (4) is enacted in the Amazon Connect Contact Flow. Next, the customer is placed in a queue and the available agents receive a video call in their agent portal with the customer name. The agent can either accept or reject the video call in the agent portal. Once the customer is accepted by an agent, they are removed from the queue, and the Amazon Connect Contact Flow is completed. Upon accepting the customer, the agent-customer connection is established and they can start communicating with each other via chat. Now, the next step is to enable real-time video by creating the Amazon Chime SDK meeting with the agent and customer as attendees.

The video call center demo uses Amazon Connect Streams APIs in the agent application to subscribe to agent and customer events.  In Amazon Connect Streams API, a customer is called a “contact.” In order to know any contact’s state in the agent application, the agent application subscribes to events such as onAccepted, onConnecting, onConnected, and onEnded. Once the contact is accepted, the agent application triggers an Amazon API Gateway POST method call. This call triggers an AWS Lambda function. The function creates an Amazon Chime SDK meeting. The function then adds two attendees, the customer and the agent. Finally, the function returns a response with the Amazon Chime SDK meeting, customer attendee, and agent attendee objects. Using these objects the agent and customer application can start the video call.

The following code is an example of the Amazon Chime SDK meeting creation AWS Lambda function:

function uuid() {
  return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
    var r = Math.random() * 16 | 0, v = c === 'x' ? r : (r & 0x3 | 0x8);
    return v.toString(16);
  });
}

const AWS = require('aws-sdk');
const chime = new AWS.Chime({region: 'us-east-1'});
chime.endpoint = new AWS.Endpoint('https://service.chime.aws.amazon.com/console');
let request = null, agentAttendee = null, customerAttendee = null;

exports.handler = async (event, context, callback) => {
    // create meeting
    let meeting = null;
    const requestId = uuid();
    const region = 'us-east-1'; // specify media placement region
    request = {
      ClientRequestToken: requestId,
      MediaRegion: region,
    };
    meeting = await chime.createMeeting(request).promise();
    
    request = {
      MeetingId: meeting.Meeting.MeetingId,
      ExternalUserId: uuid()
    };
    agentAttendee = await chime.createAttendee(request).promise();
    
    request = {
       MeetingId: meeting.Meeting.MeetingId,
       ExternalUserId: uuid()
    };
    customerAttendee = await chime.createAttendee(request).promise();
    
    let response = {
      statusCode: 500,
      body: JSON.stringify('Internal error occured, check lambda execution logs'),
      meetingId: null,
      agentAttendeeId: null,
      customerAttendeeId: null,
      meeting: null,
      agentAttendee: null,
      customerAttendee: null,
    };
    
    if( meeting && agentAttendee && customerAttendee ) {
      const joinInfo = {
        meeting: meeting,
        agentAttendee: agentAttendee,
        customerAttendee: customerAttendee
      };
      response = {
        statusCode: 200,
        body: JSON.stringify('Meeting created successfully with customer and agent added'),
        meetingId: meeting.Meeting.MeetingId,
        agentAttendeeId: agentAttendee.Attendee.AttendeeId,
        customerAttendeeId: customerAttendee.Attendee.AttendeeId,
        joinInfo: JSON.stringify(joinInfo)
      };
    }
    
    callback(null, response);
};

Once the meeting and attendee objects are received in the agent application, the Amazon Chime SDK for JavaScript creates a MeetingSessionConfiguration object. A “Join video” button is enabled on the agent’s CCP UI for the agent to join the video call. Simultaneously, the agent application passes the meeting object and the customer attendee object to the customers widget using Amazon Connect Streams API’s sendMessage method.

The customer application intercepts the incoming message and gets the meeting and customer attendee objects. Using these objects, it creates the meeting session configuration to join the meeting. Similar to the agent application, a join button is enabled for the customer to join the video call started by the agent. Once the join button is enabled, the customer can click on it to join the video call with the agent. Both the customer and agent are now connected to the same meeting with video and audio.

The customer and agent side of the meeting uses similar code to initialize the MeetingSessionConfiguration and connect to the meeting:

import {
  ConsoleLogger,
  DefaultDeviceController,
  DefaultMeetingSession,
  LogLevel,
  MeetingSessionConfiguration,
} from 'amazon-chime-sdk-js';

async function initializeMeetingSession() {
  const logger = new ConsoleLogger('ChimeMeetingLogs', LogLevel.INFO);
  const deviceController = new DefaultDeviceController(logger);
  // meeting = meeting object received from create meeting AWS Lambda function
  // attendee = attendee object received from create meeting AWS Lambda function
  // At customer side attendee = customerAttendee object
  // At agent side attendee = agentAttendee object
  const configuration = new MeetingSessionConfiguration(meeting, attendee);
  meetingSession = new DefaultMeetingSession(configuration, logger, deviceController);

  // Attach the observer to get access to event updates
  meetingSession.audioVideo.addObserver(observer);

  try {
    // list and choose audio input
    const audioInputs = await meetingSession.audioVideo.listAudioInputDevices();
    await meetingSession.audioVideo.chooseAudioInputDevice(audioInputs[0].deviceId);
    // list and choose video input
    const videoInputs = await meetingSession.audioVideo.listVideoInputDevices();
    await meetingSession.audioVideo.chooseVideoInputDevice(videoInputs[0].deviceId);
  } catch (err) {
    // handle error - unable to acquire audio device perhaps due to permissions blocking
  }

  // bind HTML audio element
  meetingSession.audioVideo.bindAudioElement(domElement('customer-audio'));
  // start meeting session with audio and video
  meetingSession.audioVideo.start();
}

const observer = {
  audioVideoDidStart: () => {
    console.log('Meeting session started');
    meetingSession.audioVideo.startLocalVideoTile();
  },
  videoTileDidUpdate: tileState => {
    const videoElement = tileState.localTile ? 'customer-local-video' : 'agent-remote-video';
    // Bind HTML video element to a video tile
    meetingSession.audioVideo.bindVideoElement(tileState.tileId, domElement(videoElement));
    // Show HTML video element
    domShow(videoElement);
  },
  audioVideoDidStop: sessionStatus => {
    console.log('Meeting session Stopped:', sessionStatus);
  },
};

Demo screenshots:

1.     Customer’s start view with a ‘Speak to an agent’ button.

2.     Once clicked on the ‘Speak to an agent’ button, the customer chat widget pops up asking whether the customer wants to chat or do a video call. The customer replies with ‘video’ and confirms by saying ‘yes’ again. Once confirmed, a call is placed from the customer to the agent.

3.     Below is the agent home view and the view when the agent home screen is receiving the call from customer after above customer confirmation.

4.     Upon accepting the customer call, a ‘Join Video’ button is enabled at the agent side. On clicking the ‘Join Video’ button, the agent video starts.

5.     Agent video starting and waiting for customer to join.

6.     The customer can click the ‘Join video call’ button to join the video call with the agent.

7.     Below are the customer and the agent view with both agent and customer video on, with chat aside.

Customer View

Agent View

8.     Once the agent clicks the ‘End Chat’ button, the meeting ends successfully. Similarly, if customer clicks on close (cross icon) above the customer chat widget, the meeting is ended successfully.

9.     A message pops up on the customer side chat widget showing the meeting ended message if the agent ends the meeting.

Setup steps

We have provided a detailed README on the GitHub repository to help you run the demo.

Cleaning up

To avoid incurring future charges, delete the Amazon Connect instance and the AWS CloudFormation templates created as part of the setup steps.

Conclusion

This example shows a simple integration using the Amazon Chime SDK. It allows a user to request a video call, have that request routed by Amazon Connect to a video-enabled agent, and have both the end user and agent join a video call within their browser.