AWS Media Blog

Building a Virtual Assistant using AWS – Part 1

In this two-part series, you will learn how to build a 3D scene using Amazon Sumerian. In part 1, you will learn to set up key AWS services like Amazon Lex and Amazon Cognito Pool ID, and how to set up an Amazon Sumerian Host. In part 2, you will focus on creating a custom script for Amazon Lex response and Google Maps, and integrating Google Maps in an Amazon Sumerian scene.

By the end of this two-part series, you will have your own virtual assistant ready to help you find nearby shops, restaurants, malls, or any other local establishment.

AWS Services used in this blog post

  • Amazon Sumerian
  • Amazon Lex
  • Amazon Cognito Pool ID
  • Amazon CloudFormation

Recommended Prerequisites

Before you begin, familiarize yourself with following:

Create an Amazon Cognito Identity Pool ID

Before we get to editing a scene in Amazon Sumerian, we need to set up an AWS CloudFormation stack.

For more information about Amazon Cognito, see the AWS CloudFormation stack template to set up your Amazon Cognito identity pool ID.

  1. Sign in to Amazon Sumerian with your AWS account and open the AWS CloudFormation stack template.
  2. Be sure you are using a region that supports Amazon Lex. For more information on availability, see the AWS Region Table.
  3. Once you have the creation page pulled up, update the Stack name and select the acknowledgement check box at the bottom of the page. Then choose Create.                                                            
  4. Once the stack loads and reads “CREATE_COMPLETE”, expand the Output section and copy the CognitoIdentityPoolID value. You will enter this value in Amazon Sumerian.                               

Create an Amazon Lex Bot

This scene uses Amazon Lex to parse user requests. We want our Amazon Sumerian Host to display local businesses like shops, malls, restaurants and movie theaters on Google Maps. To achieve this, we will connect Amazon Lex and Amazon Sumerian. Because of Amazon Lex’s natural language understanding capabilities, Amazon Sumerian Host can understand utterances like “show me {localBusinesses}”, “where is {localBusinesses}” and “where are {localBusinesses}”.

For this tutorial, we first need to create an Amazon Lex bot.

  1. Download the following Amazon Lex bot example zip file.
  2. Under Actions, navigate to the Amazon Lex console and select Import.                                                                                                                                                                                                                                      
  3. Upload the file you just downloaded.                                                                                                                                                                                                                                                     
  4. Once the Import is complete, Build and Publish your MapConciergeBot Amazon Lex Bot.                                                                                                                                                                                            

Create Your Own Virtual Assistant with Amazon Sumerian

Now that we have an Amazon Lex Bot, we need to connect it to a host in our Amazon Sumerian Scene.

Let’s get started on creating your own virtual assistant scene.

Step 1: Create New Scene and Add Cristine

  1. Create a new scene using the Default Lighting template.
  2. Select the top-level entity in the Entities panel and navigate to the Inspector pane. Insert your Cognito Identity Pool ID into the AWS configuration component.                                                      
  3. Choose Import Assets in the top center menu above the canvas. Search for and add Cristine.                                                                                                                                                                                                
  4. Navigate to the Assets panel. Under the Host asset pack, find the Host entity (hexagon icon) and drag the entity into your scene, as shown in the below screenshot.                                                                    

Step 2: Add the Dialog Component

  1. With the Host entity still selected in the Entities panel, add a new component and select the Dialogue component.
  2. Insert the values for Name and Alias. These inputs reference the Amazon Lex bot you are using. In this case, this will be the Amazon Lex bot you imported to your account in step 2. The default values for this step are “MapConciergeBot” and “$LATEST”. These values are case sensitive. Note: Choose Open AWS Console to see your bots and find their names and aliases.                            

Step 3: Create HTML3D backdrop to display Google Maps

In this step, we will place HTML3D entity in our Amazon Sumerian scene and configure Html and JavaScript code on HTML3D entity to display Google Maps. (Code will be configured later in this tutorial.)

  1. Click Create Entity above the canvas to open the Create Entity Panel.
  2. Choose HTML 3D from the Others section.                                                                                                                                                                                                                                                                                            
  3. With the HTML 3D entity selected, change the transform values as shown below. Note: At the time of writing, the html 3D entity does NOT work in AR nor VR – only on desktop and mobile. It also does not work when Post Effects are applied.                                                                                                                                                                                                                                                                
  4. Click on Tools above the canvas and select Text Editor.                                                                                                                                                                                                                                                                    
  5. Edit HTML3d Entity file. Copy and paste the below html code in the editor.
    <span id='map'
  6. Remember to save the file, as shown in the screenshot below.                                                                                                                                                                                                                                                              


Now you know how to set up Amazon Cognito Pool ID and Amazon Lex in support of an Amazon Sumerian scene including a Host (Cristine) and HTML3D entity. In part 2 of this blog post series, you will learn how to create your virtual assistant using Amazon Sumerian and apply components such as Entities, State Machine, Host Behvaior and Script, as well as how to integrate Google Maps into your Amazon Sumerian scene.

Additional Reading

Stylized AWS activate banner with the words AWS Activate Founders for Self-Funded Startups

Leo Chan

Leo Chan

Leo Chan is a Senior Specialist Solutions Architect working with AR/VR technologies. He loves working at the intersection of Technology and Art and has built tools such as Maya and Houdini for the Film and Video Gaming industries and created block buster content with studios including Pixar and Electronic Arts.

Chirag Oswal

Chirag Oswal

Chirag Oswal is a solutions architect and AR/VR specialist working with the Public Sector India team. He works with AWS customers to help them adoption of cloud operating model at a large scale.

Jacob Smeester

Jacob Smeester

Jake is a Creative Content Specialist on the Amazon Sumerian team, focusing on building valuable educational content, both written and video, for Sumerian customers at all levels of expertise.