AWS Spatial Computing Blog

How to: Create a VR application with user insights using AWS Amplify and WebXR

In this blog you will learn how to create a virtual reality experience with user insights to improve your product using AWS Amplify and Babylon.js’s WebXR implementation. Using this combination of technologies is just one way to help reduce the barrier of getting your first Virtual Reality (VR) application up and running.

This how-to will cover the steps to create a simple VR enabled application that uses AWS Amplify to host the full-stack application, Babylon.js’s WebXR implementation for VR scenes and functionality, and Amazon DynamoDB to store and report on user events. The application will be setup and deployed using Amplify and Bablyon.js, controller input data will be enabled using WebXR, and the user input events will be stored and reported on using DynamoDB, Amazon Athena, and Amazon Quicksight. 3D models used in the application will be stored using Amazon Simple Storage Service (Amazon S3). Amplify will use AWS CodeCommit for continuous deployment.

The end product will be a VR application using a 3D model that will record and report on the user inputs to provide insights on how to improve the user experience.

In the Amplify application, a VR controller selects a blue color from the Color GUI. The event of changing the color sends the color value to the database.

DynamoDB table is populated with the color values for each event.

Quicksight Dashboard is used to show data from DynamoDB.

Architecture

Users connect to the AWS Amplify website with their VR enabled device. Textures and objects are loaded from Amazon S3 into the application. Objects can be managed in Amazon S3 outside of AWS Amplify. Users interact with objects and controls available in the XR scene and each user event, such as a click, color selection, or texture selection, is sent to DynamoDB. Amazon Athena makes the data available for Quicksight to provide user insights. AWS Amplify Administrators can deploy changes based on the user insights to the hosted site.

Prerequisites

Walkthrough

To deploy and run the sample application, perform the following steps:

Step 1: Set Up A React Application
Create a React application using Getting started – React – AWS Amplify Docs and directory structure using create-react-app and test that it starts on a local system.

Step 2: Install AWS Amplify and Babylon.js
Install AWS Amplify CLI, Babylon.js and Babylon.js modules using Getting Set Up | Babylon.js Documentation.
AWS Amplify is a set of purpose-built tools and features that enables frontend web and mobile developers to quickly and easily build full-stack applications on AWS.

Step 3: Initialize Backend with Amplify
For this project we will be using Amplify’s integration with AWS AppSync, Amazon S3, Amazon Cognito, and the Complete guide to full-stack CI/CD workflows with AWS Amplify to build and deploy our application and provision the necessary AWS resources. To do this, we will use the Amplify CLI to initialize and edit our Amplify environment.

Step 4: Create Backend Resources with Amplify
Create and push Amplify backend resources using Storage – Overview – AWS Amplify Docs, authentication using Tutorial – Add authentication – React – AWS Amplify Docs, and Tutorial – Connect API and database to the app – React – AWS Amplify Docs to keep data in sync and stored in DynamoDB.

Step 5: Upload texture to Amazon S3
Go to the AWS Management Console and view the resources that were created. Navigate to the Amazon S3 console and look for the bucket that you created in Step 4. In the top directory, make a folder called ‘public’ then enter the folder. Inside of this folder can click ‘Upload’ and upload our texture into the bucket. This is now accessible in the code.

Step 6: Create the Application Scene
Babylon.js is a popular 3D engine based on JavaScript, an open-source framework, which is used to develop 3D applications and games for the web. It offers advance features and
functionality, which makes it easy to create realistic movements and interactions between objects, which is essential for creating a VR application.

Use the Starter HTML Template from the Babylon.js documentation to create the index.html file. On your local system, copy the code and paste it into src/index.js, src/App.js, and src/Scene.js in the README.md.

Step 7: Create CodeCommit for git-based CI/CD deployments
To host the application, utilize the AWS Amplify Hosting service along with AWS CodeCommit. Amplify Hosting is a fully managed CI/CD and hosting service and AWS CodeCommit is fully managed source control service, which will allow git-based CI/CD deployments using the document Setting up Amplify access to GitHub repositories.

Step 8: Connect Amplify to CodeCommit for continuous deployment
From the AWS Management Console, open the Amplify console to connect the CodeCommit repository to enable continuous deployment.

Step 9: Create Amazon Athena and DyanamoDB Connector
The Amazon Athena DynamoDB connector enables Amazon Athena to communicate with DynamoDB so that you can query your tables with SQL. This will allow the use of QuickSight to visualize the event data ingested from the user actions in the Scene.

Step 10: Create a QuickSight Dashboard
Create a QuickSight Dashboard to gain insights from event data stored in DynamoDB. Amazon QuickSight is a cloud-scale business intelligence (BI) service that you can use to deliver easy-to- understand insights.

Step 11: View and use the VR Application
Each time a new CodeCommit push command is performed, Amplify will run a new build and deploy process. Once the deploy process has completed, the latest version will remain at the same Production branch URL located in the app hosting environment section

Step 12: View Quicksight Dashboard
The QuickSight shows 172 VR controller clicks and the RGB value of the colors of each click.


Step 13: Restrict Public Access or Clean up
To keep control of costs, either use Restricting access to branches instructions to password protect the Amplify URL or delete all of the created resources.

Conclusion

This blog showing how to use Amplify, Babylon.js, and Babylon.js’ WebXR stack provides just one of many ways to get started with WebXR. Using Amplify will provide a continuous deployment environment to allow for iterating over your XR ideas. To get inspiration of what to try next with Bablyon.js, visit the The Playground. This blog example can be expanded even more with gathering more event data with additional objects, textures, and controls as well as creating a more comprehensive dashboard, or adding AI/ML to expand on the user insights.

Next Step

The full step-by-step walkthrough, including the specific technical commands used to replicate the setup outlined in this blog, can be found at Create a VR application with user insights using aws amplify and the webxr stack.