BUILD, DELIVER, MANAGE BLOG SERIES: BUILD

Introduction

Amazon Web Services (AWS) enterprise customers are using the cloud to build immersive spatial experiences. As an AWS Spatial Prototyping Lead, my team works backwards with these customers to identify spatial challenges (enterprise augmented reality, virtual reality, metaverse, and 3D technology) and resolve those challenges using the cloud. I work with a variety of teams that are developing strategies or developing enterprise immersive technologies. Spatial, like metaverse, has definitions that are broad and varying, and as a team we understand that all organizations have their own definitions. When building these spatial experiences there are two common challenges my team and our customers run into: communicating what spatial infrastructure is to a technical audience and building applications that scale.

To solve these challenges, I needed to start by defining spatial experiences to my team. From working on a number of customer engagements I pieced together an enterprise focused definition that we use: “A spatial experience imports, changes, analyzes, simulates, and distributes data for visual user experiences, to augment a human being.” Based on this definition, my team is better able to communicate what we are building, and then build it to scale. This blog post discusses how I learned through experience to resolve the challenges of communicating these concepts to a technical audience and then building technologies to scale.

Challenge 1: Communicating to your audience

When communicating my team’s spatial definition to executives, infrastructure architects, or developers, I realized I needed to meet the customer where their current understanding is. I try to communicate in today’s infrastructure terms, if they are familiar with that.

To do this I began by taking a much broader look at our team’s definition. I noticed that the spatial architectures we build for customers do not differ drastically from existing (standard, or typical) infrastructure workloads at AWS.  Whether that is building AI models, constructing a social network, moderating game content, or cloud rendering visual effects.  The common denominator between a spatial experience and these technologies is that they both ingest data, compute data outputs, and distribute output data.  Because of these similarities, I have started referring to the base infrastructure as a “spatial data plane”.

Example Spatial Data Plane

In order to meet our customers in their domain, the team started referring to backend spatial infrastructure as spatial data planes. For example, I now say that, “I am building multiple authenticated spatial data planes and interpreting their outputs with a visualization application.” This perspective is earning trust and increasing credibility in knowing we are working with existing proven technologies, while also achieving the goal of building a future-looking application.

Now, as you strategize or build your immersive experiences, focus on looking at your audience.  Do not alienate them with terminology, but meet them in their existing knowledge plane.

Challenge 2: Building to scale

When I build spatial data planes and visualizations, I separate the infrastructure from the human interface. The two primary infrastructure challenges I see today are interoperability, and scalable application development.

Interoperability

Building spatial experiences presents many challenges, such as: aligning on tools, interoperability, and unified interaction design. However, the largest barrier to adoption is quality of the developer experience. Focusing on a single set of software development tools limits the application reach, whether the team is using a monitor, mobile device, or augmented and virtual reality headset. My experience has taught me that we should build independently from devices, due to the rapidly evolving device landscape, which means we need to standardize on interaction and content from the beginning of development.

When standardizing on interaction, I see real-time rendering engines and browser-based augmented and virtual reality taking great strides. However, in order for increased adoption of interaction tools there also must be a focus on developer experience. In order to facilitate a consistent experience I borrow from Mark Zuckerberg’s motto, in this article from CNET, “Move fast with stable infrastructure.” My team primarily focuses on easing the developer experience, with real-time engine plugins and the AWS Cloud Development Kit (AWS CDK). These tools allow the team to iterate rapidly on infrastructure and user interfaces during prototypes. For example, to ease the development pipeline, the solution has been to prototype data planes that are available to all rendering engines via a single data access point, similar to the following dynamic API calls over HTTPS below. For more information on the graphic below, see Build a Serverless Web Application.

A Sample Serverless Web Application

Scalable Application Development

The team has used the architecture above in multiple prototypes to build highly scalable augmented and virtual reality applications. I combine that architecture and open-source standards, such as glTF files (a Khronos Group open standard) which gives customers the ability to download and instantiate assets in applications at runtime. Furthermore, I have even used the Amazon API Gateway with AWS Lambda to debug my virtual reality applications by logging errors. This was done by submitting a POST request to API Gateway then Lambda parsed the POST data and printed to a log in Amazon CloudWatch. Now you can utilize the power of Amazon CloudWatch to check application status. The described infrastructure architecture does not require managing your own servers, while also providing a standard data plane for both debugging and data retrieval.

Using open standards and a single data plane now gives the developer the ability to expand a single bespoke application to serve many use cases. A single application can now be a collaboration app, while also being a visualization tool for mechanical computer aided drafts. This can be successfully executed through the standardization of runtime application scripts, parsing metadata retrieved at runtime with cloud infrastructure, and instantiation of assets with the scripts at runtime.  We prototyped the backend infrastructure to support this methodology for a system called the Visual Asset Management System (VAMS). We have open sourced the VAMS code in our GitHub repository. VAMS can not only store the assets, but also run your own containerized code against the assets themselves to produce new 3D objects or code metadata.

As you build spatial applications, first look at what you are trying to achieve, look for common development denominators, plan for scale, and then move forward. Otherwise, trying to fix these challenges at the end will be more problematic. For example, while building a virtual reality training application, take into account that not only will you need to deliver tens or hundreds of trainings, but the customer needs to be updated, which means they need a way to build the trainings that the expert trainers can utilize.

Conclusion

When building spatial data planes you must focus on communicating with the customer and easing their current pain points while foreseeing future obstacles and building to scale. Removing communication barriers will facilitate better communication and a stronger partnership in the long run. In using open standards and interoperability you reduce future risks in tool changes, while also prepping your customer for future technologies that would accept those standards. These two areas will lead to further adoption of immersive technologies, while enabling rapid innovation.

A spatial data plane is the first step in your spatial journey. And so, you may ask yourself how do I get started? You should start with VAMS.  Upload your 3D models and build automated pipelines for your artistic, machine learning, or other 3D workflows. Then you can integrate with browser-based rendering tools such as Three.js or Babylon.js to get started quickly to reach your destination.

Lastly, the most successful developers and entrepreneurs encounter pressure to deliver results. However, successful developers focus that pressure on fixing real problems and taking chances to innovate. Developers and businesses should not discourage innovation and instead encourage ideas that create efficiencies in contributing to the open-source community. Immersive experiences will only succeed through usable, inclusive environments where anyone can contribute and innovate.