Build for Any VR Device with Lumberyard Beta 1.3

The gaming industry is always evolving, and one of the most exciting new technologies to capture the imagination of our customers is VR. Lumberyard is already a powerful graphics engine, which makes it ideal for VR content creation. Lumberyard Beta 1.3, which we’re releasing in a few weeks, includes support for VR game development and new VR devices. Our VR implementation uses Lumberyard’s Gems, which are self-contained packages of assets and features that you can drop into your game. Lumberyard Beta 1.3 includes Gems for both the Oculus Rift and the HTC Vive. Moreover, these Gems serve as templates for customers to build their own Gems and support any VR devices they want.

Gems make adding support for new VR devices easy, with no engine code editing required. Developers won’t have to wait for an official patch or a new plug-in to start using commercial devices (or prototype devkits) as they hit the market. In this post, I’ll take a look at how Lumberyard supports new devices and speeds up VR development with the new VR Preview function.

We’ve been working on VR support for a while, and gave our first preview at this year’s Game Developer’s Conference (GDC). In fact, our approach for VR device support crystalized at the conference. Our VR demo at the show included elevator rides, deep space, and an attack by giant robot squids. It was also integrated with Twitch using ChatPlay, allowing the audience, both at GDC and online, to cast a vote to determine whether the player lived or died.

Our GDC demo was built by Climax Studios using Lumberyard.

Once the demo was over and the adrenaline wore off, people asked the same question over and over: “What VR devices does Lumberyard support?” Rift and HTC Vive support were top requests (our demo was presented on the Rift), but many developers were just as interested in other devices, like the Samsung GearVR, PSVR, and OSVR. Rift support in our demo had been written into the engine code itself. However, with customers asking to use so many different VR devices—and with new devices being announced all the time—it would be cumbersome and code-intensive for an engine to have hard-wired support for all of them. We knew that we had to find a way for customers to add support to any VR device they liked. We found our solution in Lumberyard’s Gems system.

How’s it Work?

Utilizing Gems, small chunks of code can be created that interact with the engine but don’t require editing the engine code itself. This means that developers can add support for any VR device without having to delve into the engine source. As long as a new VR device conforms to the public interfaces that Lumberyard has defined, the engine will automatically use it. Developers can create their own integrations for additional devices without having to wait for an official Lumberyard update, as they would in other engines. With so many new VR devices coming out soon, we wanted to provide a way for customers to make their own support decisions. Additionally, developers can easily override existing device support to add any experimental features that may be important for their gameplay. Below is a high-level diagram of the way this works inside the engine.

The HMDManager contains an IHMDDevice, which is then implemented by a device-specific Gem. The manager takes care of device initialization and device-abstracted head-mounted display (HMDs) interaction with the rest of the system. On the rendering side, Lumberyard’s stereo renderer makes use of the D3DHMDRender object, which takes care of creating graphics-API-specific render targets, social screen rendering, and frame submission to the VR device. To add support for any new VR devices, you simply wrap the vendor-specific SDK in a Gem as defined by IHMDDevice. That’s it! There’s no need to edit Lumberyard’s underlying HMD code, which is represented by the Lumberyard Engine section of the diagram.

On engine startup, the selected HMDs are scanned for connectivity and selected for use. If you want to support both the Rift and the Vive, for example, simply go into the Project Configurator, enable both Gems, and the engine will pick which one to use at runtime based on which device is plugged in.

Device-Specific Functionality

A good example of device-specific functionality is in the differences between the Rift and the Vive controllers. To handle VR device variations, Gems allow the user to supply custom Flow Graph nodes to the engine, meaning that each device-specific Gem can create its own nodes that appear when a specific Gem is enabled. These nodes can expose device-specific functionality for the end user, such as different controller schemes, tracking information, and room states. Because VR is so new, a lot of experimentation is taking place.  VR devices are often quite different from one another. It’s likely that, over time, devices will all start to share popular features that used to belong to only one device. When that happens, features that used to be device-specific will then be put into Lumberyard’s device abstraction and shared across all devices.

Enabling VR

Any Lumberyard project can easily add VR support by enabling one or more of the supported Gems when configuring the game project. However, even though a Gem is enabled, that doesn’t mean that any connected HMD will actually get used until the proper cvar (output_to_hmd) is set. The reasoning behind this choice is so that games can ship with VR support enabled in the game’s executable, but support will only turn on if the end user actually wants to use it. The current state of VR rendering can be queried in Flow Graph, so game designers can make different choices based on if the game has entered VR mode or not.

Developing in VR

Game developers need to be able to see what they’re doing in the editor at all times. Without a way to see VR in the editor, developers would have to export a level, load it into the launcher, enable VR, and take a look around. This is obviously inefficient. The Lumberyard Beta 1.3 editor will have full VR Preview support built in. VR Preview utilizes the same Gems system as the engine runtime, and it works in a similar fashion. We’ve added the “VR Preview” button to the editor, which you can click to see in VR right away. This allows developers to make VR-specific adjustments to their level designs right in the editor, which reduces iteration time. Flow Graph nodes are an important part of developing in Lumberyard, but they can only be debugged in the editor. With VR Preview, users can debug their VR Flow Graph nodes and see what they’re doing.

To use VR preview support, just click the new “VR Preview” button.

Performance

Enabling these Gems has zero (yes, literally) performance impact on a finished game. Performance will only be impacted when the VR code is actually running. Developers don’t have to worry about having this feature always on, so a game can ship with support for multiple VR devices without sacrificing performance.

What’s Next?

We’re just getting started with VR in Lumberyard, and will be continuing to reduce iteration times, improve rendering performance, and enable new features to take VR to the next level. When Lumberyard Beta 1.3 arrives in a few weeks, try it out and tell us what you think!

Cody White is a graphics and VR engineer who has a background in both feature film and real-time rendering. He’s responsible for Lumberyard’s VR system and spends most days in alternate realities.