How the cloud is advancing virtual production

Virtual production has reshaped movie making in many different ways, providing filmmakers with unique capabilities ranging from remote collaboration and pre-visualization to final pixel in-camera visual effects (ICVFX). In the last three years, virtual production has catapulted into the mainstream. While technological advances have made the process more accessible, complex virtual production techniques continue to prompt new challenges for creators. Amazon Web Services (AWS) recently caught up with three industry experts to find out how the cloud presents solutions to streamline and evolve virtual production, while accelerating timelines and reducing costs.

Challenges and opportunities

Committed to educating creators and filmmakers on what is possible with virtual production, Ken Nakada, Head of Virtual Production Operations at Amazon Studios, oversaw the creation of innovative LED volume Stage 15 on Amazon Studio’s Culver City lot. This education is paramount, as certain capabilities, like shooting ICVFX, require a huge mental shift. The approach requires more upfront content development and often means that some creative decisions, such as the design of virtual environments, must be made earlier in the process.

“Filmmakers and production crews know their business very well, but when you introduce a new tool, it can become very uncomfortable,” explained Nakada. “It’s our job to help them better understand the tools, the possibilities, and the stage.”

Cloud possibilities

Erik Weaver, Head of Virtual & Adaptive Production at the Entertainment Technology Center (ETC) at the University of Southern California (USC), is more than familiar with the advantages of virtual production. He’s pushed the limits of what is achievable with real-time technology, and showcased the boundless potential for filmmaking in the cloud for the last several years. His credits include short films Ripple Effect and Fathead, both of which experimented with ICVFX. Aside from the actual filming, Fathead was created almost entirely on AWS. ARRI Alexa camera files were sent directly to the cloud at an astonishing 800GB per second, and all essential post operations, including color, were performed in the cloud.

Commenting on the rise of virtual production, Weaver said, “We’re still at the ‘proving it’s possible’ stage and encouraging people to be open-minded and push boundaries. This is the best time to educate oneself. Introspection and education are what we should be focusing on.”

A less glamorous piece of the puzzle

As co-founder of Back to One Salons and a consultant for Tech & Content LLC, Lisa Gerber is focused on advanced networking connectivity for virtual production stages. Since, by its very nature, networking is made to be invisible, delivering on-demand cloud access, internet, and point-to-point connectivity is not often a discussion point. Yet, behind the scenes, networking engineers are implementing the latest advances to eliminate connectively bottlenecks on set.

Gerber noted, “If you have a network-as-a-service, and you have optimized connectivity that can scale and turn on a dime, that’s going to be a huge bonus for production, especially virtual production. A lot of folks don’t necessarily think about that because it’s not the most glamorous aspect, yet more and more studios are recognizing that they can’t take it for granted.”

Moving the needle forward with standardization

Virtual production has been like the Wild West for some time, primarily due to the speed of technological advances and infrastructure. “The infrastructure and networking companies that are invested in seeing virtual production grow are likewise invested in being a part of the standards discussion,” said Gerber. “They want to talk about how to standardize on-premises data centers, power, and cabling, and make sure that it’s all redundant. We’re seeing a lot of companies come in to fortify that process.”

Weaver added, “AWS has brought things to the next professional level for infrastructure components. For standards, there’s SMPTE’s Rapid Industry Solutions (RIS), camera lens metadata, thinking about how stages can bid consistently, and so on. We’re seeing standardization begin to rise across the board.”

“Standardization is necessary for innovation,” emphasized Nakada. “Standardization simply means stabilizing how we move forward and coming to a place that we can agree on so that the work put in today doesn’t disappear tomorrow.”

As part of this effort, SMPTE ST 2110 is becoming a more integral part of the conversation around standardization in virtual production. SMPTE 2110 is a suite of standards developed by the Society of Motion Picture and Television Engineers (SMPTE) that encompasses the packaging and transportation of video, audio, and ancillary data over IP links, maintaining synchronization and quality.

The future of virtual production & the cloud

Before too long, the panelists predict that new processes popularized in virtual production may be able to condense a nine-month production pipeline into nine weeks by intelligently incorporating technologies like neural radiance fields (NeRF), artificial intelligence (AI), and machine learning (ML). However, successful implementation requires the proper infrastructure, connectivity, and education to be in place.

Hear more from Nakada, Weaver, and Gerber from the session at the 2023 AWS for M&E Symposium. For a deeper dive, read the Fathead virtual production white paper.