AWS for M&E Blog

Virtual production on the cloud: Grup Mediapro unleashes creativity with AWS

This blog was coauthored by Emili Planas, CTO and Operations Manager at Grup Mediapro and Isaac Bergadà, Virtual Production Lead Manager at Grup Mediapro.

The Media & Entertainment industry is undergoing a significant shift. Within this landscape, virtual production has gained significant traction. Mediapro Virtual Production embarked on a challenging journey to harness the power of the cloud to unleash new creative possibilities and drastically reduce production times.

The rise of virtual production

One of the trends in the changing media landscape is the growing adoption of virtual production. This is fueled by the rise of streaming platforms, the change of viewing behaviors, the high demand for content in social media and the advent of hyper-realistic 3D environments and generative AI.

Virtual production enables the real-time tracking of camera position and orientation in a 3D space while on a physical film set. This technology allows for the integration of real actors into virtual environments. Typically, these sets consist of LED walls with a ceiling, but the system can also be used with green screens. In both cases, actors can see the virtual environment on screens during the performance, helping them more naturally interact with their surroundings.

One of the primary advantages of virtual production is the immediate visual feedback it provides to directors, supervisors, and actors. They can instantly see the composition of the scene, which is a seamless blend of 3D environments with real objects and actors. This real-time visualization significantly reduces the need for reshoots and extensive post-production work, saving time and resources.

Cinematographers benefit from enhanced control over lighting, adjusting virtual light sources on the fly to perfectly complement the physical set. Sound directors can optimize audio capture by understanding how the virtual environment affects acoustics. Directors gain unprecedented creative freedom by being able to quickly iterate and refine scenes within the same virtual space-time, reworking key aspects as needed.

The results are a more creative, controlled, and efficient approach to filmmaking that the industry is rapidly embracing to meet the growing demand for high-quality content creation.

New challenges on stage

While virtual production offers significant benefits, it also brings new technological challenges. Virtual production requires shooting stages to have an unprecedented level of computing power to create, render and mix the virtual scenario with the real images captured with the cameras on set.

For the team at Grup Mediapro, this raises an important question. Isaac Bergadà, Virtual Production Lead Manager at Grup Mediapro asked, “Is it necessary for thousands of shooting stages around the world to maintain high-computing environments on-site, or can we leverage the AWS Cloud for ‘deferred rendering’?”

At Amazon Web Services (AWS), we believe that by utilizing cloud computing, studios can achieve high-quality final images while saving on costs. The cloud approach allows studios to maintain only the compute capacity necessary for real-time rendering and fast visualization on set, while offloading more intensive processing to the cloud.

First steps in virtual production

At the end of 2023, Mediapro Virtual Production, the unit specialized in virtual production within Grup Mediapro, took an important first step. They produced the first European film made entirely using virtual production techniques in its Barcelona studio location. The movie, Parenostre (2024), generated valuable learnings as part of the film was shot using green screen technology and the rest with rear LED screen technology. Both synchronized with a 3D virtual environment. The film featured both indoor and outdoor settings, but all the shooting was done within a virtual production stage in the center of Barcelona.

One of the first challenges that Grup Mediapro´s technical team encountered was the need to re-render most of the film material, as the quality of the rendered scenes didn’t meet the expected bar. In order to enhance the rendering quality of the green scenes, they needed to improve lighting and add intricate details to reinforce the script intention. The team estimated that, with the available human and technical resources, this would result in losing half a year in post-production. Not wanting to lose so much time a solution was needed.

Grup Mediapro’s solution was to execute a fast transition to the Amazon Web Services (AWS) Cloud. They deployed parallel 3D environment rendering, which mostly use Amazon Elastic Compute Cloud (Amazon EC2) GPU based instances.

Accelerating rendering on AWS

By leveraging AWS services, like Amazon EC2 GPU based instances orchestrated by a computing node management service, Mediapro Virtual Production empowered its creative team with streamlined web access to on-demand rendering environments. This shift enabled them to integrate a global creative team, all within a best-in-class security framework.

This way, creatives were able to open and close work environments according to their creation needs. They were able to multiply their parallel work capabilities without the necessary cloud knowledge to deploy, operate, and maintain software-defined infrastructures.

Geographically distributed teams could now work on the same project with parallelized computing on the cloud to accelerate the results of the creative effort. This enabled them to work with more computing power than they could ever have had on a singular workstation alone.

“The outcome was an 83 percent reduction in the estimated post-production time, implying a decrease in overall costs of the film and a faster time to market,” said Emili Planas, CTO and Operations Manager at Grup Mediapro.

Following is the architecture scheme implemented.

Reference cloud deployment with multi-instance capabilities for remote creatives. The diagram leverages the use of Amazon EC2 instances for compute, Amazon EBS volumes and Amazon S3 for storage.

Figure 1: Reference cloud deployment with multi-instance capabilities for remote creatives.

Parallel rendering of 3D environments is an evolution of what Grup Mediapro have been doing for many services in the media and entertainment sector, as well as other sectors such as healthcare, fashion, and utilities. These industries increasingly need to manage audiovisual content at scale, which is not straightforward.

Real-time synchronization

The next challenge that Grup Mediapro faced was synchronizing the 3D environment with the studio during real-time shooting. Specifically, how do they send camera data to a 3D environment in the cloud and return the 3D environment’s framing to the studio in real-time, with low latency, less than 10 frames, and with high-quality expectations?

To address this, Grup Mediapro set out to equip their dozens of studios, across four continents, with real-time synchronization technology between the 3D environment and the studio. This would allow them to adopt virtual production technology from diverse locations, such as Miami, Barcelona, New York or Buenos Aires.

The first step was deploying an Amazon EC2 instance on AWS with the chosen 3D environment engine. Synchronizing the studio camera with the virtual camera was straightforward, because the synchronization data flow is very light, less than 5 Mbps. Once Grup Mediapro had this synchronization, the main challenge was to return the video stream generated from the 3D engine’s virtual camera to the studio. Although it seems simple, it involves coordinating multiple talents including artists controlling the creative quality aspects, 3D engine operators, cloud architects, video encoding experts, and local connectivity managers in the studios.

Initial tests showed a promising low latency that was approximately 200 ms between the studio and the cloud environment, even though the computing center and production were located thousands of kilometers apart. However, the quality did not meet the demanding standards required for their non-standard large LED screens. These screens changed proportion needs for every project, particularly in moving objects such as snow and tree branches where polygons appeared due to the usage of standard streaming protocols, such as NDI.

It is important to remember that there are many types of LED walls. Grup Mediapro’s virtual production quality requirements involve projecting on a LED screen of over two hundred square meters and forty million pixels. These dimensions require managing video with multiple times the 4K-HDR quality achieved by generating several 4K environments in parallel.

In subsequent tests, Grup Mediapro dedicated a squad of cloud architects and media streaming experts for two, one-week sprints to try various video encoders and media services on AWS. These ranged from AWS Elemental MediaLive combined with AWS Elemental MediaConnect using AWS Cloud Digital Interface (AWS CDI), to AWS Elemental MediaConnect JPEG-XS flows on-site, all with an agile mindset. However, the different configurations affected the project’s constraints of high quality and low latency.

Finally, after much fine-tuning of key parts of the audiovisual flow in the cloud, their team of video encoding experts found a straightforward and effective way to capture the necessary video quality directly from the cloud GPUs thanks to the internal development of a GPU streaming framework. This technical solution allowed Grup Mediapro to eliminate many processing steps, drastically reducing latency, and manage the quality and formats to operate non-standard configurations while delivering a customized streams to the studios’ LED Screens.

To facilitate video transmission to the studio, the team established a 1 Gbps AWS Direct Connect connection to the cloud. Direct Connect is also prepared to scale up to 10 Gbps to accommodate any increased demands from the creative stream.

Currently, studios located in more than 50 Grup Mediapro locations, have the capability to include virtual production in their live broadcasting TV Shows, news and sports allowing creativity to have no limits.

The following scheme was implemented.

The diagram shows the reference deployment to connect a tracked camera in the studio with 3D Engine Server in the cloud to live-showcase the virtual camera on the studio led. It leverages the use of AWS Direct Connect, AWS Elemental Media Connect and Amazon EC2 instances.

Figure 2: Reference deployment to connect a tracked camera in the studio with 3D Engine Server.

A global network of talent

Thanks to deploying all the creativity of 3D engines in the AWS Cloud and synchronizing them in real-time with any studio on the planet, Mediapro Virtual Production has uncovered an additional benefit in this new way of working: the Global Creative Office.

3D engine creatives are a scarce resource, as it is a relatively recent discipline in high demand across industries like video games and advertising. These talented individuals often choose which projects to work on based on their passions and interests.

By leveraging a decentralized computing environment in the cloud, Grup Mediapro has been able to reach the best talent for their projects. They are able to bring together geographically distributed profiles, not just in major global metropolises, but also in rural areas where many creatives have chosen to live.

The evolution of the industry

At the beginning of the 21st century, audiovisual production centers were among the places with the highest computing concentration due to the demanding nature of video processing, storage, and transport. However, the landscape has shifted dramatically.

With the advent of new screens, high-capacity connectivity, and large software defined data centers, these production centers are now behaving more like an edge within the distributed computing network provided by the cloud. This shift offloads much infrastructure and instead leverages the innovation of cloud services, including high levels of processing power.

The technological revolution has firmly taken hold in the media and entertainment sector, with the cloud as one of the key enablers for generating new, meaningful, and relevant experiences for audiences.

Conclusion

Grup Mediapro’s journey with AWS exemplifies the transformative potential of cloud technology in the Media & Entertainment industry. By harnessing the power of AWS services, they have shattered creative boundaries, streamlined workflows, and unlocked a global talent pool. As Mediapro Virtual Production continues to gain momentum, AWS stands as a catalyst, empowering their visionaries to push the limits of what’s possible and redefine the art of storytelling.

Contact an AWS Representative to know how we can help accelerate your business.

Further reading

Raúl Mansilla

Raúl Mansilla

Raúl Mansilla drives cloud transformation for Media & Entertainment customers at Amazon Web Services (AWS), leveraging 20+ years of industry expertise to help companies innovate and scale their digital infrastructure. As a Senior Solutions Architect, he empowers M&E organizations to unlock new possibilities through cloud technologies.