AWS for M&E Blog
Cosm scales high-resolution Shared Reality experiences with AWS
Consumer appetite for live entertainment is stronger than ever. From concerts to sporting events, the bar for audience expectations continues to go up, with Shared Reality emerging as an exciting new way to engage with content. Cosm is a leading pioneer in reimagining how fans can experience entertainment, with 8K and higher resolution programming uniquely formatted for its immersive LED dome displays and virtual reality (VR) headsets.
Combining artistic vision and nearly 80 years of technical expertise, the experiential company deploys proprietary software and techniques to create content for its unique Shared Reality venues. These establishments, located in Los Angeles and Dallas, have seen fans enjoy live, immersive broadcasts of sports, including the National Basketball Association (NBA), English Premier League, NCAA College Football, and World Wrestling Entertainment (WWE). Cosm has introduced a new way to experience cinema with The Matrix, reimagined for Cosm’s immersive dome theater, with Willy Wonka & The Chocolate Factory set to debut in Shared Reality next. To provide high-quality and low-latency live event streaming to its venues and to be able to deliver content to partners at scale, Cosm relies on Amazon Web Services (AWS).
“We produce live events nonstop, not just for our dome distributions,” explained Devin Poolman, Chief Product and Technology Officer at Cosm. “There are times when we’re handling more than 16 live events in a single day, so the ability to access over a thousand cloud instances that can spin up in an automated fashion is critical for scaling the business.”
“We’re living on the bleeding edge with what we’re building at Cosm, which requires complex infrastructure and finding the right partners to help us achieve our ambitious creative vision. AWS helps us blur the lines of digital and physical reality,” added Ryan Cole, Vice President of Video Technology at Cosm.
Image 1: Baseball fans celebrate a victory while watching the game at Cosm.
Elevating visual quality
Video quality has been a focus for Cosm since day one. To deliver truly immersive experiences in a 180-degree format on 12K+ LED domes, which can also be delivered to VR headsets, the Cosm team realized that content must be in 8K resolution or higher. Although Cosm’s production teams capture footage at the highest resolution possible, they often need to enhance and denoise video sources. Then, they must compress the final deliverable and share it with end users. Engineers initially focused on validating point-to-point distribution using the secure reliable transport (SRT) protocol to stream its first experience over the internet to its research and development (R&D) facility in Salt Lake City, Utah. However, this approach limited the company’s ability to scale and meet audience demand.
“Our early tests provided a great validation point, and latency was incredible, but we also recognized that we’d need to build for scale and future-proof to be able to handle a hundred venues,” said Poolman. “Moving from a stream-based workflow to a file-based workflow on AWS has allowed us to transcode in the cloud for 8K and beyond. We’re already working in 10.5K and we’re prepared to move to 12K and 16K live video workflows.”
“I think a lot of people in the industry are moving away from traditional approaches to enable higher resolution and higher frame rate video delivery seamlessly across the globe,” noted Cole. “We’re currently running compressed workflows in AWS at upwards of 200 megabits per second for high resolution video delivery.”
Centralized live cloud production
Today, Cosm takes a remote integration model (REMI) approach to producing its immersive broadcasts. For live events, on-site teams capture content, then feed it into AWS and back to a REMI room at Cosm’s production facility in its Playa Vista, California, headquarters. There, producers, technical directors, and engineers produce a 180-degree version that complements the live broadcast for delivery to its venues’ LED dome displays in seconds using playout with Amazon Simple Storage Service (Amazon S3).
“Due to our ability to scale up, we can have concurrent productions with multiple REMI setups doing multiple games simultaneously, and it’s all taking advantage of this connectivity through AWS on the backend,” shared Poolman.
Cosm venues present these live immersive feeds on their 12K+ LED domes using the company’s own real-time engine, adding real-time graphics above the immersive display through the software engine’s deep integration with Epic Games’ Unreal Engine. Cosm also insert the event broadcast feed on virtual jumbotrons in the venue using their own technology, and in-venue teams control all elements using proprietary CX Pro software, which composites the experience. This means that producers can show key broadcast moments, such as slo-mo replays, on the dome.
AWS foundations
To achieve the scalability and reliability needed to support a growing number of venues, Cosm has created a workflow that comprises custom solutions layered on top of Amazon S3 and Amazon Elastic Compute Cloud (Amazon EC2). Video is captured on site and sent using SRT to AWS Elemental MediaConnect for transcoding and delivery. Cosm’s MediaConnect settings route content through multiple Regions as a failsafe. Dedicated AWS Direct Connect network connections in Cosm venues help accelerate content delivery, and using Amazon S3 Glacier Instant Retrieval means that Cosm can keep content accessible for replays while minimizing storage costs.
“We need to be live one hundred percent of the time at all of our venues, and our delivery through MediaConnect helps to ensure that path,” emphasized Cole. “AWS enables us to distribute high-res streams worldwide and implement different solutions as we adapt and scale. We’re constantly evolving and pushing technical boundaries as we grow, and AWS has been a big partner in that. They have helped us tie a lot of our fundamental operations together.”
Live syncing broadcasts
Cosm strives to deliver a seamless experience across each venue, including the immersive LED display in The Dome, as well as multiple live feeds on the wall-to-wall LED display in the venue’s Hall. As a result, the company is focused on marrying all event video sources in a cohesive manner and keeping the action in sync. The Cosm cloud-based immersive capture workflow is generally much faster than a network television broadcast, so the team might delay its feed up to 20 seconds to match the event broadcaster, using automated time code embedding. This also provides adequate time to incorporate any commercial advertisements when needed due to broadcast rights deals.
“Ultimately, it all comes down to the experience,” Poolman concluded. “We want to make sure our domes provide the best of both worlds. Audiences get to experience the immersive crowd aspect of being in a traditional venue, but with that layer of broadcast commentary.”
Learn more about building live video workflows on AWS or get in touch with an AWS for Media & Entertainment representative.