CaptionHub Live on AWS expands the reach of live video with subtitling

Live event subtitling is a challenging task, especially when it involves translating into multiple languages simultaneously. CaptionHub Live provides a high degree of accuracy while supporting near real-time latency. Subtitles are accurately synchronized to the audio without affecting the broadcast stream.

Traditional solutions are often expensive and complicated, deliver sub-optimal results, and affect the broadcast critical path.

Enter CaptionHub Live, ground-breaking technology that simplifies live event subtitling. It uses advanced technology that synchronises, orchestrates and distributes a number of complex processes to deliver precisely timed subtitles in up to 128 languages. Amazon Web Services (AWS) are used to provide precise translation and global distribution at scale.

Traditional subtitling workflows

Traditional live subtitling involves complex workflows and significant costs. Subtitle creation often requires manual transcription and translation, which are then synchronised with the live video stream. This process can introduce noticeable delays and often limits the number of languages supported due to high cost and logistical constraints.

Existing live localization efforts are both expensive and resource-intensive. Technologies often fall short due to several issues:

• High costs make live subtitling financially prohibitive, limiting the number of streams that can be captioned.
• Traditional solutions suffer from latency issues that impair the viewing experience for those who rely on them.
• Many systems intercept the video stream to insert captions, then rebroadcast the stream, introducing a critical point of failure.
• Early AI technologies failed to deliver the necessary accuracy, particularly for specialised terminology.
• Poor formatting and inconsistent caption styles negatively impact the viewer experience.

The solution

CaptionHub Live, powered by AWS, addresses the core challenges head-on. It offers a cost-effective, scalable, and reliable solution for live video subtitling, supporting multiple languages with precise synchronisation and advanced functionality.

The CaptionHub Live system subscribes to an HTTP Live Streaming (HLS) broadcast by the media provider. The system extracts the audio from each segment, and transcribes it using Speechmatics, which runs as containers on Amazon Elastic Compute Cloud (Amazon EC2). These containers use modern AI models running on GPUs for optimal performance.

When the system receives the timed transcript back from the transcription container, it is processed using CaptionHub’s Natural Captions engine to split it into human-friendly captions. The captions are then sent to Amazon Translate, after which CaptionHub Live reassembles the translated captions into additional text tracks. These captions are served as JSON files from the CaptionHub Live application, using Amazon CloudFront as a content delivery network (CDN).

When a user views the stream, a JavaScript plugin, located in the web page that is hosting the stream, requests the caption files and injects the timed text captions into the player. This is accomplished within the typical delay found in HLS players, due to the buffering of three video segments before play starts. CaptionHub Live can complete all the required processing and delivery of the captions, exactly in sync, without adding any additional latency.

Figure 1: CaptionHub Live architecture.

Key features of CaptionHub Live include:

• Cost-efficiency: AI-driven solutions drastically reduce the costs associated with live subtitling, making it feasible to caption more events.
• Full synchronisation: Patent-pending technology ensures captions are perfectly timed with the live video, eliminating synchronisation issues.
• Multilingual support: Native support for multiple caption tracks expands the reach of live streams to diverse audiences.
• Advanced functionality: Features like term bases, custom dictionaries, profanity filters, and formal/informal settings enhance accuracy and viewer experience.
• Uninterrupted broadcast stream: Captions are injected directly into the video player, leaving the broadcast stream untouched and reducing points of failure.

Decoupling from the broadcast

A critical feature of CaptionHub Live is its ability to decouple captions from the broadcast stream. In other words, unlike traditional solutions that intercept and then embed captions back into the transport stream, CaptionHub Live injects captions directly into the video player through the video player plugin architecture for players including Brightcove, Video.js, and THEOplayer. This approach maintains the integrity of the original broadcast stream and removes the risk of disruptions by CaptionHub Live.

Key design considerations

CaptionHub Live is designed with several key considerations in mind to ensure optimal performance. Latency is minimised through CaptionHub’s Multi-clock Synchronisation System, ensuring perfectly timed captions with no noticeable delay. Reliability is a cornerstone of the platform, with robust performance guaranteed by carefully built infrastructure, ensuring continuous service even during high-traffic events. Additionally, CaptionHub Live offers remarkable flexibility by supporting various broadcast formats and integrating seamlessly with existing workflows. It is an ideal solution for broadcasters looking to enhance their live streams without the need for significant system overhauls.

Real-world applications of CaptionHub Live

CaptionHub Live has rapidly gained traction since its introduction in Fall 2023, finding immediate application at prominent tech events. For instance, it seamlessly integrated into a 30,000-attendee consumer launch and a live tech company marquee event that garnered 228,000 views. What’s noteworthy is its potential for future use in various contexts. Companies are already considering its incorporation into upcoming events, indicating its versatility and promise in enhancing accessibility and inclusivity in content delivery across different industries.

CaptionHub Live powered AWS LIVE News desk at NAB 2024

CaptionHub Live partnered with AWS as the live subtitling solution for the NAB Show LIVE News Desk during the NAB Show 2024 in Las Vegas in April 2024.

At this prestigious event, attended by over 61,000+ industry professionals, CaptionHub Live, powered by AWS, took centre stage. The NAB Show LIVE News Desk (programmed jointly by NAB and AWS—produced in partnership with Broadcast Beat Studios featured a dynamic program. The program was comprised of live segments from the AWS News Desk and immersive reports from the Show Floor with exhibitors and attendees.

Following the success of the NAB Show 2024, AWS re-engaged CaptionHub Live for AWS re:Invent in December 2024. Partnering with live stream service provider Corrivium, CaptionHub developed a seamless integration. This integration was led by Corrivium and in collaboration with Dolby and THEO Technologies, to power live captions for one of the most prestigious tech industry events. Using CaptionHub Live, AWS re:Invent was streamed in 15 languages, reaching 51 countries with over 99 percent accuracy. This collaboration set a new benchmark for high-stake events, demonstrating the power of scalable, live accessibility solutions.

Throughout these events, CaptionHub Live seamlessly provided live captioning support, facilitating accessibility and inclusivity for all attendees, both in person and virtually. CaptionHub Live’s contribution to the NAB Show LIVE News Desk and AWS re:Invent exemplifies its versatility and reliability in demanding real-world broadcasting environments. By offering precise synchronisation, multilingual support, and uninterrupted broadcast streams, CaptionHub Live facilitated a seamless viewing experience for audiences worldwide. This underscores its potential to revolutionise accessibility and inclusivity in content delivery across diverse industries.

Summary

Experience the future of live video subtitling with CaptionHub Live, powered by AWS. Enjoy unparalleled cost-efficiency, synchronisation, multilingual support, and reliability. Transform your live events with perfectly timed, accurate captions that expand your reach and enhance viewer engagement.

To learn more or request a demo. Start revolutionising your live broadcasts today with CaptionHub Live available on AWS Marketplace.

Check out more AWS Partners or contact an AWS Representative to know how we can help accelerate your business.