Unleash creativity at the speed of automation: Streamline story creation using AI and the cloud

This blog was co-authored by Eric Chang, Marketing Advisor, TVU Networks.

Introduction

Today’s audiences want engaging video content across platforms. To meet this demand, broadcasters must quickly turn live and recorded video into compelling stories. AI-powered tools like TVU Search help them efficiently find and utilize content to accelerate story creation.

This blog explores how TVU Search, a cloud-native AI-driven application from TVU Networks, helps broadcasters efficiently find and utilize content to make stories faster for TV, web, and mobile.

Key challenges

Broadcasters in the M&E industry have massive amounts of live and archived video, with more added every minute. While this raw material can create any story, finding the right content is difficult and time-consuming. Manually sifting through hours of video to find specific segments is inefficient. In today’s social media world, stories unfold quickly. Broadcasters must report on them rapidly to capture viewer attention.

What is TVU Search?

TVU Search is an AI-powered video ingest and discovery tool tailored for media professionals. It streamlines finding live or archived clips to immediately playout, download, or share.

With a user-friendly web interface, TVU Search scales limitlessly for any number of concurrent users and live sources. It indexes video metadata like speech transcription, facial recognition, and captions. Real-time monitoring and custom views add convenience. Intuitive charts and graphs provide insights into clip usage and performance.

Built on Amazon Web Services (AWS), TVU Search leverages cloud scalability, flexibility, and resilience. Its microservices architecture enables rapid deployment and updates.

Features and benefits

By simplifying and speeding up content discovery and utilization, TVU Search helps broadcasters:

• Quickly capitalize on trending news and social media stories
• Engage viewers with timely, relevant content
• Drive greater value from live and archived video assets

The Timeline view in TVU Search provides an overview of content feeds

TVU Search’s speech recognition feature lets users find video content based on phrases or keywords, and highlight an area in the transcript to set a clip’s mark-in and mark-out timecode.

The speech recognition panel displays the transcript generated by TVU Search AI

The Live Monitor page offers customizable multi-view content sources based on a user’s chosen criteria and search parameters

Users employ keywords, phrases, and images to search content. The “search by face” feature includes image search. When an image is uploaded, TVU Search finds content that matches people based on facial recognition capabilities, such as in video clips.

“Search by face” allows for identification matching using images

Facial detection is another feature that facilitates rapid search. AI and machine learning (ML) are used to detect and display faces as thumbnails in a marked area on the video. Clicking on one of the face thumbnails lets the user navigate directly to the timestamp in the video where the person appears.

A user can easily navigate to the exact timestamp in a video by clicking on faces from a thumbnail display.

What makes TVU Search effective?

TVU Search lets users manage a large number of live IP video streams, including monitoring, ingesting and recording live content from virtually anywhere with automatic AI-generated metadata. Users can quickly search for specific live or recorded content based on customizable criteria such as keywords, faces, or spoken words.

Users follow simple, intuitive steps when searching for content in TVU Search:

1. Automated or hands-on Search: Users can take advantage of the AI-driven Active Search feature for live workflows, enabling time-saving automation of various actions, such as creating and exporting a video clip, notifications, and applying searchable tags based on specific criteria within user-defined search profiles. Alternatively, users have the flexibility to search manually for content based on specified criteria.
2. Live monitoring: TVU Search lets users create personalized and practical multi-views of live sources, including the ability to customize the display type, search and filter sources, and make adjustments to the preview position. Additionally, users can select sources to be displayed and effectively manage live recordings and outputs.
3. Thumbnail preview and real-time transcription: Once a search is performed, the system presents users with search results accompanied by visual thumbnails and relevant metadata, letting users quickly assess content. A rolling transcription accompanies search results, allowing users to preview the content alongside the synchronized transcription.
4. Content download: Recorded content metadata for speech transcription and facial recognition is indexed with timecode. Users can easily configure the mark-in and mark-out timecode of their clips by highlighting text from the speech or manually setting the timecode using the player tools. This flexibility gives users complete control over extraction of desired content.
5. Immediate Distribution: After extracting desired content, users can route it for immediate playout, download, or sharing for broadcast on multiple viewing platforms, including social media.

Architecture diagram

Behind the scenes, TVU Search uses AWS services and serverless architecture to power its capabilities enabling users to capture, analyze, record, extract, transcode, and share content in real-time.

As video sources upload to TVU Search, the platform processes incoming streams using Amazon Elastic Container Services for Kubernetes (EKS) for content transcoding, restoration, exporting, and search. Rapid content delivery to the cloud is supported by leveraging Amazon Cloudfront alongside storage and database services including Amazon Simple Storage Service (Amazon S3), Amazon Relational Database Service (Amazon RDS), and others.

TVU Search also uses Amazon Rekognition – a service that makes it easy to add powerful visual analysis to your applications – to extract content metadata for facial recognition, facial tracking, and scene analysis, among other functions. The information from the video streams are then parsed and supplemented with additional metadata from various sources within the TVU ecosystem.

Transcoding of content is performed by Amazon Elastic Transcoder and AWS Lambda, where AWS Lambda operations are orchestrated using AWS Step Functions.

Supported formats are:

• Input: TVU Grid, SRT, RTMP, UDP, HLS, RTSP
• Output: HLS, RTP-FEC, SRT, Zixi, RTMP(S), UDP, RTSP
• Export: MPEG-4, MXF, MOV, MP4, TS (video); SRT, TXT, DOCX (captions)

Summary

TVU Search, powered by AWS, is changing the media and entertainment landscape by transforming content discovery and creation processes. Instead of taking hours or days to search, locate, and clip video content, users can now find the content they need in minutes, and immediately share it with viewers globally.

Learn more about TVU Search here.

About TVU Networks

TVU Networks is a global provider of SaaS and cloud-based workflow solutions across multiple industries, including news, entertainment media, sports, corporate, streaming, houses of worship, and government. Through the use of AI, microservices, and automation-driven technology, TVU helps enterprises realize an efficient metadata and story-centric workflow through its ecosystem for live video acquisition, production, and distribution. The company has been on the forefront of developing the tools needed to streamline and monetize the Media Supply Chain. TVU is a critical part of the operations of major media companies worldwide and is a Technology and Engineering Emmy Award winner.