What does this AWS Solutions Implementation do?

This solution automatically generates multi-language subtitles for live streaming video content.

Using Machine Learning (ML) services for transcription and translation, this solution enables you to automate the subtitling process without the need for a human stenographer and allows for a single input language and up to five translated caption languages. Similar to a stenographer, the subtitles are slightly time-delayed from the audio. This solution might require additional customization or human supervision for broadcast-grade applications.  

AWS Solutions Implementation overview

AWS provides a real-time subtitling solution for live streaming video content that combines Amazon Transcribe, Amazon Translate, and AWS Lambda to build a serverless architecture that automatically generates multi-language subtitles for your live streaming videos. The diagram below presents the architecture you can automatically deploy using the solution's implementation guide and accompanying AWS CloudFormation template.

Live Streaming with Automated Multi-Language Subtitling | Architecture Diagram
 Click to enlarge

Live Streaming with Automated Multi-Language Subtitling architecture

The solution’s AWS CloudFormation template deploys Live Streaming on AWS with additional services:

 

In order to get WebVTT subtitles into AWS Elemental MediaPackage, we use the Lambda@Edge function in Amazon CloudFront to insert subtitles into the HLS stream, which is sent from AWS Elemental MediaLive to AWS Elemental MediaPackage. Amazon MediaLive outputs HLS to an Amazon CloudFront endpoint. This endpoint passes through the video files, manifests, and only invokes the Lambda@Edge function for the WebVTT subtitle files passing from AWS Elemental MediaLive to AWS Elemental MediaPackage. Subtitles are inserted into these WebVTT files before they are passed on to Amazon MediaPackage.

AWS Elemental MediaLive outputs an audio-only User Datagram Protocol (UDP) stream to an Amazon ECS container. This container transmits the audio to Amazon Transcribe Streaming, which receives the text contained in the stream as asynchronous responses and writes each text response to an Amazon Dynamo DB table. This Amazon ECS container also sends Amazon SNS notifications to an Amazon Translate Lambda function, which creates translated subtitles that are written to the same Amazon Dynamo DB table.

Each WebVTT file invokes the Lambda@Edge function, which inserts the subtitles and then transmits them onto the MediaPackage. MediaLive provides the authentication headers.

MediaPackage ingests the files and packages them into formats that are delivered to four MediaPackage custom endpoints.
 
An Amazon CloudFront distribution is configured to use the MediaPackage custom endpoints as its origin. This CloudFront URL is what is provided to the viewers of the live steam.

Live Streaming with Automated Multi-Language Subtitling

Version 2.0.0
Last updated: 12/2020
Author: AWS

Estimated deployment time: 20 min

Use the button below to subscribe to solution updates.

Note: To subscribe to RSS updates, you must have an RSS plug-in enabled for the browser you are using. 

Did this Solutions Implementation help you?
Provide feedback 

Features

Accessibility

Help improve the reach and access of your live video streaming content by exposing your content to a much larger audience with multi-language subtitles.

Flexibility

You can use this solution out-of-the-box, customize the solution to meet your specific use case, or leverage APN partner products to implement an end-to-end subtitling workflow.
Solving with Solutions: Live Streaming with Automated Multi-Language Subtitling
Build icon
Deploy a Solution yourself

Browse our library of AWS Solutions Implementations to get answers to common architectural problems.

Learn more 
Find an APN partner
Find an APN Partner

Find AWS certified consulting and technology partners to help you get started.

Learn more 
Explore icon
Explore Solutions Consulting Offers

Browse our portfolio of Consulting Offers to get AWS-vetted help with solution deployment.

Learn more