What does this AWS Solutions Implementation do?
This solution automatically generates multi-language subtitles for live streaming video content.
Using Machine Learning (ML) services for transcription and translation, this solution enables you to automate the subtitling process without the need for a human stenographer and allows for a single input language and up to five translated caption languages. Similar to a stenographer, the subtitles are slightly time-delayed from the audio. This solution might require additional customization or human supervision for broadcast-grade applications.
AWS Solutions Implementation overview
AWS provides a real-time subtitling solution for live streaming video content that combines Amazon Transcribe, Amazon Translate, and AWS Lambda to build a serverless architecture that automatically generates multi-language subtitles for your live streaming videos. The diagram below presents the architecture you can automatically deploy using the solution's implementation guide and accompanying AWS CloudFormation template.
Live Streaming with Automated Multi-Language Subtitling architecture
The solution’s AWS CloudFormation template deploys Live Streaming on AWS with additional services:
- AWS Elemental MediaLive
- AWS Elemental MediaPackage
- Amazon CloudFront
- Amazon CloudWatch Events
- Amazon Simple Notification Service (Amazon SNS)
- Amazon Simple Storage Service (Amazon S3)
- Amazon Transcribe
- Amazon Elastic Container Service (Amazon ECS)
- Amazon Translate
- AWS Lambda
In order to get WebVTT subtitles into AWS Elemental MediaPackage, we use the Lambda@Edge function in Amazon CloudFront to insert subtitles into the HLS stream, which is sent from AWS Elemental MediaLive to AWS Elemental MediaPackage. Amazon MediaLive outputs HLS to an Amazon CloudFront endpoint. This endpoint passes through the video files, manifests, and only invokes the Lambda@Edge function for the WebVTT subtitle files passing from AWS Elemental MediaLive to AWS Elemental MediaPackage. Subtitles are inserted into these WebVTT files before they are passed on to Amazon MediaPackage.
AWS Elemental MediaLive outputs an audio-only User Datagram Protocol (UDP) stream to an Amazon ECS container. This container transmits the audio to Amazon Transcribe Streaming, which receives the text contained in the stream as asynchronous responses and writes each text response to an Amazon Dynamo DB table. This Amazon ECS container also sends Amazon SNS notifications to an Amazon Translate Lambda function, which creates translated subtitles that are written to the same Amazon Dynamo DB table.
Each WebVTT file invokes the Lambda@Edge function, which inserts the subtitles and then transmits them onto the MediaPackage. MediaLive provides the authentication headers.
Note: To subscribe to RSS updates, you must have an RSS plug-in enabled for the browser you are using.
Browse our library of AWS Solutions Implementations to get answers to common architectural problems.
Find AWS certified consulting and technology partners to help you get started.
Browse our portfolio of Consulting Offers to get AWS-vetted help with solution deployment.