Processing multilingual track content using AWS Elemental MediaConvert

With the advent of over-the-top (OTT) platforms, users expect to consume content in their native language or language of choice, regardless of the content’s origin. It is therefore imperative for a platform to ingest multi-track audio in source format and convert it to the required output profile with all the original tracks intact. In this blog post, we describe a simple step-by-step process to use fully managed services from Amazon Web Services (AWS)—in particular, AWS Elemental MediaConvert (MediaConvert), a file-based video transcoding service with broadcast-grade features—to ingest a multilingual media file and process it to the required output profile with multi-audio tracks associated with respective end-user languages.

Step 1: Understand the multilingual track source

Let’s consider three variants of source for processing multi-audio track content.

• Language code–embedded audio tracks—Some audio tracks have a three-letter language code, ISO 639, embedded in them. In the source information example that follows, audio stream #0:1 has “eng” language code, audio stream #0:2 has “hin,” and audio stream #0:3 has “tam.”

• Partial-language-code audio tracks—Some audio tracks have language code embedded in them, and some audio tracks within the same source have missing language codes. In the following example, audio stream #0:1 has language code “eng,” audio stream #0:2 has language code “und,” and audio stream #0:3 has language code “tam.”

• No-language-code audio tracks—Sometimes language code is not associated with any audio tracks in the source. In the following sample output, the language code for all audio streams is configured as “und.”

Step 2: Create a MediaConvert  job

Follow these steps to create a MediaConvert job.

• Open the MediaConvert console.
• Choose Get started.

• Choose the Create job

Step 3: MediaConvert job—Source with language code for all tracks

1. Configure input—For sources that have the language codes embedded in all tracks, follow these steps.

• On the Create jobpage, in the Job pane on the left, choose Input 1.
• In the Input 1pane, provide the URL to your video input file that is stored in Amazon Simple Storage Service (Amazon S3), which offers cloud object storage built to retrieve any amount of data from anywhere, or on an HTTP(S) server. For Amazon S3 inputs, you can specify the URL directly or choose Browse to select from your Amazon S3 buckets. For HTTP(S) inputs, provide the URL to your input video file. For more information, see HTTP input requirements.
• In the Audio selectorssection, under Audio Selector 1, under Selector type, choose Language code.
• For Language code, select English (matching the language of Stream #0:1).

• Select Add audio selector to create additional audio selector for audio stream #2.
• For Selector type, select Language code.
• For Language code, select Hindi (matching the language of Stream #0:2).

• Select Add audio selector to create additional audio selector for audio stream #3.
• For Selector type, select Language code.
• For Language code, select Tamil (matching the language of Stream #0:3).

2. Configure output

• File group
  • On the Create job page, Jobpane, in the Output groups section, choose Add.

• • Select File group and choose Select.

• • Select Output 1 in Outputs.
  • Select Audio 1 in Encoding settings.
  • For Audio source, select Audio Selector 1.
  • In the Advanced group, for Language code control, select Follow input.

• • Select Add audio to add one more audio track.
  • Select Audio 2 in Encoding settings.
  • For Audio source, select Audio Selector 2.
  • In the Advanced group, for Language code control, select Follow input.

• • Select Add audio to add one more audio track.
  • Select Audio 3 in Encoding settings.
  • For Audio source, select Audio Selector 3.
  • In the Advanced group, for Language code control, select Follow input.

• Apple HLS: For output as HLS, the only difference is that we create separate outputs for audio and video. The rest of the settings remain the same.
  • Select the created HLS output group.

• • In the output section, click on Add output with captions or audio to add three different outputs, as shown.

• • Select Output 2 in Output settings. In Audio track type, select Alternate audio, auto select, default. This will be our default audio track that the client will play. Leave outputs 3 and 4 blank.

• • For encoding settings, follow the steps mentioned in the File group
• DASH ISO: When we select DASH as the Output group, by default, it creates separate video and audio outputs.
  • Select DASH output group

• • In the Outputs section, click on Add output with captions or audio to add three different outputs, as follows.

• • For encoding settings, follow the steps mentioned in the File group

Step 4: MediaConvert job—Source with partial language code for all tracks

1. Configure input—For sources that have partial language codes without any duplicates, follow the input configuration steps for the MediaConvert job.

• On the Create jobpage, in the Job pane on the left, choose Input 1.
• In the Input 1pane, provide the URL to your video input file that is stored in Amazon S3 or on an HTTP(S) server. For Amazon S3 inputs, you can specify the URL directly or choose Browse to select from your Amazon S3 buckets. For HTTP(S) inputs, provide the URL to your input video file.
• In the Audio selectorssection, under Audio Selector 1, under Selector type, choose Language code.
• For Language code, select English (matching the language of Stream #0:1).

• Select Add audio selector to create additional audio selector for audio stream #2.
• For Selector type, select Track.
• For Tracks, enter 2 (matching the track index of audio Stream #0:2; the Hindi language will be set on the output configuration).

• Select Add audio selector to create additional audio selector for audio stream #3.
• For Selector type, select Language code.
• For Language code, select Tamil (matching the language of Stream #0:3).

2. Configure output

• File group
  • On the Create job page, Jobpane, in the Output groups section, choose Add.

• • Select File group and choose Select.

• • Select Output 1 in Outputs.
  • Select Audio 1 in Encoding settings.
  • For Audio source, select Audio Selector 1.
  • In the Advanced group, for Language code control, select Follow input.

• • Select Add audio to add one more audio track.
  • Select Audio 2 in Encoding settings.
  • For Audio source, select Audio Selector 2.
  • In the Advanced group, for Language code, select Hindi.

• • Select Add audio to add one more audio track.
  • Select Audio 3 in Encoding settings.
  • For Audio source, select Audio Selector 3.
  • In the Advanced group, for Language code control, select Follow input.

Step 5: MediaConvert job—Source with no language code for all tracks

1. Configure input—For sources that have no language codes, follow the input configuration steps for the MediaConvert job.

• On the Create jobpage, in the Job pane on the left, choose Input 1.
• In the Input 1pane, provide the URL to your video input file that is stored in Amazon S3 or on an HTTP(S) server. For Amazon S3 inputs, you can specify the URL directly or choose Browse to select from your Amazon S3 buckets. For HTTP(S) inputs, provide the URL to your input video file.
• In the Audio selectorssection, under Audio Selector 1, under Selector type, choose Track.
• For Tracks, enter 1 (matching the stream index of Stream #0:1).

• Select Add audio selector to create additional audio selector for audio stream #2.
• For Selector type, select Track.
• For Tracks, enter 2 (matching the stream index of Stream #0:2).

• Select Add audio selector to create additional audio selector for audio stream #3.
• For Selector type, select Track.
• For Tracks, enter 3 (matching the stream index of Stream #0:3).

2. Configure output

• File group
  • On the Create job page, Jobpane, in the Output groups section, choose Add.

• Select File group and choose Select.

• • Select Output 1, in Outputs.
  • Select Audio 1 in Encoding settings.
  • For Audio source, select Audio Selector 1.
  • In the Advanced group, for Language code control, select Follow input.

• • Select Add audio to add one more audio track.
  • Select Audio 2 in Encoding settings.
  • For Audio source, select Audio Selector 2.
  • In the Advanced group, for Language code, select Hindi.

• • Select Add audio to add one more audio track.
  • Select Audio 3 in Encoding settings.
  • For Audio source, select Audio Selector 3.
  • In the Advanced group, for Language code, select Tamil.

 Step 6: Examine the output

o   File group

Examine the generated MP4 file in the Amazon S3 bucket by using tools like media info or ffprobe. We should see the language code tagged to the audio tracks.

o   DASH ISO

In DASH output, we will see a master mp4 file. Simply download it and open in any text editor. You should see three adaptation sets for the three audio tracks that we chose.

o   HLS

To examine the HLS output, open the master m3u8 in any text editor. You should see the three audio tracks with language codes tagged.

Conclusion

In this post, we showed you step by step how to process multilingual audio track media content using MediaConvert to generate output types in File group, Apple HLS, and DASH ISO.

Learn more

AWS provides a growing number of services designed to assist in the building of media-related workflows. If you would like to explore additional applications for video streaming, processing, and delivery, visit AWS Media Services.