AWS Media Blog

Processing multilingual track content using AWS Elemental MediaConvert

With the advent of over-the-top (OTT) platforms, users expect to consume content in their native language or language of choice, regardless of the content’s origin. It is therefore imperative for a platform to ingest multi-track audio in source format and convert it to the required output profile with all the original tracks intact. In this blog post, we describe a simple step-by-step process to use fully managed services from Amazon Web Services (AWS)—in particular, AWS Elemental MediaConvert (MediaConvert), a file-based video transcoding service with broadcast-grade features—to ingest a multilingual media file and process it to the required output profile with multi-audio tracks associated with respective end-user languages.

Step 1: Understand the multilingual track source

Let’s consider three variants of source for processing multi-audio track content.

  • Language code–embedded audio tracks—Some audio tracks have a three-letter language code, ISO 639, embedded in them. In the source information example that follows, audio stream #0:1 has “eng” language code, audio stream #0:2 has “hin,” and audio stream #0:3 has “tam.”

Highlighted Input audio tracks with embedded language codes

  • Partial-language-code audio tracks—Some audio tracks have language code embedded in them, and some audio tracks within the same source have missing language codes. In the following example, audio stream #0:1 has language code “eng,” audio stream #0:2 has language code “und,” and audio stream #0:3 has language code “tam.”

Highlighted Input audio track for second input type where some tracks have undefined lang code

  • No-language-code audio tracks—Sometimes language code is not associated with any audio tracks in the source. In the following sample output, the language code for all audio streams is configured as “und.”

Highlighted audio tracks where none of the tracks have language codes

Step 2: Create a MediaConvert  job

Follow these steps to create a MediaConvert job.

  • Open the MediaConvert console.
  • Choose Get started.

MediaConvert Console to get started

  • Choose the Create job

Create MediaConvert Job

Step 3: MediaConvert job—Source with language code for all tracks

  1. Configure input—For sources that have the language codes embedded in all tracks, follow these steps.
  • On the Create jobpage, in the Job pane on the left, choose Input 1.
  • In the Input 1pane, provide the URL to your video input file that is stored in Amazon Simple Storage Service (Amazon S3), which offers cloud object storage built to retrieve any amount of data from anywhere, or on an HTTP(S) server. For Amazon S3 inputs, you can specify the URL directly or choose Browse to select from your Amazon S3 buckets. For HTTP(S) inputs, provide the URL to your input video file. For more information, see HTTP input requirements.
  • In the Audio selectorssection, under Audio Selector 1, under Selector type, choose Language code.
  • For Language code, select English (matching the language of Stream #0:1).

Selector Type and Language Code selection in Audio selectors section

  • Select Add audio selector to create additional audio selector for audio stream #2.
  • For Selector type, select Language code.
  • For Language code, select Hindi (matching the language of Stream #0:2).

Selector Type and Language Code selection for 2nd Audio Selection

  • Select Add audio selector to create additional audio selector for audio stream #3.
  • For Selector type, select Language code.
  • For Language code, select Tamil (matching the language of Stream #0:3).

Selector Type and Language Code selection for 3rd Audio Selector

2. Configure output

  • File group
    • On the Create job page, Jobpane, in the Output groups section, choose Add.

Add Output Group

    • Select File group and choose Select.

Select File Group as Output Group Type

    • Select Output 1 in Outputs.
    • Select Audio 1 in Encoding settings.
    • For Audio source, select Audio Selector 1.
    • In the Advanced group, for Language code control, select Follow input.

Language code control configuration for Audio 1

    • Select Add audio to add one more audio track.
    • Select Audio 2 in Encoding settings.
    • For Audio source, select Audio Selector 2.
    • In the Advanced group, for Language code control, select Follow input.

Language code control configuration for Audio 2

    • Select Add audio to add one more audio track.
    • Select Audio 3 in Encoding settings.
    • For Audio source, select Audio Selector 3.
    • In the Advanced group, for Language code control, select Follow input.

Language code control configuration for Audio 3

  • Apple HLS: For output as HLS, the only difference is that we create separate outputs for audio and video. The rest of the settings remain the same.
    • Select the created HLS output group.

Add output group for HLS

    • In the output section, click on Add output with captions or audio to add three different outputs, as shown.

Adding outputs for 3 different audio tracks for HLS output

    • Select Output 2 in Output settings. In Audio track type, select Alternate audio, auto select, default. This will be our default audio track that the client will play. Leave outputs 3 and 4 blank.

Setting Audio Track Type for HLS Audio output

    • For encoding settings, follow the steps mentioned in the File group
  • DASH ISO: When we select DASH as the Output group, by default, it creates separate video and audio outputs.
    • Select DASH output group

o	For encoding settings, follow the steps mentioned in the File group section. •	DASH ISO: When we select DASH as the Output group, by default, it creates separate video and audio outputs. o	Select DASH output group

    • In the Outputs section, click on Add output with captions or audio to add three different outputs, as follows.

Creating multiple outputs for DASH output

    • For encoding settings, follow the steps mentioned in the File group

Step 4: MediaConvert job—Source with partial language code for all tracks

  1. Configure input—For sources that have partial language codes without any duplicates, follow the input configuration steps for the MediaConvert job.
  • On the Create jobpage, in the Job pane on the left, choose Input 1.
  • In the Input 1pane, provide the URL to your video input file that is stored in Amazon S3 or on an HTTP(S) server. For Amazon S3 inputs, you can specify the URL directly or choose Browse to select from your Amazon S3 buckets. For HTTP(S) inputs, provide the URL to your input video file.
  • In the Audio selectorssection, under Audio Selector 1, under Selector type, choose Language code.
  • For Language code, select English (matching the language of Stream #0:1).

Selecting Language Code for the Audio Selection where input has Language Code

  • Select Add audio selector to create additional audio selector for audio stream #2.
  • For Selector type, select Track.
  • For Tracks, enter 2 (matching the track index of audio Stream #0:2; the Hindi language will be set on the output configuration).

Setting Selector Type and Tracks for input where Language Code is not defined

  • Select Add audio selector to create additional audio selector for audio stream #3.
  • For Selector type, select Language code.
  • For Language code, select Tamil (matching the language of Stream #0:3).

Selecting Language Code for the Audio Selection where input has Language Code is Tamil

2. Configure output

  • File group
    • On the Create job page, Jobpane, in the Output groups section, choose Add.

Add output group

    • Select File group and choose Select.

Selecting output group type

    • Select Output 1 in Outputs.
    • Select Audio 1 in Encoding settings.
    • For Audio source, select Audio Selector 1.
    • In the Advanced group, for Language code control, select Follow input.

Setting Language code control as follow input for Audio 1

    • Select Add audio to add one more audio track.
    • Select Audio 2 in Encoding settings.
    • For Audio source, select Audio Selector 2.
    • In the Advanced group, for Language code, select Hindi.

Setting Language code as Hindi for Audio 2

    • Select Add audio to add one more audio track.
    • Select Audio 3 in Encoding settings.
    • For Audio source, select Audio Selector 3.
    • In the Advanced group, for Language code control, select Follow input.

Setting Language code control as follow input for Audio 3

Step 5: MediaConvert job—Source with no language code for all tracks

  1. Configure input—For sources that have no language codes, follow the input configuration steps for the MediaConvert job.
  • On the Create jobpage, in the Job pane on the left, choose Input 1.
  • In the Input 1pane, provide the URL to your video input file that is stored in Amazon S3 or on an HTTP(S) server. For Amazon S3 inputs, you can specify the URL directly or choose Browse to select from your Amazon S3 buckets. For HTTP(S) inputs, provide the URL to your input video file.
  • In the Audio selectorssection, under Audio Selector 1, under Selector type, choose Track.
  • For Tracks, enter 1 (matching the stream index of Stream #0:1).

Setting Selector Type and Tracks for input where Language Code is not defined

  • Select Add audio selector to create additional audio selector for audio stream #2.
  • For Selector type, select Track.
  • For Tracks, enter 2 (matching the stream index of Stream #0:2).

Setting Selector Type and Tracks for input where Language Code is not defined

  • Select Add audio selector to create additional audio selector for audio stream #3.
  • For Selector type, select Track.
  • For Tracks, enter 3 (matching the stream index of Stream #0:3).

Setting Selector Type and Tracks for input where Language Code is not defined

2. Configure output

  • File group
    • On the Create job page, Jobpane, in the Output groups section, choose Add.

Add output group

  • Select File group and choose Select.

Selecting Output Group Type

    • Select Output 1, in Outputs.
    • Select Audio 1 in Encoding settings.
    • For Audio source, select Audio Selector 1.
    • In the Advanced group, for Language code control, select Follow input.

Setting Language code as English for Audio 1

    • Select Add audio to add one more audio track.
    • Select Audio 2 in Encoding settings.
    • For Audio source, select Audio Selector 2.
    • In the Advanced group, for Language code, select Hindi.

Setting Language code as Hindi for Audio 2

    • Select Add audio to add one more audio track.
    • Select Audio 3 in Encoding settings.
    • For Audio source, select Audio Selector 3.
    • In the Advanced group, for Language code, select Tamil.

Setting Language code as Tamil for Audio 3

 Step 6: Examine the output

o   File group

Examine the generated MP4 file in the Amazon S3 bucket by using tools like media info or ffprobe. We should see the language code tagged to the audio tracks.

Highlighted Language in the output for File Group

o   DASH ISO

In DASH output, we will see a master mp4 file. Simply download it and open in any text editor. You should see three adaptation sets for the three audio tracks that we chose.

Highlighted Lang Code in output for DASH

o   HLS

To examine the HLS output, open the master m3u8 in any text editor. You should see the three audio tracks with language codes tagged.

Highlighted Language in the output for HLS

Conclusion

In this post, we showed you step by step how to process multilingual audio track media content using MediaConvert to generate output types in File group, Apple HLS, and DASH ISO.

Learn more

AWS provides a growing number of services designed to assist in the building of media-related workflows. If you would like to explore additional applications for video streaming, processing, and delivery, visit AWS Media Services.

Yash Goel

Yash Goel

Yash Goel is an Enterprise Solutions Architect at AWS.

Maheshwaran G

Maheshwaran G

Maheshwaran G is a Specialist Solution Architect working with Media and Entertainment Supporting Media companies in India to accelerate growth in an innovative fashion leveraging the power of cloud technologies. He is passionate about innovation and currently holds 8 USPTO granted Patents in diversified domains.