AWS for M&E Blog
Anatomy of a live streaming AWS CloudFormation template
To build an end-to-end, live video streaming workflow with AWS, you typically begin by setting up each of the required AWS Media Services through the AWS Management Console. The console is a great tool to explore all the features each Media Service offers, and the settings you can enable, disable, or tweak as you put all the pieces together. However, once you set up the resources exactly as you want, you may find it tedious to use the console to replicate your setup in other Regions, or recreating a previous workflow for another event. Automating this process, and only using the console to make minor changes to resources generated by your automation, may be the best way to work.
AWS CloudFormation is the typical choice to automate live streaming workflows, as it has native support for Media Services like AWS Elemental MediaLive and AWS Elemental MediaStore. You can find a number of published examples of CloudFormation templates to deploy a live steaming workflow from AWS and other media enthusiasts on sites like GitHub. The automation typically includes a MediaLive input, a MediaLive channel, and a destination, which is either a MediaStore container or an AWS Elemental MediaPackage channel.
If you’re new to CloudFormation, however, the example templates can look daunting and might appear hard to decipher. So, in this blog, we take a fully deployable simple, live streaming CloudFormation template and evaluate each of its sections: MediaStore container, MediaLive HLS input, and single-pipeline MediaLive channel. You may find that the resources and settings defined in the template typically correspond to the same resources and settings that you would configure in the console.
You can download the sample CloudFormation template that we deconstruct here. It is written in JavaScript Object Notation (JSON) format. Throughout this blog, the CloudFormation documentation for MediaStore and MediaLive is cross-referenced. If you’re a CloudFormation novice, it is highly recommended that you give this CloudFormation template structure documentation a quick read, as it goes over the typical sections of a CloudFormation template.
MediaStore Container
The first resource defined in the CloudFormation template is the MediaLive channel’s destination, which is a MediaStore container.
{
"Resources": {
"MediaStore": {
"Type": "AWS::MediaStore::Container",
"Properties": {
"ContainerName": {
"Fn::Sub": "${AWS::StackName}"
},
"AccessLoggingEnabled": true,
"Policy": {"Fn::Sub": "{\n \"Version\" : \"2012-10-17\",\n \"Statement\" : [{\n \"Sid\" : \"PublicReadOverHttps\",\n \"Effect\" : \"Allow\",\n \"Principal\" : \"*\",\n \"Action\" : [\n \"mediastore:GetObject\",\n \"mediastore:DescribeObject\"\n ],\n \"Resource\" : \"arn:aws:mediastore:${AWS::Region}:${AWS::AccountId}:container/${AWS::StackName}/*\",\n \"Condition\" : {\"Bool\" : {\n \"aws:SecureTransport\" : \"true\"\n }\n }\n }]\n}\n"},
"CorsPolicy": [
{
"AllowedHeaders": [
"*"
],
"AllowedMethods": [
"GET"
],
"AllowedOrigins": [
"*"
],
"ExposeHeaders": [
"*"
],
"MaxAgeSeconds": 3000
}
],
}
}
}
}
The preceding JSON structure represents the section of the template that defines the MediaStore container. It has a few defined properties including ContainerName, which takes on the value of the StackName you provide during deployment. It is the only required parameter by the MediaStore container. The built-in CloudFormation function Fn::Sub is used to get the value of the StackName at runtime. By doing this little trick, the container name becomes dynamic and the user doesn’t have to provide a separate container name every time the template is deployed. CloudWatch logging is enabled by setting AccessLoggingEnabled to true. A container Policy is set to allow anyone to read items from the container but through secure transport only (HTTPS), as stipulated by the Condition statement in the Policy. This allows the video stream to be played back through the MediaStore container’s data endpoint later on. The function Fn::Sub is used once more to put together the MediaStore container ARN by replacing the Region, AccountId, and StackName with actual values at runtime. The Policy is required to be in a string format, hence all the escaped quotes. The CorsPolicy allows client web applications, like an embedded web video player, that are loaded in one domain to access the video files, which are in another domain. MediaStore uses the same structure for CorsPolicy as S3 and you can learn more about it here.
MediaLive Input
Moving on to the next section, let’s look at the MediaLive HLS input the MediaLive channel uses.
{
"Resources": {
"MediaStore": {
...
},
"MediaLiveInput":{
"Type": "AWS::MediaLive::Input",
"Properties": {
"Name": {
"Fn::Sub": "${AWS::StackName}-HLSInput"
},
"Type": "URL_PULL",
"Sources": [
{
"Url": "http://d2qohgpffhaffh.cloudfront.net/HLS/vanlife/sdr_uncage_vanlife.m3u8"
}
]
}
}
}
} The HLS input has a few defined properties: Name, Sources, and Type. To make the Name dynamic, the variable StackName is prepended to the string –HLSInput. This way, if you deploy the stack more than once, you don’t end up with multiple HLS inputs with the same name. If you deploy this template with a Stack Name “MyLiveStreaming”, the name of the HLS input created is “MyLiveStreaming-HLSInput”. The Type corresponds to an HLS input is URL_PULL. Finally, the URL of the HLS source is provided. The Sources property takes a list of URLs, but this template creates a single pipeline channel, so only one source is needed.
MediaLive Channel
And now, we examine the final piece, the MediaLive channel.
{
"Resources": {
"MediaStore": {
...
},
"MediaLiveInput":{
...
},
"MediaLiveChannel": {
"Type": "AWS::MediaLive::Channel",
"Properties": {
"Name": {
"Fn::Sub": "${AWS::StackName}-Channel"
},
"ChannelClass": "SINGLE_PIPELINE",
"RoleArn": {"Ref": "MediaLiveRoleArn"},
"EncoderSettings": {
"AudioDescriptions": [
{
"Name": "audio_1",
"CodecSettingsChoice": "AAC",
"LanguageCodeControl": "FOLLOW_INPUT",
"AudioTypeControl": "FOLLOW_INPUT"
}
],
"OutputGroups": [
{
"OutputGroupSettings": {
"HlsGroupSettings": {
"InputLossAction": "EMIT_OUTPUT",
"DirectoryStructure": "SINGLE_DIRECTORY",
"SegmentsPerSubdirectory": 10000,
"OutputSelection": "MANIFESTS_AND_SEGMENTS",
"Mode": "LIVE",
"TsFileMode": "SEGMENTED_FILES",
"StreamInfResolution": "INCLUDE",
"ManifestDurationFormat": "FLOATING_POINT",
"SegmentLength": 10,
"IndexNSegments": 10,
"KeepSegments": 21,
"SegmentationMode": "USE_SEGMENT_DURATION",
"IFrameOnlyPlaylists": "DISABLED",
"ProgramDateTime": "EXCLUDE",
"ProgramDateTimePeriod": 600,
"ClientCache": "ENABLED",
"CodecSpecification": "RFC_4281",
"ManifestCompression": "NONE",
"RedundantManifest": "DISABLED",
"IvInManifest": "INCLUDE",
"IvSource": "FOLLOWS_SEGMENT_NUMBER",
"CaptionLanguageSetting": "OMIT",
"TimedMetadataId3Frame": "PRIV",
"TimedMetadataId3Period": 10,
"Destination": {
"DestinationRefId": "destination1"
},
"HlsCdnSettings": {
"HlsMediaStoreSettings": {
"MediaStoreStorageClass": "TEMPORAL",
"ConnectionRetryInterval": 1,
"NumRetries": 10,
"FilecacheDuration": 300,
"RestartDelay": 15
}
}
}
},
"Outputs": [
{
"OutputName": "HLSOutput",
"OutputSettings": {
"HlsOutputSettings": {
"NameModifier": 1,
"HlsSettings": {
"StandardHlsSettings": {
"AudioRenditionSets": "program_audio",
"M3u8Settings": {
"ProgramNum": 1,
"AudioFramesPerPes": 4,
"PmtPid": "480",
"VideoPid": "481",
"PcrControl": "PCR_EVERY_PES_PACKET",
"AudioPids": "492-498",
"Scte35Behavior": "NO_PASSTHROUGH",
"Scte35Pid": "500",
"TimedMetadataBehavior": "NO_PASSTHROUGH",
"TimedMetadataPid": "502"
}
}
}
}
},
"VideoDescriptionName": "video_1",
"AudioDescriptionNames": [
"audio_1"
]
}
]
}
],
"VideoDescriptions": [
{
"Name": "video_1",
"ScalingBehavior": "DEFAULT",
"Sharpness": 50,
"RespondToAfd": "NONE"
}
],
"TimecodeConfig": {
"Source": "EMBEDDED"
}
},
"Destinations": [
{
"Id": "destination1",
"Settings": [
{
"Url": {
"Fn::Sub": [
"mediastoressl://${EndpointSansProtocol}/out/index",
{
"EndpointSansProtocol": {
"Fn::Select": [
1,
{
"Fn::Split": [
"://",
{
"Fn::GetAtt": [
"MediaStore",
"Endpoint"
]
}
]
}
]
}
}
]
}
}
]
}
],
"InputAttachments": [
{
"InputAttachmentName": "HLSInput",
"InputId": {
"Ref": "MediaLiveInput"
},
"InputSettings": {
"SourceEndBehavior": "LOOP"
}
}
]
}
}
},
"Parameters": {
"MediaLiveRoleArn": {
"Description": "ARN of the IAM role that MediaLive is allowed to assume in order to perform the operations on the resources specified in the policies.",
"Type": "String"
}
},
"Outputs": {
"LiveStreamUrl": {
"Description": "LiveStream Playback URL",
"Value": {
"Fn::Sub": "${MediaStore.Endpoint}/out/index.m3u8"
}
}
}
}
As you can see from the preceding structure, this is a much more substantial resource definition compared to other resources defined so far. There are also two new CloudFormation sections outside of the Resources section, namely the Parameters and Outputs section.
Let’s start with MediaLive Channel resource definition first. As with the MediaLive input name, the variable StackName is prepended to the word -Channel to make the channel Name dynamic. ChannelClass is set to SINGLE_PIPELINE to override the default Standard channel type. In order to give MediaLive the permission to access resources needed to run the channel, a RoleArn is provided. You can generate this IAM role manually via the MediaLive console by following the instructions here. You can then copy the role ARN and provide it as an input when deploying the template. But how, exactly, do you get this input from the user? As you probably have guessed, from the Parameters section. The MediaLiveRoleArn required from the user is given an appropriate description to give guidance on what this value should be. Its type must also be declared, which in this case is a String.
Now back to the rest of the MediaLive channel’s properties. The EncoderSettings define the video and audio settings for the channel’s outputs. Admittedly, this part looks overwhelming, but this is where you rely on the channel(s) you previously created on the MediaLive console to figure out what goes in this section. To build out this part of the template, export the data of an existing MediaLive channel that has the settings you want from the MediaLive console. Then, take the EncoderSettings definition of that exported data and use it in this section. We won’t walk through each and every single parameter of the EncodingSettings, but if you want to see the required parameters and how to specify them, consult the MediaLive API. There are a few items worth highlighting:
- The OutputGroups defines a HlsGroupSettings since we want an HLS output.
- The HlsCdnSettings defines a HlsMediaStoreSettings since our destination is MediaStore.
- The VideoDescriptionName used in the Outputs settings must match the VideoDescriptions section’s Name.
- The AudioDescriptions name used in the Outputs section must match the AudioDescriptions section’s Name.
- The DestinationRefId in HlsGroupSettings must match the Id in Destinations.
The Settings parameter in the Destinations section seems complicated so let’s take a closer look. What’s happening here is the Url value is set to: mediastoressl://${EndpointSansProtocol}/out/index. But what exactly is the ${EndpointSansProtocol} variable? Well, it’s the MediaStore container’s endpoint but without the https protocol. To get that value, first, get the MediaStore container’s endpoint (which typically looks something like https://ojb5o3ll2w42uk.data.mediastore.us-east-1.amazonaws.com) using the Fn::GetAtt function on the MediaStore resource. That URL is split apart using the function Fn::Split with a :// delimiter in order to get at the domain portion of the URL. The Fn::Select function is used to actually pick out the domain. So at runtime, that URL gets evaluated to something like mediastoressl://ojb5o3ll2w42uk.data.mediastore.us-east-1.amazonaws.com/out/index. Again, this is simply taking advantage of the built-in functions that CloudFormation provides in order to construct the URL we need.
In the InputAttachments, the Ref function is used to pass along the Input Id that is generated when the HLS input is created at runtime. The input is looped by setting the SourceEndBehavior to LOOP, otherwise MediaLive stops processing any video as soon as it gets to the end of the file.
And now for the final bit, the Outputs section. This section is not required, but allows you to display certain values created by the stack after deployment. This section gives the full URL path needed to watch the video stream. You can also get this path by going directly to your MediaStore container once it’s been created, getting its data endpoint, and constructing this path yourself. But here, it’s made available in the Outputs section of CloudFormation as soon as the template is fully deployed.
And that’s it! If you’d like to see this simple live streaming workflow in action, deploy the sample CloudFormation template in your own account. Once deployed, you should see the following resources created in your account:
MediaStore container created by template
MediaLive input created by template
MediaLive channel created by template
Verify that all the settings are as expected, and when you’re ready, start the MediaLive channel. To play back the stream, go to the Outputs section of your CloudFormation stack, copy the stream URL, and paste that URL on your Safari browser, or any other HLS player of your choice.
Outputs tab of the deployed CloudFormation template
In production, you won’t want to access or share that MediaStore endpoint URL directly; instead use a content distribution network (CDN) like AWS CloudFront. CloudFormation has support for CloudFront, so the template provided here can be extended to accommodate that if needed.
When you finish testing the template, stop the MediaLive Channel, empty the MediaStore container, then delete the stack from the CloudFormation console. If you don’t empty the container first, CloudFormation fails to delete your MediaStore container. By deleting the stack, all resources it created are deleted and avoid incurring any unintended charges.
We hope that this walkthrough helps demystify the main components needed to automate a simple live streaming workflow using CloudFormation. The following resources are used throughout this blog, which should help you make further customizations to the template presented here, or write your own.