Discover, visualize, and monitor your live video workflows on AWS

Introduction

If you broadcast, stream, or distribute live video in the cloud, chances are you’ve spent time building, testing, and securing your workflows. Amazon Web Services (AWS) makes these tasks easy, so you can deliver high-quality experiences to your viewers. But how much time have you spent with the ongoing monitoring of your workflows?

If you dive into monitoring, you’ll find there’s a range of service-specific alerts and dashboards available for each AWS service. However, no single overview is available to view alarms, metrics, resource alerts, and health across an entire media workflow. When relationships between resources are not clear, that leads to time-consuming manual diagnoses and remediation when issues arise. You also may not know which metrics are most useful for monitoring, and find you need a repeatable means to deploy and maintain alarms and notification rules across multiple workflows.

Discover and visualize your live video with the workflow monitor

To help solve these issues, AWS launched the workflow monitor for AWS Media Services and Amazon CloudFront. This new capability is designed to discover, visualize, and monitor your live video workflows. You use the workflow monitor to deploy and maintain best practice monitoring of your media workloads, with access via the AWS Management Console, AWS Elemental MediaLive API, and AWS CDK.

Services discoverable by the workflow monitor.

In just a few clicks, the workflow monitor can discover and visualize the resources associated with your media workflows. Discovery can begin at any supported service, and once a resource is selected, the workflow monitor creates an end-to-end graphical signal map. A signal map represents the resources in a given workflow across AWS Elemental MediaConnect, AWS Elemental MediaLive, and AWS Elemental MediaPackage, along with Amazon Simple Storage Service (Amazon S3) buckets and Amazon CloudFront distributions.

In a signal map, you can view at a glance which resources are in use and the status of each resource, which can be updated every second. You can also see how each of the resources are connected without having to look across multiple consoles. Certain resources, such as a MediaLive channel or MediaPackage endpoint, display dynamically updating image thumbnails from the live video, if available.

Signal map showing a live video distribution (click image to zoom in).

Apply and customize monitoring and alerting with industry-designed best practices

Once signal maps are created, you use the workflow monitor to create and apply alarm and notification templates to alert you when issues arise. Monitoring and alerting are done using Amazon CloudWatch alarms and Amazon EventBridge rules, but the configuration of these resources is managed with the workflow monitor console and deployed via Amazon CloudFormation. This means your video-centric engineering teams don’t need to become well versed in using these services. To make monitoring of similar workloads repeatable, the workflow monitor uses the concept of templates and template groups. Templates describe the configuration of a CloudWatch alarm or an EventBridge rule and are grouped to provide a hierarchical object that can be associated to one or more signal maps.

Template groups can be sliced different ways to best support your operational model. For example, you could divide alarm templates into groups of severity, based on the relative value of a given workflow, or based on regional differences. Plus, the workflow monitor takes CloudWatch alarm templates a step further and provides a set of best practice recommendations, created by a team of industry experts, that you can import into your own alarm template groups. Once imported, these templates can be modified to meet the specific requirements of your workflow. For example, you might change the video frame rate to 50 instead of 60 frames per second or adjust a duration threshold based on your specific tolerances.

Once you associate one or more template groups to a signal map, you can quickly deploy them from the workflow monitor. In the background, a CloudFormation stack is created and saved to Amazon S3, and then deployed to your workflow. This stack is accessible in case you want to reference it later, or use it as part of another CloudFormation deployment. When the deployment is complete, the workload is now monitored.

View status across your video workflows

The overview page in the workflow monitor provides at-a-glance health status across all of your signal maps. You can also filter the list in several ways. For instance, if you filter based on status, you can create a “penalty box” to only show signal maps that are in an alarm state. Monitoring by exception like this is useful when you have a large number of signal maps and only want to see those that might need an operator’s attention.

The overview shows the status of each signal map (click image to zoom in).

You can click on any signal map that’s being monitored and see which resources have active alarms, which are visually indicated by a red background on the resource. Selecting the resource filters the list of alarms applied to the signal map to just that resource. This allows you and your operators to quickly identify where a problem lies in a given workflow and visually debug it. You can then use the built-in links to get directly into the resource itself, CloudWatch, or CloudWatch Insights to query logs. Previously, isolating an issue and getting to the right spot to troubleshoot took time and relied on specialized knowledge of a workflow.

Maintain consistency in monitoring

Beyond the creation and deployment of best practice monitoring, the workflow monitor helps you maintain consistency with your operations. Once alarm template groups are associated to signal maps, the alarm templates can be updated at any time and new configurations trickle down to signal maps. For example, you might need to change a threshold to make an alarm less noisy. Instead of manually changing a particular threshold across every resource where the alarm is used, you update it once in the alarm template and then re-deploy the monitoring. Template groups also allow you to effectively stage changes across all of your workloads and then deploy when the time is right. For example, you might be updating video encoding from AVC to HEVC and need to adjust bitrate settings on certain alarms. Instead of updating each alarm individually, you can update all of them in one spot in the alarm template and deploy when you’re ready.

The workflow monitor user guide also has suggestions for IAM roles. The permission boundaries help you to create non-destructive roles for operators to use the workflow monitor. Additionally, there’s a role suggestion for an experienced engineer to create, update, and deploy monitoring. These two roles help provide a secure starting point to use the workflow monitor.

Conclusion

The workflow monitor for AWS Media Services and Amazon CloudFront discovers, visualizes, and monitors the resources associated with your live video stream, broadcast, or distribution. It displays the relationships between resources in a graphical signal map, so you can see which resources are in use and how they are connected. Start with best practice monitoring and notification templates or create your own, and monitor your video to get alerts when issues arise.

To learn more, read the User guide. To get started, visit the workflow monitor console.