Near real-time notifications for AWS Media Services

A real-time live video workflow doesn’t offer second chances. The stakes are high, especially during marquee events that gather millions of viewers, so it is important to be able to react quickly and effectively to any potential issues affecting a live stream. This blog post details how to set up near real-time notifications on system state change events for media workflows comprising AWS Media Services. Event notifications emit in near real-time and can help pinpoint the location of a potential problem in the video processing chain, reducing time to resolution. Amazon EventBridge, a serverless event bus that makes it easier to build event-driven applications at scale, allows you to create event-driven workflows and respond automatically to system state changes. When used with Amazon Simple Notification Service (Amazon SNS), a fully managed messaging service, you can set up near real-time notifications on system events so your team can promptly take action on potential QoE-affecting complications.

How it works

Amazon EventBridge

Almost all AWS services, including AWS Media Services, generate events, which indicate a change in the state of your application or services. When emitted, these events are received by an event bus within the AWS EventBridge service. Within EventBridge, you can create rules that filter the events coming into your event bus and determine the route to selected targets. Amazon SNS can be used as a target, which is demonstrated in this guide, to send notifications to Amazon SNS subscribers. For a comprehensive list of targets available in EventBridge, visit this link.

Amazon Simple Notification Service

Amazon SNS is a fully managed publisher and subscriber service for publishing and processing messages in AWS. Within Amazon SNS, there are two main concepts, topics and subscriptions. Topics are logical entry points that act as a communication channel to one or multiple subscriptions, such as AWS Lambda, which lets you run code without thinking about servers or clusters, and Amazon Simple Queue Service (Amazon SQS), a fully managed message queuing service, as well as HTTP/S or an email address. In this case, the Amazon EventBridge service will publish the service event message to an Amazon SNS topic, and the subscriptions (email, SMS, or others) will receive the event for processing. For a comprehensive list of endpoints available in Amazon SNS, visit this link.

How to set up automatic notifications for AWS Media Services events

As of the publishing of this blog, the following AWS Media Services send state-change event messages to Amazon EventBridge.

• AWS Elemental MediaConnect
• AWS Elemental MediaLive
• AWS Elemental MediaConvert
• AWS Elemental MediaPackage
• AWS Elemental MediaStore
• Amazon Interactive Video Service

Be sure to visit the service documentation links listed previously to understand what events are emitted by each service and what the event messages indicate. To avoid extraneous message processing, determine which events are important to you and worth notifying on. The following example demonstrates how to set up event-based notifications to an email address for AWS Elemental MediaConnect flow-state change events. This event is published when a flow’s state has changed from or to any of the following statuses: standby, active, updating, deleting, starting, stopping, or error. The same steps can be replicated for each service, desired event, and different Amazon SNS subscriptions. Depending on the urgency of the event, you will want to configure your endpoints accordingly. For example, high-priority events can be sent to a paging service, like Pager Duty, so your team can immediately respond to an incident. Here’s a guide on how to seamlessly integrate AWS with Pager Duty. A thorough notification system will have a combination of Amazon SNS subscriptions, such as a paging service, email, SMS, mobile push notifications, or Slack.

First, create an Amazon SNS topic and Amazon SNS subscription(s):

1. From the AWS console, navigate to the Amazon SNS service and choose “Topics” from the navigation menu
2. Choose “Create Topic”
3. Provide the following information:
   • Type: choose “Standard”
   • Name: provides a name for the topic
   • Encryption: (optional) facilitate server-side encryption
   • Access policy: (optional) this policy defines who can access your topic
   • Delivery retry policy (HTTP/S): (optional) This policy defines how Amazon SNS retries failed deliveries to HTTP/S endpoints
   • Delivery status logging: (optional) These settings configure the logging of message delivery status to logs on Amazon CloudWatch, a monitoring and observability service
   • Tags: (optional)
   • Choose “Create”
   • 
4. Choose “Subscriptions” from the navigation menu
5. Provide the following information:
   • Topic ARN: ARN of the topic you created
   • Protocol: email
   • Endpoint: email address to receive notification email
   • Subscription filter policy: (optional)
   • Redrive policy (dead-letter queue): (optional)
   • Note: For email subscriptions, be sure to accept the confirmation email after creation
6. To publish messages to Amazon Chime (a communications service), Slack, or Microsoft Teams, follow the instructions outlined in this link.

Second, define your EventBridge rule:

1. From the AWS console, navigate to the Amazon EventBridge service and choose “Rules” from the navigation menu
2. Choose “Create Rule”
3. Provide the following information:
   • Name: name of the rule
   • Description (optional): brief description of the rule
   • Event bus: By default, all events are emitted to the default event bus. Optionally, create a new event bus to emit events to.
   • Rule type: For this example, I use an event pattern to match emitted service events instead of schedule, which is a rule that runs on a schedule.

4. Choose “Next” to configure an event pattern

• Event source: AWS services
• Event pattern:
  • Event source:MediaConnect
  • Event type:MediaConnect flow status changed

Keep in mind, this event pattern will send email notifications for all MediaConnect flows and all MediaConnect status changes. Optionally, create an advanced EventBridge rule for a more granular filter. For example, create a rule that encompasses only a subset of your most critical MediaConnect flows and accept only the events when the status changes to “ERROR”. This blog post provides an example of how to use the content filtering syntax in Amazon EventBridge.

5. Choose “Next” to configure an event target

• Target Types: AWS service
• Target: Amazon SNS topic
• Topic: Choose your Amazon SNS topic

6. Choose “Next” to optionally configure tags

7. Choose “Next” to review and create your Amazon EventBridge rule

Conclusion

The capabilities of Amazon EventBridge and Amazon SNS go far beyond email notifications. These services also help you to easily create decoupled event-driven architectures. Check out this blog post for a great example of how you can use events on Amazon Simple Storage Service (Amazon S3)—an object storage service built to retrieve any amount of data from anywhere—and Amazon SNS to create a serverless watch folder workflow using AWS Elemental MediaConvert. You can take your media services monitoring framework a step further by implementing Media Services Application Mapper (MSAM), which allows you the ability to visualize the real-time status of linear video services.

In this post, I showed you how to set up publisher and subscriber notifications for application-to-application (A2A) and application-to-person (A2P) communication. Receiving event-based notifications in near real time can drastically reduce monitoring overhead and increase observability into your video workflow on AWS. By following the steps above, you can quickly notify your team about important MediaConnect state changes. Monitoring a video pipeline can be overwhelming, but AWS services such as Amazon CloudWatch, Amazon EventBridge, and Amazon SNS provide flexibility, allowing you to quickly and appropriately respond to potential issues.