How to integrate Audible Magic with Amazon IVS to identify copyrighted content

This blog was co-authored by Dan Curry, Sr. Product Manager, Content Identification at Audible Magic, and Andrew DeWitt, Director, Product Management at Audible Magic.

Introduction

Amazon Interactive Video Service (Amazon IVS) from Amazon Web Services (AWS) is a managed live streaming service that enables businesses to build interactive experiences and applications to delight their customers with low-latency live video streams.

Customers have leveraged Amazon IVS and built user generated content (UGC) platforms for use cases ranging from live shopping to fitness, education, gaming, and live event experiences over live video. As users engage with these platforms built on top of Amazon IVS, there is the potential for both bad actors and well-intentioned users to stream copyrighted content. This may pose challenges for customers that need to receive and process takedown requests. Even with proper licensing, the use of copyrighted content may require attribution and reporting. In both cases, the ability to properly identify copyrighted content becomes important.

How Audible Magic Technology works

Audible Magic’s Emmy-winning automatic content recognition (ACR) technology presents a possible solution. Leveraging longstanding first-party relationships with thousands of record labels, music publishers, and other rights holders, the integration of Audible Magic’s ACR technology within Amazon Interactive Video Service may help to provide a protective shield for platforms and rights holders, while helping to establish a trustworthy environment for creators and consumers. Audible Magic’s solution enables UGC platforms to actively monitor and filter out unauthorized content while enabling the recognition and reporting of licensed content. It is designed to help reduce DMCA takedown notice disruptions and ensure accurate license administration.

Audible Magic’s ACR technology is tuned to identify copyrighted content embedded in user-generated media. When media is uploaded to a platform, the platform sends digital fingerprints of the media to Audible Magic’s information service, where they are compared to reference fingerprints submitted directly to Audible Magic by content owners. The service then returns results indicating a match or no-match condition. If matching content is found, the content owner’s business rules are returned by Audible Magic’s information service. Business rules express the content owner’s wishes to either block or allow content in a media sharing environment.

The next section of this blog post describes how to integrate Audible Magic’s ACR technology with Amazon IVS to enforce copyrights in live-streamed content.

IVS and Audible Magic end-to-end solution architecture

The simplest of the multiple ways to integrate Audible Magic’s ACR technology service with Amazon IVS.

When a stream begins, it generates a “stream started event” on the event bridge, which triggers an event rule. This trigger pushes a message containing stream information to a SQS queue, where an ECS service reads the message and starts scanning the stream with Audible Magic’s Toolkit. Whenever copyrighted material is identified, a JSON response file is generated with information about the match and is saved to an Amazon S3 bucket. Additionally, an email notification is sent pointing to the location of the JSON response file.

Handling toolkit responses

There are many ways you can use the responses from Audible Magic to take action on the detection of copyrighted content. For example: 1) sending notifications to an AWS Lambda function to display the currently playing song in an overlay on the screen, 2) muting portions of the stream that contain copyrighted content, or 3) automatically changing the song. Ultimately, Audible Magic’s Toolkit responses can extend out through a variety of applications and enable businesses to take actions that align with their content policies.

Email notifications can also help keep track of identifications that occurred during a stream. Further analysis of these identifications may aid with: 1) policies to prevent streaming copyrighted material in the future, 2) reporting on music usage, or 3) making informed edits to a video before VOD publishing.

Integrating Audible Magic – detailed steps

This section provides a walk through the steps to achieve the integration outlined in the end-to-end architecture diagram. If you would like to try these steps in your account, use the step-by-step guide with code samples linked here: AMToolkit identifyStream on AWS ECS

Integration prerequisites

Before any integration can take place, there are a few prerequisites that will allow for the entire system to work as intended. You will need:

• An Amazon IVS channel with streaming software setup. Refer to the Getting Started with Amazon IVS guide to learn how to create a channel and set up a live stream.
• An Audible Magic Toolkit package that contains the software needed to make identifications. You will be provided with an Audible Magic Toolkit package after you have signed up for Audible Magic Identification services.
• An Audible Magic Toolkit configuration file that informs the Audible Magic Toolkit software on how to perform according to your specifications. You will be provided with an Audible Magic Toolkit configuration file after you have signed up for Audible Magic Identification services.

To sign up for Audible Magic Identification services, please contact Audible Magic through its Contact page.

Once you have these prerequisites in place, you can proceed through the rest of this blog post to set up stream identification with Audible Magic. Let’s get started with the integration steps.

1)    Create an Amazon S3 Bucket

Create an S3 bucket to hold the responses delivered by the Audible Magic toolkit.

A JSON response file is generated with information about matches found, and is saved to this S3 bucket.

2)    Create an Amazon SQS queue

Create an Amazon SQS queue for messages about newly started streams. When a stream begins, it generates a “stream started event” on Amazon EventBridge, which triggers an Event Rule. A message containing stream information gets pushed to an SQS queue, where Amazon Elastic Container Service reads the message and starts scanning the stream with Audible Magic’s Toolkit.

3)    Create an EventBridge event rule

Create an EventBridge Event Rule to generate a message when a new stream starts. This message will be sent to the SQS Queue.

4)    Attach EventBridge event rule to SQS queue

With the SQS Queue and EventBridge Event Rule created, connect them so the EventBridge can send messages to the SQS Queue whenever an Event Rule is triggered.

5)    Create toolkit wrapper application

An application needs to be written that interfaces with Audible Magic’s identifyStream binary, which is embedded within the Audible Magic Toolkit. This application will perform the following while parsing live-streamed content:

1. Pull message from SQS queue
   1. Parse for channel-arn
   2. Parse for stream-id
2. Retrieve stream playback URL using channel-arn
3. Start scanning stream with identifyStream
4. Monitor for Identification Response files
5. Push Identification Response files to the S3 Bucket

Pull message from SQS queue

Messages about new streams will be pushed to the SQS queue created earlier. The Toolkit wrapper application will need to pull, parse, and delete these messages while taking the appropriate action on each.

Retrieve stream playback URL using channel ARN

After pulling the message from the queue, parse the “channel_arn” and use that to obtain the stream playback URL.

Start scanning stream with identifyStream

At this point, all necessary pieces to scan the live stream for copyrighted content using identifyStream are configured. Next, create a function that will run identifyStream using the appropriate parameters, and place the results in an accessible location.

Monitor for identification responses

Since identifyStream responds with matches in real-time during a broadcast, it’s important to monitor for results as they are generated. Identification results are saved locally; next, write a function that will monitor the directory for creation of new files. Files are saved in “/app/{stream_id}” directory.

Push Identification response files to S3 bucket

When new files (identification match results) are detected in the monitored directory, push these files to the S3 Bucket created earlier.

6)  Create Docker image of the toolkit wrapper application

With the application now written, create a Docker image using Dockerfile and Docker command.

7)   Upload the toolkit wrapper application Docker image to AWS Elastic Container Registry (ECR)

Create the Elastic Container Registry (ECR) repository in which the Docker image will reside.

Prior to pushing the Docker image to ECR, verify that the ECR repository can be connected via Docker using the username “AWS”. Then, tag your local Docker image with the appropriate AWS ECR location and push it to the ECR repository. Ensure the presence of the Docker image in ECR by listing the repositories with the appropriate name Ex. “am-stream-identification”.

8)   Create an ECS task

Create an ECS task that defines the criteria needed to run the Docker image. Ensure the task definition contains information about: 1) the SQS Queue from which the application will read messages, 2) the S3 Bucket to which the application will save identification results, and 3) the image that the Docker container will use as its base image.

9)   Create ECS service

Create an ECS service that will launch the ECS task. ECS service is named as “am-stream-identification-service”. The ‘desired count’ of this service is set to 1. This will automatically start a single instance of the am-stream-identification-task.

10)  Adding email notification integration

Without having to check the S3 bucket for new JSON response files, you can set up email notifications using AWS Simple Notification Service (SNS) to email you whenever an identification match has is made. The email notification feature is optional and can be skipped.

Create a Simple Notification Service topic

The first step in setting up email notifications via SNS is to create an SNS topic. Allow S3 to publish to the SNS Topic. Once the SNS topic can receive notifications from the S3 bucket, add the push notification capability to the S3 bucket that is triggered whenever a new JSON response file is created in the identifications bucket.

Link email address to be sent notifications

In order to receive an email notification via SNS, link the email address that should receive the email notifications. Now, whenever a JSON response file is saved to the S3 bucket, an email message is sent (formatted in JSON) with information about the JSON file, including the location (key) and size.

11)  Testing the Integration

With the ECS Service fully set up and running, you can test the Audible Magic identification service against a live stream. Start a stream to your Amazon IVS channel. A message should automatically be sent to the SQS queue that was set up previously. The Audible Magic identification service will scan the stream as long as it is active. Whenever the service finds a match to copyrighted content, a JSON response file with the match results will be produced and saved to the S3 bucket that was created earlier in this guide, within a folder named after the stream-id of the active stream.

Conclusion

In this article, we demonstrate how to set up a fully functioning automatic content recognition solution in just a few simple steps using Audible Magic’s ACR technology integrated with Amazon IVS. If you’re interested in validating this solution using a proof of concept, please visit https://www.audiblemagic.com/contact/.

About Audible Magic

For more than 20 years, Audible Magic has innovated B2B solutions in content identification (both music and TV/Film), music catalog fulfillment, and rights administration. Audible Magic powers billions of monthly transactions to serve customers all around the world. Its music identification services are used to help reduce the number of DMCA takedown notices and administer licenses for user generated content (UGC). Its catalog fulfillment services are a robust set of APIs that enable you to easily integrate and customize music experiences for your app. Audible Magic is a trusted intermediary and approved vendor between major platforms and rights holders (including music labels, distributors, publishers, collectives, and TV/Film studios). For more information, visit https://www.audiblemagic.com/.