Getting started with the AWS Media Asset Preparation System (MAPS)

Introduction

In the introductory blog post for the AWS Media Asset Preparation System (AWS MAPS), we provided an overview of how MAPS addresses pain points involving uploading media to AWS, organizing files/folders, file movement across AWS storage volumes, media preparation, permission assignment, search, filtering, and delivery of content. In this post, we provide an overview of how MAPS plays a pivotal role in the production process and provide a step-by-step guide that details how to deploy MAPS in your environment. Once MAPS is deployed, this guide helps you to start preparing media for your production pipeline.

Production use case

Imagine the day’s shoot just finished, and the footage for the day is uploaded into Amazon Simple Storage Service (Amazon S3) as part of the production’s content lake strategy. The media manager now has to sift through those assets, organizing them into logical groups, and prepping them for the editors to use the next day.

There are a few major challenges with this workflow:

How does the Media Manager visualize the assets that live in an S3 bucket?
How does the Media Manager easily move and organize assets, create new folders, and assign permissions based on the editor group?
How do the editors get the content locally on their workstation so they can complete their editing tasks?

This is where MAPS comes in. MAPS provides an interface for Media Managers — and others in a production — to visualize their assets and intuitively organize them with a few clicks. The user interface removes the underlying technical aspects and allows the Media Manager to focus on the content and production requirements. Once the assets are organized, the Media Manager can add permissions and help prepare the editors for their upcoming edit session. When edit sessions are complete, the Media Manager can use MAPS to clean up and organize files promoting best practices across various storage volumes. These efforts result in storage savings and cost optimizations regardless of the size of your production.

Step-by-step deployment

In this section, we guide you through required steps to deploy MAPS in your AWS environment.

Requirements

An AWS Account
A code repository (for example, GitHub)

Step 1: Deploy using AWS Amplify 1-Click

Click on the following big orange button to deploy MAPS into your AWS account.

You are then redirected to the AWS Amplify service page in the AWS Management Console.

AWS Amplify Console page for connecting to a GitHub repository.

Choose Connect to GitHub. This creates a fork of the main MAPS repository in your own GitHub account.
If you have not authorized AWS Amplify before, you are redirected to GitHub to provide authorization.

In order to connect AWS Amplify and GitHub, you must authorization the Amplify console.

After you authorize GitHub, you are automatically redirected to the Amplify console. If you do not already have an Amplify backend service role, choose Create New Role.

In the Amplify console, you provide a name for the deployment and a role so Amplify can deploy resources on your behalf.

Click Save and deploy.
Once all three steps in the deployment process have been completed, choose Continue.

When you click Save and Deploy, Amplify will fork the repository, create your Amplify console application, and kick-off the build process for the backend.

The deployment will take ~20 minutes to complete.

Configuration and setup

Step 2: Create an admin user

By default, all new users who register using the MAPS sign-in page are added to the editors user group.

In the Amazon Cognito User Pools console, choose mapsuserpool-<your_env>.

In the Amazon Cognito console, this is the name of the generated User Pool.

Choose Users and Groups then choose Create User.

In the Cognito User Pool, select Users and Groups from the General Settings side bar.

Fill in the required information and choose Create User.

When you create a user in Cognito, you need to specify a username and temporary password for the new user.

Choose the newly created user and choose Add to Group.

Once a user is created, you can add the user to a group by clicking the Add to group button.

From the drop-down, choose admin, and choose Add to Group.

Add the user to the admin group using the drop down. This step is critical or else your access to MAPS will be restricted.

Note: it is critical that your admin user is added to the admin group. Failure to complete this step will not allow you to interact with MAPS.

Step 3: Configure the MAPS interface

In the AWS CloudFormation console, locate the amplify-maps-<your_env>-<your_app_id>-resMAPS

In CloudFormation, you can see your stacks and nested stacks. We need to locate the resMAPS stack.

Choose the Outputs tab, and copy the MediaBucketName

Within the resMAPS stack, the Outputs tab has detailed output from the stack including the Media Bucket Name which we need to configure to MAPS user interface.

Navigate to the AWS Amplify console and retrieve the production URL for your MAPS deployment.

When the MAPS deployment has completed, all stages of the process will turn green with a check mark. We can also retrieve the unique URL to access the MAPS user interface.

Open a web browser (Chrome is preferred) and open the URL.
You are prompted with a sign-in page. Use the admin user you created in step 2.3.

When you first open the MAPS URL, you are prompted to log in using your Cognito credentials.

The first time you log in, you are prompted to change your password.

On first log in, you are shown a screen to reset your password.

Once logged in, you can see the MAPS interface.

After you log in, the MAPS user interface is displayed. The interface is not configured yet.

To configure the interface, choose the gear icon in the controls toolbar.
On the settings page, enter the MediaBucketName and choose Save, then choose the back arrow to return to the main view. This configures the bucket for the entire MAPS deployment and only needs to be set once by an admin.

The Settings page in the MAPS UI. Enter the Media Bucket name and choose Save Settings.

Note: if you receive an error, verify that there are no leading or trailing spaces in your bucket name that were included as part of the copy.

Step 4: Connect Amazon FSx for Windows File Server to MAPS

MAPS enables the movement of assets between storage volumes like Amazon S3 and Amazon FSx for Windows File Server. To take advantage of this feature, you need an Edit in the Cloud environment deployed in your AWS account. If you do not have an existing edit in the cloud deployment, you can deploy one from this GitHub repo.

Retrieve your Amazon FSx for Windows File Server DNS name from the console.

In the Amazon FSX console, in the network and security tab. We need to retrieve the DNS name for the file share.

Navigate to the MAPSApiProcessing-<your_env> Lambda function.

In the AWS Lambda console, we search for the MAPS API Processing function.

Navigate to the Configuration settings and choose Environment Variables from the left-hand side.

In the Lambda function configuration tab, we edit the FSX Mount environment variable.

Update the FSX_MOUNT environment variable with the Amazon FSx DNS name from step 4.1.

Create a Lambda environment variable FSX_MOUNT and assign its value to the FSX DNS name.

The backend currently requires Amazon FSx movement commands to be initiated from an EC2 edit instance with a public, Elastic IP address.

Download the MoveMedia.ps1 script, located here, to the following directory on the EC2 instance: C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts. This script is completed by AWS Systems Manager (SSM) to move media files between Amazon S3 and Amazon FSx.
Ensure that the SSM Agent is installed on the edit instance, and the instance profile has the AmazonEC2RoleforSSM policy attached.

Step 5: Using MAPS to prepare media

Once your MAPS deployment is configured, you can start preparing media for downstream workflows.

Controls toolbar

The controls toolbar provides a set of controls for interacting with the media. The following is an overview of the controls.

The MAPS UI toolbar with options to download, delete, create new, create new folder, and access the settings.

Download – download an asset via an Amazon S3 presigned URL.
Delete – delete an asset.
Create New – upload a new asset.
Create Folder – create a new folder.
Settings – configure the S3 bucket used for MAPS.

MAPS user interface view

There are two view layouts in MAPS: table layout and card layout. The table layout provides a folder structure view, where the card layout allows you to visualize a thumbnail preview of your media in addition to technical metadata about the asset.

This is the MAPS table view which displays all assets in a table.

This is the MAPS card layout. This shows media assets with a thumbnail preview.

Creating folders

To create a folder, choose the Create Folder button in the controls toolbar.
Enter a name for the folder and choose Create Folder.

This is the create folder window. You can enter a folder name and choose create folder.

Upload media

To upload a file, choose the Create New button in the controls toolbar.
Choose the Browse button and select the file you want to upload, then choose Upload.

This is the upload media window. You can choose Browse and select the media you want to upload then choose upload to start the upload process.

You can see a progress bar indicating what percentage of the file has been uploaded so far.

This is the upload progress bar that describes the status of the upload.

Once the file has been successfully uploaded to Amazon S3, a new row appears in the MAPS table view with some information about the file.

This is the MAPS tables view for an asset that has been uploaded but has not been fully processed yet.

After the backend processes the file, the UI updates automatically to include the File Format This is an indication that the transcoding process was successful and your file is ready for preview and further operations.

This is the MAPS table view for an asset that has been fully processed.

Preview media

To preview media, double-click on the table row that corresponds to the media file you want to preview or choose the media’s thumbnail in the card layout.
A media player opens and begins playing the proxy version of the media file. You can also see the technical metadata extracted from the asset.

The media proxy view shows the user a media player which can play the asset. The view also shows detailed technical metadata about the asset.

Organize media

To organize your media into folders, select all the media you want to move into a folder by selecting the files in the table layout.

In the MAPS table view, users can select multiple assets by selecting the checkbox in each row.

Right click on one of the rows that is highlighted, and choose Move to Folder.

When a user right clicks on any of the selected rows, the options menu appears. The options include Move to folder, Copy to FSx, and Remove from FSx.

Using the folder navigation tool, navigate to the folder you want to move your selected assets to and choose the Move

When you choose Move to folder, another window opens allowing you to select which folder you want to move the assets to.

Note: it may take a few seconds to move your media depending on how many files you are moving and how large they are.

Move media

To move media between Amazon S3 and Amazon FSx, select all the media you want to move by selecting the files in the table layout.

In the MAPS table view, users can select multiple assets by selecting the checkbox in each row.

Right click on one of the rows that is highlighted, and choose Copy to FSx.

When a user right clicks on any of the selected rows, the options menu appears. The options include Move to folder, Copy to FSx, and Remove from FSx.

When you are ready to remove the files from Amazon FSx, select the files you want to remove from Amazon FSx, right click on one of the rows that is highlighted, and choose Remove from FSx.
To confirm success of these operations, the status field of the assets should be S3_FSX when the media is in Amazon S3 and copied to Amazon FSx, and S3 when the media is just in Amazon S3.

When you copy assets to FSX, their status will change in the MAPS UI to indicate that they are currently copied to FSX.

Note: it may take a few seconds to copy your files to FSx for Windows File Server and remove them from the same network share.

Manage folder permissions

By default, there are two permission groups when you deploy MAPS: admin and editors. When a user creates a folder, the folder and all assets within it inherit that user’s group permissions. The admin group is automatically added to all new folders. Users in the admin group are the only ones who can update the permissions of a given folder. Adding more groups must be done in the Amazon Cognito console. For example, you can create separate permission groups for editors on different projects and then group the users into those user groups.

To change the permissions of a folder, right-click on the folder and choose Permissions.

When you right click on a folder, a permissions drop down opens.

Select or clear the groups to allow/deny permissions, then choose Save.

Choosing the permissions option lets you change which Cognito groups have access to the folder.

Note: you will not be able to remove the admin permission group fully.

Clean up

To clean up the MAPS deployment, you must empty all resources from the three Amazon S3 buckets that were created. The bucket names should be similar to the following:

maps-media-<region>-<aws-account-id>-<unique-id>
maps-metadata-<region>-<aws-account-id>-<unique-id>
sm-output-<region>-<aws-account-id>-<unique-id>

Once you have emptied the contents of the buckets, navigate to the AWS Amplify MAPS project. Select the Actions dropdown and choose Delete App. This deletes all resources associated with the MAPS deployment.

If you deployed the Edit in the Cloud solution, delete the associated CloudFormation stack to remove all resources.

Conclusion

In this post, we provided an overview of how MAPS fits in the production process and walked through a step by step for deploying MAPS into your own environment. For future code updates and feature releases, please visit the MAPS GitHub repository and star it to get notified of updates.

AWS has unmatched experience supporting supply chain from ingest, processing, creation, and distribution to and from the cloud—as well as more than 80 AWS Partners with dedicated tools for Media Supply Chain & Archive applications. Explore https://aws.amazon.com/media/supply-chain-archive/ to learn more.

AWS for M&E Blog