Getting started with the AWS Media Asset Preparation System (MAPS)
In the introductory blog post for the AWS Media Asset Preparation System (AWS MAPS), we provided an overview of how MAPS addresses pain points involving uploading media to AWS, organizing files/folders, file movement across AWS storage volumes, media preparation, permission assignment, search, filtering, and delivery of content. In this post, we provide an overview of how MAPS plays a pivotal role in the production process and provide a step-by-step guide that details how to deploy MAPS in your environment. Once MAPS is deployed, this guide helps you to start preparing media for your production pipeline.
Production use case
Imagine the day’s shoot just finished, and the footage for the day is uploaded into Amazon Simple Storage Service (Amazon S3) as part of the production’s content lake strategy. The media manager now has to sift through those assets, organizing them into logical groups, and prepping them for the editors to use the next day.
There are a few major challenges with this workflow:
- How does the Media Manager visualize the assets that live in an S3 bucket?
- How does the Media Manager easily move and organize assets, create new folders, and assign permissions based on the editor group?
- How do the editors get the content locally on their workstation so they can complete their editing tasks?
This is where MAPS comes in. MAPS provides an interface for Media Managers — and others in a production — to visualize their assets and intuitively organize them with a few clicks. The user interface removes the underlying technical aspects and allows the Media Manager to focus on the content and production requirements. Once the assets are organized, the Media Manager can add permissions and help prepare the editors for their upcoming edit session. When edit sessions are complete, the Media Manager can use MAPS to clean up and organize files promoting best practices across various storage volumes. These efforts result in storage savings and cost optimizations regardless of the size of your production.
In this section, we guide you through required steps to deploy MAPS in your AWS environment.
- An AWS Account
- A code repository (for example, GitHub)
Step 1: Deploy using AWS Amplify 1-Click
- Click on the following big orange button to deploy MAPS into your AWS account.
- You are then redirected to the AWS Amplify service page in the AWS Management Console.
- Choose Connect to GitHub. This creates a fork of the main MAPS repository in your own GitHub account.
- If you have not authorized AWS Amplify before, you are redirected to GitHub to provide authorization.
- After you authorize GitHub, you are automatically redirected to the Amplify console. If you do not already have an Amplify backend service role, choose Create New Role.
- Click Save and deploy.
- Once all three steps in the deployment process have been completed, choose Continue.
The deployment will take ~20 minutes to complete.
Configuration and setup
Step 2: Create an admin user
By default, all new users who register using the MAPS sign-in page are added to the editors user group.
- In the Amazon Cognito User Pools console, choose mapsuserpool-<your_env>.
- Choose Users and Groups then choose Create User.
- Fill in the required information and choose Create User.
- Choose the newly created user and choose Add to Group.
- From the drop-down, choose admin, and choose Add to Group.
Note: it is critical that your admin user is added to the admin group. Failure to complete this step will not allow you to interact with MAPS.
Step 3: Configure the MAPS interface
- In the AWS CloudFormation console, locate the amplify-maps-<your_env>-<your_app_id>-resMAPS
- Choose the Outputs tab, and copy the MediaBucketName
- Navigate to the AWS Amplify console and retrieve the production URL for your MAPS deployment.
- Open a web browser (Chrome is preferred) and open the URL.
- You are prompted with a sign-in page. Use the admin user you created in step 2.3.
- The first time you log in, you are prompted to change your password.
- Once logged in, you can see the MAPS interface.
- To configure the interface, choose the gear icon in the controls toolbar.
- On the settings page, enter the MediaBucketName and choose Save, then choose the back arrow to return to the main view. This configures the bucket for the entire MAPS deployment and only needs to be set once by an admin.
Note: if you receive an error, verify that there are no leading or trailing spaces in your bucket name that were included as part of the copy.
Step 4: Connect Amazon FSx for Windows File Server to MAPS
MAPS enables the movement of assets between storage volumes like Amazon S3 and Amazon FSx for Windows File Server. To take advantage of this feature, you need an Edit in the Cloud environment deployed in your AWS account. If you do not have an existing edit in the cloud deployment, you can deploy one from this GitHub repo.
- Retrieve your Amazon FSx for Windows File Server DNS name from the console.
- Navigate to the MAPSApiProcessing-<your_env> Lambda function.
- Navigate to the Configuration settings and choose Environment Variables from the left-hand side.
- Update the FSX_MOUNT environment variable with the Amazon FSx DNS name from step 4.1.
The backend currently requires Amazon FSx movement commands to be initiated from an EC2 edit instance with a public, Elastic IP address.
- Download the
MoveMedia.ps1script, located here, to the following directory on the EC2 instance:
C:\ProgramData\Amazon\EC2-Windows\Launch\Scripts. This script is completed by AWS Systems Manager (SSM) to move media files between Amazon S3 and Amazon FSx.
- Ensure that the SSM Agent is installed on the edit instance, and the instance profile has the AmazonEC2RoleforSSM policy attached.
Step 5: Using MAPS to prepare media
Once your MAPS deployment is configured, you can start preparing media for downstream workflows.
The controls toolbar provides a set of controls for interacting with the media. The following is an overview of the controls.
- Download – download an asset via an Amazon S3 presigned URL.
- Delete – delete an asset.
- Create New – upload a new asset.
- Create Folder – create a new folder.
- Settings – configure the S3 bucket used for MAPS.
MAPS user interface view
There are two view layouts in MAPS: table layout and card layout. The table layout provides a folder structure view, where the card layout allows you to visualize a thumbnail preview of your media in addition to technical metadata about the asset.
- To create a folder, choose the Create Folder button in the controls toolbar.
- Enter a name for the folder and choose Create Folder.
- To upload a file, choose the Create New button in the controls toolbar.
- Choose the Browse button and select the file you want to upload, then choose Upload.
- You can see a progress bar indicating what percentage of the file has been uploaded so far.
- Once the file has been successfully uploaded to Amazon S3, a new row appears in the MAPS table view with some information about the file.
- After the backend processes the file, the UI updates automatically to include the File Format This is an indication that the transcoding process was successful and your file is ready for preview and further operations.
- To preview media, double-click on the table row that corresponds to the media file you want to preview or choose the media’s thumbnail in the card layout.
- A media player opens and begins playing the proxy version of the media file. You can also see the technical metadata extracted from the asset.
- To organize your media into folders, select all the media you want to move into a folder by selecting the files in the table layout.
- Right click on one of the rows that is highlighted, and choose Move to Folder.
- Using the folder navigation tool, navigate to the folder you want to move your selected assets to and choose the Move
Note: it may take a few seconds to move your media depending on how many files you are moving and how large they are.
- To move media between Amazon S3 and Amazon FSx, select all the media you want to move by selecting the files in the table layout.
- Right click on one of the rows that is highlighted, and choose Copy to FSx.
- When you are ready to remove the files from Amazon FSx, select the files you want to remove from Amazon FSx, right click on one of the rows that is highlighted, and choose Remove from FSx.
- To confirm success of these operations, the status field of the assets should be S3_FSX when the media is in Amazon S3 and copied to Amazon FSx, and S3 when the media is just in Amazon S3.
Note: it may take a few seconds to copy your files to FSx for Windows File Server and remove them from the same network share.
Manage folder permissions
By default, there are two permission groups when you deploy MAPS: admin and editors. When a user creates a folder, the folder and all assets within it inherit that user’s group permissions. The admin group is automatically added to all new folders. Users in the admin group are the only ones who can update the permissions of a given folder. Adding more groups must be done in the Amazon Cognito console. For example, you can create separate permission groups for editors on different projects and then group the users into those user groups.
- To change the permissions of a folder, right-click on the folder and choose Permissions.
- Select or clear the groups to allow/deny permissions, then choose Save.
Note: you will not be able to remove the admin permission group fully.
To clean up the MAPS deployment, you must empty all resources from the three Amazon S3 buckets that were created. The bucket names should be similar to the following:
Once you have emptied the contents of the buckets, navigate to the AWS Amplify MAPS project. Select the Actions dropdown and choose Delete App. This deletes all resources associated with the MAPS deployment.
If you deployed the Edit in the Cloud solution, delete the associated CloudFormation stack to remove all resources.
In this post, we provided an overview of how MAPS fits in the production process and walked through a step by step for deploying MAPS into your own environment. For future code updates and feature releases, please visit the MAPS GitHub repository and star it to get notified of updates.
AWS has unmatched experience supporting supply chain from ingest, processing, creation, and distribution to and from the cloud—as well as more than 80 AWS Partners with dedicated tools for Media Supply Chain & Archive applications. Explore https://aws.amazon.com/media/supply-chain-archive/ to learn more.