Rendering with AWS portal in Thinkbox deadline

Post by Bryson Ding, AWS

INTRODUCTION

The new AWS Portal feature in Deadline 10 supports many content creation applications, including Autodesk Maya and Autodesk Arnold. This feature allows you to use the distributed computing power of Amazon EC2 Spot Instances to optimize your Maya/Arnold workflow, and in this post, we will outline the necessary steps required to render with Maya/Arnold on AWS.

Note that this process is very similar to other products supported by AWS Portal, so if you plan on using other content creation applications, most of this post is still applicable.

REQUIREMENTS

This post assumes you already have the following prerequisites:

• You have set up the AWS Portal feature in Deadline 10.
• You know which AWS instance types you plan to use (read Part 1 and Part 2 of the blog entry for choosing your AWS instance types).
• You have purchased Maya, Arnold, and Deadline Render Time from the Thinkbox Marketplace and configured Deadline to use Deadline Usage Based Licensing and 3rd Party Usage Based Licensing (including running the License Forwarder).
• You have installed the Maya integrated submitter.

Use the links in the items above to learn more about these prerequisites.

SETTING UP JOB LIMITS FOR USAGE BASED LICENSING

Before you can start rendering your Maya/Arnold jobs with AWS Portal, you first need to create Limits to handle Usage Based Licensing. This is an important step, because it ensures your AWS Portal render nodes will be able to use the render time you purchased from the Marketplace.

Using the Limits panel in the Deadline Monitor, create a Limit for both Arnold and Maya and select the appropriate License Application for each of those Limits. For this example, set the following in your Limit configuration for each 3rd party product:

• Name: Set this to “arnold” and “maya”, respectively.
• Limit: Because we are only using Usage Based Licensing, set this to “0”.
• Limit Overage: To ensure all AWS render nodes can use Usage Based Licensing, enable the “Unlimited Overage” option.
• Use Based Third Party Licensing: Enable this option, and then set the License Application to “arnold” and “maya”, respectively. This ensures each Limit uses the appropriate 3rd party render time.

For any setting not listed above, you can leave them as their default setting.

SYNCING RENDER ASSETS TO THE CLOUD

To ensure your AWS Portal render nodes can access the render assets required to render your Maya/Arnold jobs (scene file, textures, etc), you need to configure AWS Portal to synchronize your assets. This is covered in the AWS Portal Setup documentation that’s referenced in the prerequisites above.

In this example, I’ll be using C:\Users\brysding as the on-premises Asset directory, and I’ll be placing my Maya/Arnold project in this folder. I’ll also be sending my rendered images to this folder so that they are synchronized back when the AWS Portal render nodes finish them. More on this later, but for now, I’ve set this directory in the AWS Portal Settings.

Now when you submit your job, any required assets are transferred from this directory to an Amazon S3 but in a pre-caching stage using the Asset Transfer System. After this process is complete, the AWS Portal render nodes will be able to access the assets. In addition, anything rendered to this location will be automatically synced back into this directory.

CREATING INFRASTRUCTURE AND RENDER NODES

In the Monitor, create an Infrastructure with the + button in the AWS Portal Panel.

It may take a few seconds for the Infrastructure to show up in the panel. Since the Infrastructure creates all the initial AWS resources required to connect AWS render nodes to your render farm, it may take up to 5 minutes for Infrastructure creation to complete. After the Infrastructure has reached a CREATE COMPLETE state, you can now start render nodes by right clicking on the Infrastructure.

In the Spot Fleet Configuration window, you can set the maximum price for each render node, as well as the AWS instance types to use. Make sure that the Maya 2017 & Arnold default AMI is selected, and then press the Launch button. The Spot Fleet Request for the render nodes will then show up in the Spot Fleets panel.

SUBMITTING A RENDER JOB TO DEADLINE

With the Limits in place, and the AWS Portal Infrastructure and render nodes up and running, you can now submit a Maya/Arnold job to Deadline. If the Maya integrated submitter has been installed, the Deadline shelf tab will be available in Maya. Select the green icon to bring up the submission window.

Set the following configurations within this submission window:

• Limit Groups: The previously created Limit groups in Deadline can now be selected in this window. Select the “arnold” and “maya” Limits by hitting the … button beside the Limit Groups field.
• Project Path: As mentioned earlier, I have created an Assets folder in C:\Users\brysding and stored my project within this folder. Since the AWS Portal Asset Service will automatically transfer asset files specified in this directory to the render nodes running on AWS, this project path must exist in the asset folder specified in the AWS Portal Settings.
• Output Path: The output render will automatically be transferred to this output path when the AWS render nodes finish rendering. Similar to the project path, this directory should be within the assets folder.
• Precache Assets for AWS: This setting ensures that all assets (projects files, source images, etc.) are transferred to AWS before a render begins.

After the job has been configured, press the Submit Job button.

THE COMPUTING POWER OF THE CLOUD

After submitting your job to Deadline, it will show up in the Jobs panel in the Monitor. Selecting this job will allow you to see the tasks associated with this job in the Tasks panel. Each task represents a unit of work that each AWS render node can pick up for rendering (typically one task per frame).

Note that because the job’s assets are being precached in AWS, the render nodes will wait for all assets to be transferred to the cloud before beginning the render job. When the job is complete, the AWS Portal Asset Service will automatically transfer the output render to the output path previously specified in the Maya Job Submission window.

CONCLUSION

In Deadline 10, the new AWS Portal simplifies the process of launching infrastructure and rendering in the cloud. This Maya/Arnold example just highlights one of the many possibilities, and you can use this as a reference for rendering with other content creation applications in the cloud. Use the power of the cloud through Deadline to meet your content creation needs!

Further information regarding the usage of Maya and Arnold in Deadline can be found in the Maya Plugin documentation.