AWS for M&E Blog

Create a tile rendering job with modifications for AWS Deadline Cloud

Rendering in tiles can reduce the memory footprint when rendering large images for outdoor advertising or high-resolution entertainment. Tile rendering is a workflow that aims to optimize render resources by splitting large images into evenly sized smaller regions to be rendered in parallel, minimizing rendering time and memory usage. Using the powerful tools Amazon Web Services (AWS) Deadline Cloud and Open Job Description (OpenJD), you can create custom rendering workflows such as tile rendering.

Diagram showing a tile rendering and assembly workflow. Left shows a render scene, middle shows the image rendered into 4 tiles, right shows the assembled image.

Figure 1: The tile rendering and assembly workflow.

OpenJD is an open specification for job templates that can support the structure of a tile rendering job by specifying a set of steps with parameters and dependencies. Deadline Cloud has plugins for certain digital content creation (DCC) applications available as open-source python packages. Within those packages, modification to the render handler, also known as the adaptor, can be made to a local copy of the source code to add configuration for tile rendering.

I’ll walk through how you can customize a job template and adaptor for Deadline Cloud to utilize tile rendering. Utilizing the example in this blog, you will use Autodesk Maya, with Arnold, to render a simple scene into tiled regions, and assemble them together with FFmpeg. This walkthrough should provide a basis on how to achieve the same functionality with other software as well.

Prerequisites

This walkthrough requires the following:

Setup a Development Version of the Maya Adaptor

Use Git to git clone the GitHub repository deadline-cloud-for-maya onto your computer. Follow the submitter and adaptor development workflow directions under DEVELOPMENT.md, adjusting the commands as needed for your OS and respective file locations.

Customizing the Job Template to Utilize Additional Job Parameters

Additional information is needed from the job template to render the image as tiles such that each tile is a task specified with task parameters. This job bundle example uses a customized job template for a Maya scene, which can be found under the deadline-cloud-samples GitHub alongside other sample job bundles. The modified template.yaml file has additional job parameters named “NumXTiles” and “NumYTiles” under “parameterDefinitions”. These will be where you can input the number of horizontal and vertical tiles you want to render.

Under the rendering step titled “tile render”, additional task parameters are defined as “TileNumberX” and “TileNumberY” under the step’s “taskParameterDefinitions”. Note that the inclusive range starts from 1 and uses the job parameter value of “NumXTiles” and “NumYTiles” as the maximum value. Task parameters will create a task for every combination of values within the range of each task parameter. Their value will also tell us which tile we are supposed to be rendering for the task.

These new variables are added to the “runData” embedded file using the parameter names “numXTiles”, “numYTiles”, “tileNumX”, and “tileNumY”. These four values provide enough information to calculate the dimensions of each tile that will be rendered. The “output_file_prefix” parameter is also added to the “runData” so that each tile prefix can be added to the original file prefix.

Modify the Render Handler to Add the Functionality of Rendering a Specified Tile

The computation of the tile rendering properties needed by the renderer are done in the render handler python file. The added processes include calculating the pixel dimensions for the specific tile being rendered, using these values to set Arnold’s render settings, and configuring the output name for each tile. For Arnold, this file is found in the deadline-cloud-for-maya repository under src/deadline/maya_adaptor/MayaClient/render_handlers/arnold_handler.py. Open this file in a text editor of your choice.

The configurations are added through the following code block, which you need to add to the start_render function in arnold_handler.py after the width and height of the image are verified to be in the render arguments. The following code can also be found in the README.me for this job bundle sample with more detailed comments that explain the code.

numXTiles = data.get("numXTiles")
numYTiles = data.get("numYTiles")

if (numXTiles is not None) and (numYTiles is not None):
    if (not isinstance(numXTiles, int)) or (not isinstance(numYTiles, int)):
        raise RuntimeError(
            "numXTiles and numYTiles variables from run-data must be integers"
        )

    tileNumX = data.get("tileNumX")
    tileNumY = data.get("tileNumY")

    if (not isinstance(tileNumX, int)) or (not isinstance(tileNumY, int)):
        raise RuntimeError("tileNumX and tileNumY variables from run-data must be integers")

    deltaX, widthRemainder = divmod(self.render_kwargs["width"], numXTiles)
    deltaY, heightRemainder = divmod(self.render_kwargs["height"], numYTiles)

    minX = deltaX * (tileNumX - 1)
    maxX = (deltaX * tileNumX) - 1
    minY = deltaY * (tileNumY - 1)
    maxY = (deltaY * tileNumY) - 1

    if tileNumX == numXTiles:
        maxX += widthRemainder
    if tileNumY == numYTiles:
        maxY += heightRemainder

    maya.cmds.setAttr("defaultArnoldRenderOptions.regionMinX", minX)
    maya.cmds.setAttr("defaultArnoldRenderOptions.regionMaxX", maxX)
    maya.cmds.setAttr("defaultArnoldRenderOptions.regionMinY", minY)
    maya.cmds.setAttr("defaultArnoldRenderOptions.regionMaxY", maxY)
    print(f"minX={minX}, maxX={maxX}, minY={minY}, maxY={maxY}")

    prefix = data.get("output_file_prefix")

    maya.cmds.setAttr(
    	"defaultRenderGlobals.imageFilePrefix",
    	f"_tile_{tileNumY}x{tileNumX}_{numYTiles}x{numXTiles}_{prefix}",
    	type="string"
	)

    print(f'Output file name: {maya.cmds.getAttr("defaultRenderGlobals.imageFilePrefix")}')

When you make these new changes to the Maya adaptor, you will need to save the file and then call the build_wheels.sh script in deadline-cloud-for-maya to create a build of your changes. This will create a folder named “wheels” in deadline-cloud-for-maya folder that contains packages for the custom build of the adaptor.

Add a Step to the Job Template to Run a Tile Assembly Script

The tile assembly step will not start until after the rendering is completed, due to its dependency on the rendering, ensuring all the tiles will be accessible through job attachments. The tile assembly is done through a bash script, which calls a program that handles stitching the tiles together, such as FFmpeg. FFmpeg can be made available to your SMF workers by listing the conda-forge community channel as a Conda channel when submitting the job to add the package to the queue’s conda environment.

For third-party programs like FFmpeg, you could also create a conda package and channel for your Deadline Cloud workers. Instructions on how to do this is covered in Create a conda package and channel for AWS Deadline Cloud. Once FFmpeg is available to your workers, they will be able to successfully run a bash script using the FFmpeg command.

In the example job bundle, the assembly step is titled “tile assembly” in the template.yaml. This step will run a bash script that is directly contained in the template as an embedded text file. The bash script makes some important assumptions about the tile render output:

  • The output folder will either contain tiles, or folders with tiles, never both.
  • The output file structure does not contain more than one layer of Maya subfolders.
  • Tiles are in the correct sequential order given assembly goes left-right from top-bottom.
  • The name.#.ext or name.ext Maya “Frame/Animation ext” render setting is used for the scene.
  • The FFmpeg command in the example expects a PNG file format for the tiles. If you choose to render tiles with a different file type, use a text editor to change all uses of “.png” in the bash script to the desired file format.

Submit the Tile Rendering Job

To submit the tile rendering job, you will need a simple Maya scene such as a learning scene offered by Autodesk Maya. In a terminal, use the following command to open the GUI submitter.

deadline bundle gui-submit <job bundle directory>

You may need to login to the Deadline Cloud monitor if you have not already. If you need to change your Deadline Cloud settings, such as which farm and queue you want to render on, you can use the “Settings…” button. In the “Shared job settings” tab, you will need to set the “Conda Packages” value to “maya=2024.* maya-mtoa maya-openjd ffmpeg” and “Conda Channels” to “deadline-cloud conda-forge”.

The tile rendering parameters that were added to the job template are found under the “Job-specific settings” tab in “Tile Rendering Settings”. Fill out all the fields in “Maya Settings” the scene you are rendering. In the “Override settings”, The “Wheels directory” should list the path for the wheels folder that was created earlier so that the workers will use the custom build of the adaptor.

Example of “shared job settings”, “job-specific settings”, and “job attachments” GUI submission settings for a tile rendering job

Figure 2: Example GUI submitter configuration settings.

When you are ready to submit, click the “Submit” in the bottom right of the GUI window. Once you get confirmation that the job has been submitted, the job will be visible in the Deadline Cloud monitor. When the job completes, you can download the output from the monitor. This should output the tiles that were created alongside the reassembled image.

Estimated Costs

As running this job uses Deadline Cloud SMF workers and Arnold to render, there are some costs for the instances used to render, and the Maya and Arnold usage-based licensing. Refer to the Deadline Cloud pricing page for more information.

Clean Up

  1. Delete any files downloaded or git cloned by using the normal deletion steps for your OS.
  2. Remove any additions made for the development version of the Maya adaptor and submitter and uninstall the python packages in your Maya’s python distribution.

Conclusion

With a tile rendering workflow, you can minimize the rendering time and memory usage for large renders. I’ve described how to modify the Arnold handler in the Maya adaptor, and a job template to render a tile rendering job using Linux workers. Working from this example, further modifications could be made to incorporate tile rendering with different renderer or tile assembly programs. By customizing the functionality of the AWS Deadline Cloud plugins and utilizing the structures provided by OpenJD, workflows like tile rendering can be customized to achieve your rendering needs with your Deadline Cloud farm.

Contact an AWS Representative to know how we can help accelerate your business.

Further Reading

Alison Hardy

Alison Hardy

Alison Hardy is a Support Engineer Intern at AWS Thinkbox.