AWS introduces Open Job Description for render pipelines

New open standard aims to make render jobs portable across any rendering pipeline

Today, Amazon Web Services (AWS) introduces Open Job Description (OpenJD), a new, flexible, open specification for defining render jobs to be portable between studios and render solutions. Render pipelines can vary widely depending upon project, technology, and administrative decisions. OpenJD is designed to maximize compatibility, reduce complexity, and allow greater ease of adapting the pipeline when these factors inevitably change. Developed by the team behind AWS Thinkbox Deadline, the production-proven compute management software for render farms, OpenJD is now available for public review and comment. We invite feedback on the new render submission specification from the developer community.

Removing bottlenecks from 2D and 3D asset pipeline innovation

In industries such as media and entertainment, game development, architecture, engineering and construction (AEC), and industrial design, workflows for 2D and 3D assets often involve several artists and teams focused on specific components of a project. They work with many different Digital Content Creation (DCC) software applications specialized for each task, including 3D modeling, animation, computer-aided design (CAD), simulation, and visual effects. Render farm products, such as Thinkbox Deadline, ingest the artist-generated work and process it across multiple compute instances, allowing artists to view the results of their work during the project and to deliver the final assets much faster than would be possible if they only had access to their own workstations. Furthermore, studios make specific selections for their render engines depending upon characteristics such as processor requirements, speed, quality, and fidelity; and choose render farm management tools based on the support the studio requires for various operating systems, computing architectures, scheduling capabilities, and administration needs.

Once a studio adopts a render farm, developers at the studio start developing tools and integrations to optimize the studio’s processes for compatibility with that specific render farm. They often build intricate pipelines for various DCCs, industry-standard render engines, and newer real-time renderers emerging in creative pipelines. The more creative tools a studio provides for its artists to work with, the more complex the development effort becomes. Trying out new variations to improve final quality, render farm efficiency, or delivery times introduces greater risk of falling out of date, and much of the pipeline work is thrown away if the team decides they need a new render solution. Historically, these teams are faced with a daunting challenge: they must choose one system that will serve all their needs, and rely on software vendors to support their solution. On the other hand, software vendors can’t always justify developing render farm integrations, because they would have to support each render farm solution for every DCC version. These development bottlenecks stall the continuous innovation of the pipeline, limit studios’ ability to build competitive differentiators, and dampen efforts to avoid commoditization.

Open Job Description solves these issues by enabling studios to make their render jobs portable across any rendering pipeline, and expand compatibility for the varying tools they use when they shift to new render workflows. With the OpenJD standard, studios and software developers will be able to freely develop render farm integrations once to create the standard jobs of their choice, and then implement them with any render farm management system that understands the OpenJD standard.

More flexibility for businesses creating 2D and 3D assets

“Choosing a render farm management tool is an exercise in scoping proprietary integration work for all the software tools artists and customers use,” said Félix Duchesneau, GM of Rendering, AWS Thinkbox. “This issue goes away when render job submissions are portable and thus compatible with a wider variety of pipelines. Once render farm integrations support the new standard, customers will have more choice when considering new pipeline tools. They can use solutions that already support OpenJD, or simply build an OpenJD translator for the new solution, creating a ‘build once, reuse often’ investment.”

Making render jobs portable across multiple pipelines also enables technical pipeline developers to more easily and flexibly test, adapt, and deploy the best render farm solutions for their projects, eliminating the need for studios and developers to recreate every render submission variant from scratch.

“Open Job Description is designed from the ground up to be portable, human and code readable, and extensible, and contain pipeline actions that describe all processing steps—and only processing steps. This makes the standard well-situated for integrating with render farm solutions,” said Pauline Koh, Sr. Product Manager for Content Creation at AWS. “It’s also future-looking to serve customers on their AWS and emerging visual compute journeys. We’re taking this first step to invite pipeline developers to review and comment on the open specification, and we look forward to the feedback they’ll provide.”

Call for community feedback

Pipeline developers are invited to review and comment on the open specification, available here. To learn more about the need for a standard approach to job submission in render pipelines, please watch Pauline Koh discuss the matter at Academy Software Foundation’s Open Source Days session Portable Render Jobs for Open Source Content Production Pipelines and view the Open Source and the Future of the Creative Pipeline presentation by Antony Passemard, GM of AWS Creative Tools, at The VIEW Conference (ticket required). Additionally, read this piece in Post Magazine authored by Passemard based on his VIEW Conference presentation.