AWS for M&E Blog

PixelStrings: a cloud-based media conversion platform powered by industry standard tools

This post is co-authored by Lance Mauerer, CEO and Founder – Cinnafilm, Inc., Marc Leonard, PixelStrings Developer – Cinnafilm, Inc., Alfred She, Director of Product Management – Cinnafilm, Inc., and Mark Stephens, Partner Solutions Architect – AWS. The content and opinions in this post are those of the third-party author and AWS is not responsible for the content or accuracy of this post.

The exponential growth of the video streaming market has been a major driving force behind the mass migration of video content and processing power into the cloud. Given the decreasing costs and increasing speeds of broadband service, among other variables, the decision to move media assets and media processing power to the cloud is a natural next step.

Scalability is a key factor in differentiating the cloud from the traditional “on-premises” approach; cloud solutions are scalable, while on-prem solutions are scalable at relatively high costs. For many, a well-designed cloud service provides a seamless way to interact with remote machines and services as if they were running locally.

Traditional media companies and television networks have been migrating more and more of their source assets, media processing tools, workflows, and final deliverables to the cloud, but there are very few cloud solutions that use best-in-class technologies cohesively.

Enter PixelStrings.

Hosted by AWS, PixelStrings is an all-in-one conversion and delivery platform that combines an easy-to-use web browser user interface (or API for enterprise clients building media supply chain automation) with industry standard tools deployed on a modern cloud infrastructure. PixelStrings is a pay-as-you-go service designed for massive scalability. It addresses the following critical “pain points” that plague more traditional approaches.

  1. Not all users can readily financially access high quality media processing tools.
    Solution: PixelStrings lets anyone, anywhere, access enterprise grade industry standard tools to create the highest quality video deliverables, stored in Amazon Simple Storage Service (Amazon S3).
  2. It is cumbersome to negotiate and manage licenses for multiple pieces of technology, such as dozens of video and audio codec libraries.
    Solution: PixelStrings manages all the necessary licenses for transcoding as well as licenses for Cinnafilm’s complete image processing library.
  3. It is cumbersome to unify the wide range of decoders and encoders into a coherent processing workflow for video transcoding and image processing.
    Solution: PixelStrings unifies the technology used in all of the processing steps of customers’ video workflows, and provides a web interface that makes it easy for customers to define workflows and manage media assets.
  4. Hardware and software infrastructure have relatively high up-front and on-going costs, and is costly to scale.
    Solution: Pixelstrings removes the need for local hardware and software infrastructure, offers a simple cost structure, and is highly scalable.
  5. Enterprise content owners are evaluating a more cohesive “golden disc” mastering format such as Interoperable Master Format (IMF) to reduce the cost and overhead associated with revision variants, and/or are required to deliver content in this format (i.e. Netflix submissions).
    Solution: Cinnafilm has partnered with industry leading IMF experts at CineCert LLC, to deploy a “simple IMF” concept that enables users to begin to wrap to this format without overthinking the problem. IMF is a wrapping format inside PixelStrings.

Best-in-class Technology

Cinnafilm, Inc., the parent company of PixelStrings, has been delivering industry standard, file-based image processing software for over a decade, including Tachyon for frame-rate conversion, deinterlacing, and broken cadence removal, Dark Energy for noise removal, re-texturing, and spatial resolution scaling, Wormhole for video and audio retiming, and the newer XenonTM for SDR-to-HDR conversion.

Several years ago, Cinnafilm made its image processing software available as a part of several cloud-based media processing services. Last year Cinnafilm augmented its image processing software by combining it with RadiantGrid, Cinnafilm’s own transcoder acquired from Wohler in 2017, which supports industry standards for professional media processing and encoding. This combination makes PixelStrings an ideal recipe for a service that users can rely on for converting high quality, enterprise-grade, mezzanine-level media for cinema, broadcast, and Over-the-top (OTT) applications. Cinnafilm focuses on ultimate picture and playback quality as the technical primary in its professional conversions, and is the only company to have won the Hollywood Professional Association (HPA) Engineering Excellence Award three times for its technologies.

Automatic Scaling for Video Processing

Cinnafilm has developed its own proprietary image processing library to handle the higher-end, more complex conversions such as motion compensated frame rate conversion, telecine correction, and noise management. This dynamic link library (dll) is broken into many subset features by functionality (presented to end users as products with names like Tachyon, Dark Energy, etc.) This library has been developed and maintained over the course of 16 years of focused R&D in cinema and broadcast, and has always been very computationally demanding; tracking pixels with 1/8 pixel accuracy, temporally, requires significant compute and memory resources, and hunting for errors/artifacts forward and backward on the timeline adds to this overhead need significantly.  The only compute resource that can handle this load in real time is the GPU (Graphics Processing Unit). Cinnafilm uses Amazon Elastic Compute Cloud (Amazon EC2) G3 and AWS EC2 P3 instances. Cinnafilm has developed its library to operate exclusively on the NVIDIA variant, relying on CUDA since its onset in 2009. This image processing library is unique to Cinnafilm, and they license components of it (such as Tachyon) to various 3rd party transcode companies such as Telestream, Imagine, Dalet, Root6, Encoding.com, OwnZones, SDVI, and many others. PixelStrings is unique in that it is a complete solution set offered only from Cinnafilm. The trick to these types of complex ecosystems is a balance of resources. For example, the transcode and audio functions (with codecs and libraries provided by other third parties as well) must be run on the CPU at approximately the same efficiency as the image processing library runs on the GPU­—dividing the work intelligently is critical for the user experience. PixelStrings is a complete system of multiple machines balancing all the work, the data, and the experience for the end-users, and requires a number of machines all talking to one another. Once the system is balanced, then it must be able to scale to handle more than a single job.

PixelStrings scales to handle a large number of simultaneous video processing jobs, and this scaling is transparent to the user. PixelStrings combines static and dynamic GPU instance management. PixelStrings maintains a fixed number of such GPU instances that have the transcoder/image processor installed—let static pool denote this group of GPU instances. When a user submits one or more jobs, PixelStrings assigns each job a GPU instance from the static pool, up to the number of available static pool GPU instances. Most of our GPU instances are g3.4xl and p3.2xl, based on availability.

CPU and GPU instances are assigned from the same AWS region as the user’s S3 bucket. When an Amazon EC2 instance and S3 bucket are in the same region, we achieve over 1 Gb/s file transfer speeds, and egress costs are avoided.

When the number of pending jobs exceeds the size of the static pool, PixelStrings launches new GPU instances using an Amazon Machine Image (AMI) that contains a pre-installed transcoder, and the newly launched instances help the static pool to process all of the jobs. When the jobs are complete, newly launched GPUs are terminated. The static pool is large enough such that most of the time, jobs start running immediately, reducing end-to-end processing times seen by the user.

PixelStrings supports a wide variety of workflows. Depending on the workflow, some jobs process at real time or faster than real time. Regardless of the workflow, all jobs fully load the GPU during the image essence processing stage.

Example of a Cost Savings Analysis

Consider the following example of PixelStrings’ value; a North American content provider has an S3 bucket containing a set of show masters in 1920×1080 29.97i (interlaced) format. When a European broadcaster wants to license that content, they may require the content to be in 3840×2160 25i format.

The North American company uses PixelStrings to meet the European broadcaster’s delivery requirements. PixelStrings reads the show masters from the S3 bucket, performs a Tachyon motion-compensated standards conversion, followed by a Dark Energy up-res step, resulting in output clips that meet the European delivery requirements. If the delivery also calls for an SDR-to-HDR conversion and/or IMF output, PixelStrings can perform all of these processing steps in a single pass. The company can automate much of this process by building out logic that enables the PixelStrings API. Since PixelStrings operates at a predictable expense rate, the company can budget accurately for such conversions in a “pay as you go” construct that is simple enough for the company and its clients to understand. This predictability, and the freedom to only pay for what they need, offsets the challenges of expensive on prem build outs that must be maintained or replaced every few years, migrating them from a fluctuating “CapEx” process to a predicable “OpEx” one.

For example; a facility could typically spend approximately $100,000 in year one on software CapEx, excluding IT and operational costs to manage it, to run these types of high quality standards conversions and optimizations for a single stream (i.e. enterprise transcode + image processing + sound processing software at ~$60k + a single RU server with NVIDIA GPU at approximately $20,000 + maintenance and support contracts + integration costs).  This one server can run around 9,000 hours in a single year of HD material and 4,500 in UHD format. Assuming a 60/40 split of HD/UHD processing, this equates to over 300k minutes per GPU per year conversion capacity for the system. If the customer has, say 80 episodes at 24 minutes each that they need to standards convert in two different (2 x workflow) resolutions using Tachyon, then they have about 3,680 minutes of work to do. This is a complete mismatch of the 300,000-minute potential they get from buying the stack (CapEx), and it if were only used for this one job they would end up paying nearly $80/min. The same exact workload described here run in PixelStrings will always cost them between $1.99 and $2.99 per minute – half that to run test files, discounted if they get a subscription; therefore, they can easily run this entire job in PixelStrings in AWS for under $10,000, and they get the same exact file outputs for delivery.

PixelStrings is designed to run the same workflows on prem as it does in the cloud, so they could also conceivably have their on-prem for much heavy volume, testing, etc. and manage these “burst” needs as they arise easily by running the workflows in the cloud. With today’s egress costs, burst may be the more practical use of this, but also it is important that AWS can throw 100+ machines at the problem and get it done 100x faster than a single GPU machine.  Convenience of delivery timing can be worth every penny and can be rolled up into their business model predictably (i.e. “overnight delivery” is something people will often pay for) – and knowing the pricing in advance gives them this option which is a huge benefit to them and their clients.

Simplifying the economics of all these moving parts makes the media supply chain simpler to manage, a critical factor given the increase in demand for video.

The ever-growing list of tools in PixelStrings includes:

  • CineCertTM IMF wrapping
  • Cinnafilm’s suite of products:
    • Retiming (Wormhole)
    • Denoise / Up Res (Dark Energy)
    • Standards Conversion / Frame Rate Conversion (Tachyon)
    • SDR to HDR Conversions (Xenon) using Dark Energy and Technicolor ITM
  • A wide variety of codecs and libraries for audio, video, captions, timecodes, and user data processing

The Software Licensing Dilemma Solved

In traditional workflows, distributors and broadcasters and networks would have many pieces of licensed software to make their video pipeline function properly. Each of these pieces of software require their own licenses, and the facility needs staff to manage licensing and installation.

For client-based facilities, it can be difficult to project forward and determine if a piece of software will be used enough in a year to justify the maintenance fees. Utilization is the biggest factor in determining whether a yearlong commitment is worth the cost.

PixelStrings removes all of these decisions.

Because PixelStrings is a pay-as-you-go platform, its users do not need to make any long-term predictions and decisions. Users pay by the media runtime minute, and only pays for what is used. This provides maximum flexibility in budgeting, with no contract lock in.

Asset Management for Enterprises and Individuals

PixelStrings’ gives users three ways to manage their media assets, to make it easy to copy their assets to the cloud for processing:

  1. Upload and process. This is the easiest way to use PixelStrings. The customer browses local content files, uploads the chosen source file(s), then send files through PixelStrings, one at a time. Customers choose to apply any or all of PixelStrings services on a file-by-file basis. There is no need for long term storage or a subscription.
  2. PixelStrings-managed storage. Customers choose one of several paid subscription plans which include a specific maximum amount of file storage space to use with PixelStrings, and a specific recurring discount on all processing charges. The storage is backed by S3. Users upload files directly to cloud storage using the web UI (via drag-and-drop or “file select” dialog box) and process their files using the same web UI. This simplifies storage management, as customers will not need to work with S3 access credentials.
  3. User-defined storage. For enterprise customers who are already using S3 storage to house content assets, PixelStrings uses a set of S3 keys with least privilege, provided by the customer, to access those assets. The customer is still in charge of their S3 buckets and S3 access policies. Customers can revoke or create S3 credentials of the assets at any time, to control which users have access to the assets, as well as whether PixelStrings has access.  For example, an organization can define separate AWS Identity and Access Management or IAM roles and keys for different individuals; this can dictate who has access to what assets.

Security commentary: Users who connect their own storage manage their own options in relation to the assets, security, and egress (i.e. implementing encryption at rest, key access and management of storage keys, which should be rotated regularly). For managed (included) storage PixelStrings provides subscribers a rest encryption feature option and maintains the keys (users will only have access to their own buckets); Cinnafilm will be evaluating improvements to asset and key management security over time. The only risk to any of the un-encrypted assets is if Amazon S3 storage hardware is physically at risk of theft, which is highly unlikely.

These three options allow a user work at the individual level or within a larger organization where customized definitions of security relating to asset access can be tailored to the user’s needs. For more options on AWS-recommended best practices, please check out AWS CloudTrail Events. Regardless of which of these storage options is used, the user has access to all of the processing features of PixelStrings.

Conclusion: The Democratization of Media Conversion

AWS offers an extensive set of cloud solutions that enable media supply chains to work flawlessly. By using web-based professional services like PixelStrings, everyone has the ability to create industry standard, broadcast quality results, without a commitment to long term license fees, service contracts or specialized hardware.

The cloud will enable companies to build complex, predictable machines that automate significant portions of the media supply chain process, enabling humans to spend more time ensuring the content looks good, sounds good, and is enjoyable for the consumers who buy it.

Please visit PixelsStrings to read more, and sign up for free.