Running scalable, cost-effective Edit in the Cloud workloads on Amazon FSx for Windows File Server

As the creative landscape has evolved over the last few years, content businesses have needed to adapt to support globally dispersed workforces, to prioritize creativity over technology, and to embrace a shift towards sustainability. Amazon Web Services (AWS) helps businesses meet these challenges by offering services that allow customers to Edit in the Cloud. By using a virtual video editing environment on AWS, editors and creative professionals can work and collaborate from anywhere, and businesses can support scalable, flexible, and more creative workforces.

Adobe Premiere Pro is an industry-leading video editing software that is used across many industry verticals and organizations. A highly performant, low latency shared storage solution is an integral part of any Adobe Premiere Pro implementation. The Amazon FSx family of services delivers fully managed file systems across four widely used file system types: Amazon FSx for Windows File Server, Amazon FSX for NetApp ONTAP, Amazon FSx for OpenZFS, and Amazon FSx for Lustre. Amazon FSx provides high-performance, shared storage of media files for creatives that need live collaboration and for content production workloads that have high performance and low latency requirements.

Recently, FSx for Windows increased the maximum I/O performance achievable on a single file system from 2 GBps to 12 GBps for throughput, and from 80k to 400k for IOPS. These performance scalability improvements allow edit-in-the-cloud use cases (such as high-throughput editorial teams or fast-paced newsroom workflows) to simultaneously playback an even larger number of video streams and support even more editors, without impacting video quality or dropping frames. In addition, FSx supports the ability to dynamically scale I/O performance up or down on a file system. Customers can use FSx for Windows to achieve scalable and cost-effective storage for their video editing environments.

Solution overview and testing procedure

Figure 1: Customer high level architecture

In order to validate the performance of FSx for Windows for Edit in the Cloud workflows, we used a combination of real-world workloads and synthetic methods. We crafted an FIO script with the purpose of simulating the playback behaviour of Adobe Premiere for a 300GB ProRes 422 4K HQ media asset at 29.97 FPS. The FIO script generates jobs mirroring the block size and bitrate used by Adobe Premiere during the playback of media using the specified codec. Each job within the script is configured to read data at the bitrate speed of the chosen codec, plus an additional 30%, aligning with the typical performance requirements of Adobe Premiere.

Importantly, the script also enforces a minimum bitrate in line with the codec’s documented specification. As with real video playback, it is crucial for each job (media stream) to have consistent throughput; if any job fails to meet the minimum bitrate, the remaining jobs will fail until stability is re-established.

Additionally, as a control measure, we ran two concurrent instances of Adobe Premiere, each handling the playback of a genuine Prores 422 4K HQ at 29.97 FPS media asset. This set up enabled us to access the application’s performance in real-time and closely monitor for any dropped frames in the Adobe Premiere instances. FIO was then used to measure the level of concurrency achieved during the test.

We tested FSx for Windows file systems at multiple throughput capacity levels, including 2GB/s, 4GB/s, 9GB/s, and 12GB/s (throughput capacity determines the level of I/O resources available to the file system). The Server Message block (SMB) configuration was standard using SMB 3 and SMB Multi-Channel.

Figure 2: Testing the high level architecture

Performance results

The workload results for file systems at different throughput capacity levels were as follows:

• The 2GB/s filesystem supported smooth playback with up to 16 streams of ProRes 422 HQ 4K 29.97 FPS within Adobe Premiere. Adding a 17th stream reliably triggered the occurrence of dropped frames.

• The 4GB/s filesystem supported 36 streams of ProRes 422 HQ 4K 29.97 playback before dropped frames were observed within Adobe Premiere.

• The 9GB/s filesystem supported 62 streams of ProRes 422 HQ 4K 29.97 FPS before dropped frames were observed within Adobe Premiere.

• The 12GB/s filesystem supported 70 streams of ProRes 422 HQ 4K 29.97 FPS before dropped frames were observed within Adobe Premiere.

Figure 3: Test results

Additional performance considerations

These results provide valuable insight into achievable performance for Adobe Premiere workloads on FSx for Windows. Note that the codecs used in these tests were on the higher end of the performance scale, with the codec ProRes 422 HQ 29.97 FPS having a throughput requirement of ~900MB/s. This is in stark contrast to XDCAM 50, which has a throughput requirement of only 50Mb/s. Keep this in mind when comparing our results with the codec you may use for your editing.

Other considerations with cloud-based editing are the number of ‘active’ layers on the Adobe Premiere timeline, which determines how much throughput is required from storage per editor. Clips that are overlapping may not impact storage performance as Adobe Premiere uses a “bird’s eye view” and will simply pull the top clip from storage. However, if a layer is visible and the overlapping layers have an fx on each of them, it will pull those clips from storage.

Scrubbing also has an impact on storage. Slight scrubbing of a few hundred frames within the timeline are often cached and will have minimal impact on storage. However, extensive scrubbing such as 2x or 3x scrubbing, or large manual scrubs, will increase throughput requirements from storage, as the application requests these frames from storage at 2x or 3x the bitrate of the media asset.

Figure 4: Active frame comparison

Summary

The FSx for Windows performance improvements allow editors to scale from 16 streams of ProRes 422 HQ 4K 29.97 FPS up to a remarkable 70 streams, using the new maximum throughput capacity level of 12GB/s. These performance levels were achieved without dropping a single frame.

In addition, customers can scale up and down the throughput capacity of their file systems at any time, allowing them to scale their storage performance as their workload requirements change. For example, customers can easily increase their file system’s throughput capacity to support growing numbers of video editors in their workforce, or to achieve cost efficiencies for fast-paced newsroom workflows with video stream requirements that vary over time.

Customers can scale even further by using features like DFS Namespaces to group file shares on multiple FSx for Windows file systems into one common folder structure (a namespace) used to access the entire file dataset.

Overall, performance scalability improvements make FSx for Windows an ideal storage solution to support scalable, cost-efficient cloud-based editing environments.