AWS for M&E Blog

Benchmarking video codec quality by comparing NDI SpeedHQ and JPEG XS

In live cloud production environments, choosing the right mezzanine codec is crucial for ensuring high-quality video throughout the signal processing chain. Two codecs frequently used in production workflows are Network Digital Interface (NDI) SpeedHQ and JPEG XS. Cloud production applications like vision mixing and graphics insertion commonly use NDI with SpeedHQ to interconnect media functions. JPEG XS, on the other hand, is most used as a modern contribution codec emphasizing minimal latency and high quality. To validate the feasibility of these codecs, we conducted an independent quality benchmark measuring image fidelity, bitrate, and generational quality loss.

In this post, we share our complete methodology, results, and analysis from this study.

Why codec benchmarking matters

Traditional on-premises broadcast facilities use serial digital interface (SDI) or SMPTE ST 2110 to interconnect media functions. These technologies operate entirely on uncompressed video. But transmitting uncompressed video to the cloud isn’t practical, primarily because of network bandwidth constraints. This creates a need for video compression algorithms that prioritize low-latency and high-fidelity to the original video.

Mezzanine codecs such as NDI SpeedHQ and JPEG XS address this issue. They differ fundamentally from distribution codecs such as H.264 and HEVC, which prioritize aggressive bitrate reduction for delivery to end viewers. Demonstrating that these mezzanine codecs produce high quality under realistic testing conditions proves their viability for live cloud production.

Methodology

To measure codec quality, we developed a standardized test procedure and applied it to both NDI SpeedHQ and JPEG XS at several different operating points. We applied the following test procedure for each set of codec parameters:

  1. We encode and decode an uncompressed source YUV video file using the codec under test with the specified quality parameters. This produces a new YUV video file (representing generation 1).
  2. For NDI SpeedHQ, we used NDI Advanced SDK for Linux version 6.3.1. For JPEG XS, we used IntroPix CPU Codec 2.40.1 for x86_64 Linux.
  3. We use the decoded output from the previous step as the input for the next encode–decode cycle. We repeat this for eight total generations.
  4. At every generation, we compare the decoded output against the original source using Peak signal-to-noise ratio (PSNR) as the quality metric. Higher PSNR indicates higher fidelity to the original video. We test multiple generations to simulate workflows where video processing occurs on several different machines, comprising a long signal chain.
  5. We computed PSNR using the standard FFmpeg PSNR filter frame by frame, and we aggregated the per-frame data into the average value across every frame.

For test content, we used frames 600–1,200 of Netflix’s Sparks open content. The source was converted to 4:2:2 chroma subsampling using FFmpeg for all frame geometries and played back the content at a frame rate of 60 fps.

We performed the codec quality tests on an Amazon Elastic Compute Cloud (Amazon EC2) C6id.12xlarge instance. This machine has an Intel Xeon Platinum 8375C CPU @ 2.90GHz processor with 48 vCPUs.

We tested the JPEG XS encoder under a variety of different compression ratios (CRs) from 7:1 to 15:1. Lower compression ratios produce higher quality because the content is less compressed. For NDI SpeedHQ, we measured quality by varying the SDK’s quality parameter (hereafter, referred to as “Q”) from 100 to 350. The quality parameter represents a percentage of the default target bitrate for the codec. For example, a target quality of 200 means NDI will target twice the default bitrate.

Results

The following graph shows the PSNR quality results of the first encoding generation with 1080p content. NDI SpeedHQ shows a small but noticeably higher PSNR over JPEG XS at lower bitrates. With both codecs, the overall quality scales linearly with the actual bitrate. At the high end, JPEG XS has a higher absolute quality measured in PSNR at the cost of a larger total bitrate. For NDI, 10-bit and 8-bit sampled video both produce nearly the same compressed bitrate at the same quality level, with 10-bit delivering slightly higher PSNR. On the other hand, JPEG XS’s bitrate scales directly with the uncompressed bitrate, so at the same compression ratio, the 10-bit video has a higher bitrate than the 8-bit video bitrate. Although not shown in the graph, we gathered results for NDI SpeedHQ quality levels above 350. However, the resulting bitrate and quality were nearly identical to NDI SpeedHQ at quality 350, suggesting the codec enforces a maximum limit to its quality setting.

Graph with bitrate on the x axis and average PSNR on the y axis. A color-coded key corresponds to the measured encoding parameters. Results are described in the text.

Figure 1: PSNR compared to bitrate – 1920×1080 (generation 1)

The following graph shows the PSNR quality results of the first encoding generation with 4K (3840×2160) content. Compared with 1080p, 4K content requires four times as many bits to represent in the uncompressed domain. JPEG XS encodes 4K content with four times the bitrate as 1080p content at the same compression ratio, while NDI SpeedHQ only uses twice the bitrate at the same quality level. The 4K results are similar to the 1080p case in that NDI has lower maximum bitrate and maximum quality but compares favorably to JPEG XS at the lower bitrates NDI SpeedHQ uses. However, the plots themselves look different because NDI targets a lower bitrate factor compared to JPEG XS, shifting the NDI SpeedHQ data points leftwards on the graph.

Graph with bitrate on the x axis and average PSNR on the y axis. A color-coded key corresponds to the measured encoding parameters. Results are described in the text.

Figure 2: PSNR compared to bitrate – 3840×2160 (generation 1)

The following graph shows the PSNR quality score across successive encode–decode cycles. Both JPEG XS and NDI SpeedHQ show minimal quality loss over generations. NDI SpeedHQ occasionally shows minor quality loss on the first generation before stabilizing quality on additional coding generations. The graph only shows a subset of scenarios, but the same pattern occurred across all test parameters.

Graph with encode generation on the x axis and average PSNR on the y axis. A color-coded key corresponds to different test scenarios. Results are described in the text.

Figure 3: Generational quality loss – NDI Q=200 compared to JPEG XS CR=7:1

Conclusion

In this study, we compared codec performance between NDI SpeedHQ and JPEG XS across a range of quality levels. JPEG XS has a higher absolute quality at low compression ratios with the trade-off of higher bitrate. NDI SpeedHQ performs slightly better than JPEG XS at lower bitrates but can’t operate at the highest bitrates and quality levels JPEG XS supports. Both codecs have less than 0.5 dB of PSNR quality loss over subsequent generations, making them safe for use with long signal chains.

Further reading