AWS Elemental Live introduces JPEG XS for low-latency, visually lossless contribution to the cloud

An introduction to JPEG XS

As video consumption increases and content providers move toward higher quality content, bandwidth capacity becomes ever more important. Broadcasters transitioning from traditional serial digital interface (SDI) infrastructure to internet protocol (IP) technology for uncompressed video transport within the broadcast facility follow Society of Motion Picture and Television Engineers (SMPTE) standards like SMPTE ST 2110 or the SMPTE ST 2022-6. Please see the excellent SMPTE ST 2110 blog written by my colleague, Brian Bedard, part one of a three-part series.

The IP environment offers bandwidth savings, decreased operational costs, and a common data center. However, a forklift approach from SDI to IP is not feasible. This means that SDI and IP will coexist for some time. As content providers push toward 4K resolution (and in some cases 8K resolution), growing bandwidth requirements puts more stress on current infrastructure. The same limitations exist for contribution to the cloud. This drives the need for lightweight compression that preserves video quality while minimizing encoding/decoding latency. AWS Elemental Live from Amazon Web Services (AWS)—which lets you encode live video on-premises for events and 24/7 streams—has delivered a feature that accomplishes just that.

Enter JPEG XS. JPEG XS (ISO/IEC 21122) is a visually lossless, low-complexity, and low-latency video codec. JPEG XS targets visually lossless quality obtained by intraframe encoding and the use of fixed compression ratios. In testing, natural video sequences are visually lossless at ratios of up to 10:1. Some content, like computer-generated imagery (CGI) video, may require a lower ratio. The JPEG XS codec offers constant bitrate that conforms nicely with the SMPTE ST 2110 suite and thus can be carried using SMPTE ST 2110-22—support for compressed video transport.

JPEG XS offers low complexity that facilitates a smaller hardware footprint without the need for additional memory. The codec can run on both CPU- and GPU-based systems while taking advantage of parallelism to achieve real-time or faster than real-time. This low complexity cuts down on power consumption, reducing energy costs.

JPEG XS provides extremely low latency. The codec boasts subframe latency from encoding to decoding, but realistically, your transit time and any buffering along the way will add additional latency. In an internal test, we output JPEG XS from AWS Elemental Live on-premises to AWS Elemental MediaConnect—a high-quality transport service for live video—then input it back to the on-premises AWS Elemental Live for JPEG XS decoding. The test proved subsecond latency through the entire path. AWS Direct Connect—a cloud service solution that makes it easy to establish a private connection between your premises and AWS—provides a dedicated network path to and from the cloud in this test. The JPEG XS codec’s constant bitrate made the bitstream compatible with IP delivery.

Applications of JPEG XS

JPEG XS can be used anywhere that uncompressed video is in use. It’s a good fit for low-latency applications like virtual reality / augmented reality, autonomous vehicles, vehicle displays, medical procedures, and mobile devices, to name a few. AWS Elemental Live can also unlock new video over IP workflows. You can use JPEG XS in your SMPTE ST 2110 uncompressed environments within a broadcast facility, and you can use the codec to for remote production of events and to improve live video monitoring.

Due to bandwidth constraints on entry to AWS cloud infrastructure, JPEG XS lends itself to hybrid video workflows, reducing the throughput strain as customers shift to playout in the cloud. AWS Elemental Live provides the low-latency, visually lossless bridge needed to move uncompressed video to the cloud, working alongside AWS Elemental MediaConnect. And MediaConnect can provide JPEG XS or AWS Cloud Digital Interface (AWS CDI) output for cloud processing. AWS CDI is a network technology that transports high-quality uncompressed video inside AWS.

AWS Elemental Live and JPEG XS

AWS Elemental Live has released its JPEG XS feature set, which makes it simple for AWS customers to ingest/decode and output/encode to the ISO/IEC 21122 standard. Both the encode and the decode functions support transport of the JPEG XS compressed video stream, and follow the SMPTE ST 2110-22 specification for compressed video transport.

AWS Elemental Live offers overall support for SMPTE ST 2110 and Networked Media Open Specifications (NMOS) for the Advanced Media Workflow Association (AMWA). It breaks down as follows:

AWS Elemental Live SMPTE ST 2110 support

2110-10: System Timing and Definitions
2110-20: Uncompressed Active Video, Support for SMPTE ST 2022-7
2110-21: Traffic Shaping and Delivery Timing
2110-22: Constant Bitrate Compressed Video, Support for SMPTE ST 2022-7
2110-30: Pulse-Code Modulation (PCM) Digital Audio
2110-31: Audio Engineering Society (AES) Transparent Transport
2110-40: SMPTE ST 291-1 Ancillary Data

AWS Elemental Live NMOS support

AMWA-IS-04: Discovery and Registration
AMWA-IS-05: Connection Management

AWS Elemental Live ingress support

AWS Elemental Live supports ingest according to SMPTE ST 2110-22, which specifies parameters for the Real-Time Transport Protocol (RTP)–based transport of constant bitrate compressed video over IP networks, referenced to a common reference clock. For resiliency, AWS Elemental Live can ingest SMPTE ST 2022-7–compliant streams.

AWS Elemental Live egress support

The AWS Elemental Live JPEG XS encoding requires an L800 series appliance with a 25 Gigabit Ethernet (GbE) interface. Standard-definition (SD) and high-definition (HD) resolutions are supported as of AWS Elemental Live version 2.21.3 (general availability). AWS Elemental Live supports compression ratios from 2:1 to 12:1. The 12:1 ratio will provide maximum density in tandem with the lowest bandwidth for contribution to the cloud. The range of compression ratios will help you make sure you can reach the subjective/objective video quality level that you desire while achieving the bandwidth savings provided by the JPEG XS codec. The AWS Elemental Live JPEG XS output is transported according to SMPTE ST 2110-22 specification, and AWS Elemental Live can output SMPTE ST 2022-7–compliant streams.

How-to steps for outputting JPEG XS to the cloud and back

Now, let’s get our hands dirty. In the example that follows, we output an AWS Elemental Live JPEG XS encoding from on-premises to AWS Elemental MediaConnect for contribution into the cloud. We then take the AWS Elemental MediaConnect JPEG XS output back down to AWS Elemental Live for ingest and distribution.

AWS Elemental Live and AWS Elemental MediaConnect need to be in the same Amazon Virtual Private Cloud (Amazon VPC), a service that gives you complete control over your virtual networking environment. AWS Direct Connect facilitates the network path to the cloud. AWS Elemental MediaConnect requires SMPTE ST 2022-7 for JPEG XS ingress and output. The following steps guide you through setting up this workload.

AWS Elemental Live JPEG XS encoding

Follow these steps to create an AWS Elemental Live event that outputs SMPTE ST 2110-22 with JPEG XS encoding:

Create a SMPTE ST 2110 Output Group.
Add an Output for which the Stream type is Video.
For Video Codec, select JPEG XS.

Select Advanced and choose a Compression Ratio (1:12 providing the best BW savings).

Add another Output for which the Stream type is Audio.
For Audio Codec, select PCM.
Use the remaining defaults.

For AWS Elemental MediaConnect, we will require SMPTE ST 2022-7 packet merging.

We provide the Primary and Secondary Destinations after setting up the AWS Elemental MediaConnect flow.
When the AWS Elemental Live event starts, a SMPTE ST 2022-7–compliant Session Description Protocol (SDP) file will be programmatically generated and will be located on the appliance in /opt/elemental_se/web/public/directory.

AWS Elemental MediaConnect JPEG XS ingest

Follow these steps to set up AWS Elemental MediaConnect JPEG XS Input:

Open the AWS Elemental MediaConnect console at https://console.aws.amazon.com/mediaconnect/
On the Flows page, choose Create flow.
In the Details section, for Name, specify a name for your flow. This name will become part of the Amazon Resource Name (ARN) for this flow.
For Availability Zone, choose the Availability Zone where your VPC subnet resides.
In the Source section, for Source type, choose VPC source.
For Name, specify a name for your source. This value is an identifier that is visible only on the AWS Elemental MediaConnect console.
Skip to the VPC interface section.
For each VPC that you want to connect to the flow, do the following:
- Choose Add VPC interface.
  - For Name, specify a name for your VPC interface. The name of the VPC interface must be unique within the flow. We’ll refer to this as “VPCa.”
  - For Type, choose EFA as the type for VPCa and ENA for VPCb.
  - For Role ARN, specify the ARN of the role that you created when you set up AWS Elemental MediaConnect as a trusted service.
  - For VPC, choose the identifier of the VPC that you want to use.
  - For Subnet, choose the VPC subnet that you want AWS Elemental MediaConnect to use to set up your VPC configuration. You can choose as many as you want, but you must choose at least one.
  - For Security groups, specify the VPC security groups that you want AWS Elemental MediaConnect to use to set up your VPC configuration. You must choose at least one security group.
- Add another VPC interface, and specify a name. We’ll refer to this as “VPCb.” Repeat the same steps to complete the VPC interface setup.

Next, create your media streams.
- In the Media streams section, choose Add media stream.
- In the Name field, specify a descriptive name that will help you distinguish this media stream from others in the flow.
- For Description, specify a description that will help you remember the use of this media stream.
- For Stream ID, specify a unique identifier for the media stream.
- Choose Advanced options to display the additional options based on your stream type.
- For Stream type, choose Video.
- Media clock rate is the sample rate for the stream and is set to 90000. This value is measured in kilohertz (kHz).
- For Video format, specify the resolution of the video.
- For Exact framerate, specify the frame rate of the video. This value should be represented in frames per second.
- For Colorimetry, specify the format that was used for the representation of color in the video.
- For Scan mode, specify the method that was used to scan the incoming video.
  - Choose Interlace if the incoming video is interlaced (for example, 480i or 1080i).
  - Choose Progressive if the incoming video is progressive (for example, 720p or 1080p).
  - Choose Progressive segmented frame (PSF) if the incoming video is PSF (for example, 1080psf).
- For TCS, specify the transfer characteristic system (TCS) that was used in the video.
- For Range, specify the encoding range of the video.
- For PAR, specify the pixel access ratio (PAR) of the video.
- Choose Add media stream.

Next, add the audio.
- Select Add media stream.
- In the Name field, specify a descriptive name that will help you distinguish this media stream from others in the flow.
- For Description, specify a description that will help you remember the use of this media stream.
- For Stream ID, specify a unique identifier for the media stream.
- Choose Advanced options to display the additional options based on your stream type.
- For Stream type, choose Audio.
- For Media clock rate, the default value of 48000 kHz, which will match our AWS Elemental Live configuration.
- Choose Add media stream.

- Scroll back up to the Sources section.
- For Protocol, choose ST 2110 with JPEG XS.
- For Max sync buffer, specify the size of the buffer that you want AWS Elemental MediaConnect to use to sync incoming source data. This value is measured in milliseconds (ms).
- For VPC interface name 1, choose VPCa as a source.
- For VPC interface name 2, choose VPCb as a source. There is no priority between VPC interfaces 1 and 2.
- For each media stream that you want to use as part of the source, do the following:
  - For Media stream name, choose the name of the media stream.
  - For Encoding name, accept the default value.
    - For ancillary data streams, the encoding name is smpte291.
    - For audio streams, the encoding name is pcm.
    - For video, the encoding name is jxsv.
  - For Inbound port, specify the port that the flow will listen on for incoming content. This value can be anything from 1024 to 65535, with the exception of 2077 and 2088 (those ports are reserved for other protocols).

Hit the Create Flow button.
Next, under the Sources, scroll down to Input Configurations and find the two IP addresses for each VPC.
Now, go back to the AWS Elemental Live Output Configuration. Update the Primary and Secondary Destinations with the two VPC IPs.
- Make sure to use the 25 GbE interface for your outputs.

Start your AWS Elemental MediaConnect flow and start your AWS Elemental Live event.

Now we have JPEG XS video in the cloud. Using AWS Elemental MediaConnect, we can take AWS CDI out to AWS Elemental MediaLive—a broadcast-grade live video processing service—or any third-party playout system that supports AWS CDI.

MediaConnect can also output JPEG XS. If you have a playout system that originates in the cloud, the signal can be brought back on-premises using and AWS Elemental Live. We will use the MediaConnect flow we just created to illustrate this point.

AWS Elemental MediaConnect JPEG XS output

Follow these steps to create an AWS Elemental MediaConnect Output:

Choose the Output tab in your MediaConnect Flow.
In the Name field, specify a descriptive name that will help you distinguish this output from others in the flow.
For Protocol, choose ST 2110 with JPEG XS.
For VPC Interface 1, choose VPCa from the drop-down menu.
For VPC Interface 2, choose VPCb from the drop-down menu.
Provide the AWS Elemental Live 25 GbE redundant interface IP addresses for the Destination IP addresses.

Scroll down to Media Stream Configuration, and choose Add Media Stream Configuration.
Select your Video Media Stream for Media Stream Name.
Provide the Destination Port. We will use 5001 in this example.
Choose Add Media Stream Configuration. We will add our audio here.
Select your Audio Media Stream for Media Stream Name.
Provide the Destination Port. We will use 5002 in this example. The ports need to be unique.
Hit the Save button.

For SDP creation, we will use an AWS Elemental MediaConnect tool called sdp_converter.py, available on GitHub.
- The sdp_converter.py script converts AWS Elemental MediaConnect JSON-formatted data to an SDP file compatible with the SMPTE ST 2110-20/-22/-30/-31/-40 specifications. The script ingests a JSON file that encapsulates the AWS Elemental MediaConnect stream data.

AWS Elemental Live JPEG XS decoding

Follow these steps to create an AWS Elemental Live event that inputs SMPTE ST 2110-22 with JPEG XS decoding:

Provide SDP Location, Media Index, and Interface values for the following sources.
- Video SDP Location (required)
- Audio SDP Location (required)
- Ancillary SDP Location (optional)

If you’re using NMOS, tick the NMOS Control box.
Provide the Video and Audio Stream Interfaces. We will use eth4 and eth5 Interfaces for SMPTE ST 2022-7 ingestion.

Note: Resolution and frame rate support is limited to the resolutions and frame rate support on any SDI interface.
Create the output type you desire and hit Start.

You now have video/audio back on-premises. Process and distribute as you wish.

Summary

In this post we talked about the stress on infrastructure created by higher resolutions and higher frame rates. This creates the need for a visually lossless, low-complexity, and low-latency codec. JPEG XS provides these qualities and is a good solution anywhere uncompressed video over IP is in use. By using JPEG XS, you can enrich many workflows.

We are excited that AWS Elemental Live and AWS Elemental MediaConnect facilitate workflows requiring SMPTE ST 2110 and contribution to the cloud on AWS for remote production, cloud playout, or cloud processing. Elemental Live and MediaConnect also provide contribution from the cloud back to on-premises. To learn more, please refer to the MediaConnect documentation.

For more information, please check out the AWS Elemental Live documentation for Working with SMPTE ST 2110. We look forward to working with our customers on video workflows that JPEG XS unlocks.

AWS for M&E Blog