Improving mission video system efficiency for government operations

The Enterprise Challenge exercise is an annual joint and coalition intelligence, surveillance and reconnaissance interoperability demonstration. You can read our preview of the event here.

In May at the annual Under Secretary of Defense for Intelligence USD(I)-sponsored Enterprise Challenge exercise, Amazon Web Services (AWS) demonstrated how government organizations can combine AWS and AWS Elemental services with traditional video infrastructure to improve video processing at the tactical edge.

During the event, attendees shared insights about the challenges associated with their mission video systems. To address those concerns, we’ve detailed a three-point strategy for implementing a hybrid workflow to enhance video performance, quality and resiliency in disconnected, intermittent, and limited-bandwidth environments.

1. Embrace Hybrid Full Motion Video (FMV) and Streaming

Until recently, most government video systems largely mirrored legacy commercial broadcast and IPTV designs – an HD camera “sensor system” connected to an AVC or HEVC encoder, compressing video, encapsulating using traditional MPEG-2 transport streams, and transmitting over a managed IP network. These flows involved laborious IP multicasting protocols that required comprehensive levels of management to strategically distribute content from source to destination.

Evolutions in media and entertainment (M&E) streaming technologies and the cloud offer government organizations new tools to implement hybrid cloud and on-premises architectures that support broadcast-quality video for enterprise and tactical operations. The architecture pulls from familiar commercial contribution techniques using MPEG-2 based transport streams, combined with ground or cloud-based Adaptive Bit Rate (ABR) video processing, aimed to improve stream reliability, quality, and accessibility across the enterprise. The best design practices for implementing a hybrid FMV streaming architecture are:

Contribution Techniques

Currently, MPEG-2 transport streams are still preferable; they provide predictable real-time performance from existing sensor systems for contribution quality from Air-to-Ground, Air-to-Cloud, or Ground-to-Cloud scenarios. To improve picture quality and reduce bitrate, HEVC/H.265 video compression should be considered as the video contribution format. Recommended by the Motion Imagery Standards Profile (MISP), HEVC is the successor to AVC/H.264 and provides a 2-to-1 compression advantage, enabling higher image quality for the same bitrate, or similar image quality at half the bitrate.

Figure 1 – EC19 experimental aircraft equipped with an AWS Elemental Live HEVC/H.265 contribution encoder.

Distribution Techniques

OTT streaming formats such as HTTP Live Streaming (HLS) or Dynamic Adaptive Streaming over HTTP (DASH) are superior for FMV stream distribution. They offer ABR profiles allowing the same live stream to be encoded at single or multiple renditions, ranging from high-quality, higher bitrate to low-quality, lower bitrate, depending upon the user’s device capabilities and network bandwidth. ABR in particular is a form of web streaming that allows users to stream live or video-on-demand content within a native browser and without dependencies on external thick client applications. Further, web environments consuming ABR content can automatically and dynamically adjust to the best stream quality/rendition for playback based on the device capabilities and available network bandwidth.

Figure 2 – EC19 Ground Operations Center generating live ABR FMV streaming.

2. Improve the Quality of Experience

Live video contribution over RF transmission systems are vulnerable to packet loss if not provisioned for error recovery. Dropped packets equate to data loss, which ultimately degrades the quality of experience (QoE) at the client side decoder. A poor QoE can be noticeable in the form of video pixelization, choppy playback conditions, artifacts, or freeze frames. While wireless RF transmission systems can be rapidly deployed in tactical environments, they can exhibit unpredictable behaviors when pushed to their limits due to bandwidth limitations and uncontrollable environmental conditions. Furthermore, within an RF mesh network, where multiple transmission paths exist, late and/or out-of-order IP packets may occur, compounding issues with live streams. Even wired networks can produce some degree of error, though these tend to be more predictable. Here are a few considerations to protect FMV traffic when up against unpredictable and predictable network conditions:

Wireless contribution from Air-to-Ground or Air-to-Cloud

By nature of the protocol, UDP/IP flows are unreliable, yet heavily deployed in mission systems today. Toggling these flows to Real-time Transport Protocol (RTP) can greatly improve stream resiliency. RTP was designed for real-time transfer of audio/video information over IP networks. It includes several techniques that can better facilitate media-based delivery systems including timestamps, sequence numbers, and payload types, which can be helpful in addressing common issues such as late or out-of-order packets. For more information about how the protocol operates, reference IETF RFC 3550, a transport protocol for real-time applications or MISB ST 0804.4, a real-time protocol for motion imagery and metadata.

In addition to general packet structure enhancements found in RTP, the protocol also allows the use of optional Forward Error Correction (FEC) by sending parity packets alongside the payload that allow media decoders to reassemble lost/dropped packets on-the-fly. Some common RTP FEC schemes use a two-dimensional matrix organized into columns and rows based on Exclusive-OR (XOR) operations. While the Column FEC is single dimension (1D), the use of Columns and Rows is referred to as two dimensions (2D).  Generally, the Column FEC provides good correction for consecutive or burst packet loss while Row FEC provides good correction of non-consecutive random packet loss.  When combined, Column and Row FEC provides a robust error protection scheme capable of correcting for both types of packet loss experienced in a network. There are several well-documented schemes used by M&E experts typically ranging from a total of 10%-20% payload overhead, yet offering different levels of protection. These schemes may be further reviewed in SMPTE ST 2022-1-2007 Forward Error Correction for Real-Time Video/Audio Transport Over IP Networks. Notice that differences between schemes will vary in the required overhead and latencies encountered to accomplish the recovery capability provided. For the FEC schemes to operate properly, it is crucial that the packet error rate is equal to or less than the protection scheme implemented. For application firewalls and network access lists, remember that FEC payloads are commonly carried on Media Port n+2 and Media Port n+4 for columns and rows, respectively.

Figure 3. Example of an AWS Elemental Live Encoder using RTP with FEC with contribution settings applied.
Media = Port 50010
Column FEC = Port 50012
Row FEC = Port 50014

There are many ways to measure the general health and performance of a network to calculate for things like adding Forward Error Correction to your stream. One practice used by many solution architects and network engineers is the basic “ping” test, which uses a combination of ICMP control messages (Echo Request and Echo Reply) to measure specific conditions such as packet loss, latency, and throughput between a source and destination. Ping is a common command line utility found on most operating systems and when used appropriately can provide good metrics about network conditions. Ping is considered a network diagnostic tool, so be sure to consult with a network administrator before use.

Other forms of contribution encoding, such as ground-to-cloud, may be better off using more advanced packet protection methods that can guarantee media delivery. During Enterprise Challenge, for instance, the ground-based system terminated the aircraft RTP-FEC flow, then re-encapsulated it as a ground-to-cloud flow using reliable transport technology from Zixi.

The Zixi transport protocol provides added network resiliency through a combination of applied FEC and ARQ (Automatic Repeat Request) for advanced error-recovery. It is offered on the AWS Elemental MediaConnect service using ground and cloud-based link components for secure, reliable live video transport. Under this mode of operation, a provisioned latency value drives performance to build a sufficient error-recovery window. The AWS Elemental MediaConnect link should be configured to utilize approximately three times the round trip time between Ground and Cloud, although any relative provisioned latency may be set by the user. This is an excellent way to trade off minimal link latency in exchange for a more resilient contribution delivery system.

Distribution from Ground-to-User or Cloud-to-User

We’ve explored several ways to get live FMV from air-to-ground and ground-to-cloud, but what about serving the end user reliably? Unlike legacy architectures that used UDP multicast to disseminate flows to users, streaming technologies leverage the TCP/IP stack, providing a reliable connection-oriented flow and guaranteed packet delivery all the way to the user. This occurs by segmenting the UDP/IP flow into a series of smaller files, generally only a few seconds in length, and distributing them to client-side decoders based on broadly adopted HTTP techniques. Essentially, segments are downloaded by the client-side player, and precisely stitched together one after another, creating a consistent, smooth playback experience. If the client player takes too long to download a given segment or file, it automatically seeks a lower bandwidth rendition based on a master playlist or manifest file. In turn, this file instructs the client player if other renditions of the content exists and where to download them for faster playback. This is fundamentally how ABR streaming protocols work.

3. Consider New Ways to Carry Key-Length-Value (KLV) Metadata over ABR

While content may be king, in tactical missions metadata is queen. Video provides situational awareness, but it doesn’t offer insight as to where activity took place, which is where metadata comes into play. Full Motion Video streams generally contain associated geospatial metadata found in a data encoding standard known as KLV (Key-Length-Value) specified in SMPTE ST 336M Data Encoding Protocol Using Key-Length-Value. The underlying metadata, outlined in MISB ST 0601 UAS Datalink Local Set, contains specific location information about the video from the camera sensor or aircraft perspective (e.g. latitude, longitude, altitude, etc.).

FMV streams are generally constructed as programs within an MPEG-2 transport stream, with each program containing all of the information needed by the decoder application to make up a channel. In any given FMV channel, the program would consist of an elementary stream for video and an elementary stream for KLV. Transport streams have proven to be well suited for carrying real-time programs or channels over IP networks; they also provide important timing information, allowing the decoder applications to reassemble the elementary streams in a time-synchronized fashion for viewer playback.

In a hybrid FMV streaming approach, the same techniques would be applied on the contribution side (e.g. air-to-ground, ground-to-cloud). However, from a distribution standpoint FMV streams are transformed from traditional MPEG-2 transport stream encapsulation to newer adaptive streaming formats such as MPEG-DASH and HLS, utilizing the ISOBMFF or fragmented mp4 (fMP4) variant as the preferred transport container type. Since these streaming protocols were originally designed to serve M&E content over the internet (such as movies and TV), there was not a well-documented methodology to carry KLV information in the consumer market like there was with MPEG-2 transport streams. However, if KLV metadata were filtered out or lost in translation, FMV content would be less useful to an analyst, providing the most basic situational awareness without the descriptive geospatial data required for processing, exploitation, and dissemination workflows.

To overcome this obstacle and preserve the mission-oriented KLV metadata in an FMV streaming environment, AWS Elemental implemented the carriage of KLV in the fMP4 containers capable of facilitating many mainstream ABR formats in use today.

Currently under evaluation by the Motion Imagery Standards Board and greater Intelligence Community, an implementation of KLV using MPEG-DASH was demonstrated and proposed to drive new standardization practices for FMV streaming within the DoD/IC. Several industry partners have already integrated the proposed method within Processing, Exploitation, and Dissemination (PED) applications, decoder clients, and test and measurement tools with demonstrated success. KLV in DASH is enabling a new era of PED processing entirely from a web-browser by leveraging the latest ABR streaming protocols combined with broadly adopted KLV metadata standards used by the community.

Mission video systems are undergoing a dramatic evolution, as OTT and streaming technology developments open up new opportunities for government agencies to access, review, and analyze video from the field like never before. Taking the above considerations into account, government organizations can greatly improve FMV services feeding the enterprise and serving disadvantaged users, while maximizing network efficiencies across their Areas of Operation (AOR).

Figure 4 – Shows an OV-1 of a hybrid FMV streaming architecture with reliable contribution and distribution techniques.

To read more about this event refer to part one and part three of the EC19 blog series.