To enhance productivity and collaboration, the Amazon Studios Technology team was looking for a way to transmit on-set camera feeds and graphical workstation outputs between on-premises locations and the cloud, so that colorists, editors, and other creative workers could view content in real time - from any location, at any location - with the lowest latency and highest image quality possible (including 4K UHD, HDR, and 4:4:4 color formats).
Video compression techniques can generally be defined by four balanced factors: quality, complexity, latency, and data rate. Enhancing any one of these factors inevitably leads to impairment of one or more of the others. In this case, quality was very important. Creative workers required precise and true representations of production imagery. Latency was important, as remote collaborators needed to respond to and communicate about the imagery in real time.
As many video processing applications would be on the cloud, complexity was also a key issue, as the solution needed to run on typical virtual cloud instances without specialized hardware. Uncompressed 4K UHD video with data rates as high as 10 Gbit/s would have been impractical to transport, thus some compression was required to realize moving a large number of live streams at scale both locally, between remote locations, and to/from the cloud. But the data rate compression level could be less than required for typical end-user over-the-top (OTT) video.
The JPEG XS codec (ISO/IEC 21122) has been developed as a new standardized low-latency and low-complexity video codec optimized for visually lossless compression. It’s very promising for high-quality mezzanine use cases, including sending live video streams between on-premises video systems and the cloud with compression ratios from 5:1 to 10:1.
The JPEG XS codec is not directly related to the JPEG codec (ISO/IEC 10918), which uses the discrete cosine transform (DCT) and is commonly used on the web. It is also not directly related to the JPEG 2000 codec (ISO/IEC 15444), a wavelet codec widely used for live contribution encoding for broadcast as well as digital cinema. Instead, JPEG XS is a unique wavelet codec, developed to deliver coding efficiency similar to that of JPEG 2000, but with sub-frame latency and using algorithms that can be readily implemented in software. The design goal of JPEG XS was to have a complexity such that 4K 60p video could be encoded on an Intel Core i7 or similar processor in real time. The efficient implementation of JPEG XS in software makes it very useful in cloud workflows.
One of the advantages of a standardized video codec is the ability for customers to pick an encoder from a range of vendors that best meets their particular needs. However, there is always a risk that vendors of a new codec may have divergent implementations due to differences in interpretation of parts of the standard. To reduce the risk of non-interoperable implementations, customers and organizations often collaborate in interoperability workshops (also known as “interops”), where multiple vendors can come together, exchange live video traffic, and ensure that their systems work together well.
On March 31 and April 1, 2022, Amazon Studios in Culver City, CA, hosted a JPEG XS interop, with support from Amazon Web Services (AWS). Due to the COVID-19 pandemic, this was the first time an in-person JPEG XS interop was possible. Eleven equipment vendors attended, and exchanged JPEG XS video streams over a 100 gigabit Ethernet LAN. JPEG XS video streams were also sent to AWS Elemental MediaConnect over AWS Direct Connect (a dedicated, private network connection to the cloud). Over two hundred different interoperation tests were ultimately performed between the participants. These tests included interlaced HD, progressive HD, and UHD video stream formats.
To memorialize and preserve results from the 48 hours of testing, the event concluded with a representative sampling of video “interoperations” which were incorporated into an offline multi-view recording as shown in the following image.
Jonathon Lee, Head of Media Engineering and Innovation for Amazon Studios, said “Amazon Studios was proud to host this first in-person JPEG XS interoperation workshop at our new facilities at the historic Culver Studios. As our productions scale globally, it becomes essential that we enable our creative and post-production teams to collaborate in real time, in different locations, using video of the highest quality, color accuracy, and lowest latency. It is therefore critical that interoperability of the JPEG XS video standard, across multiple vendors, both on-premises and in the cloud, is tested and validated. It was great to be able to support this effort, along with our AWS counterparts.”
Amazon Studios and AWS extend their sincere thanks to: Appear, Evertz Microsystems, Imagine Communications, Grass Valley, Matrox Graphics, Media Links, Nevion, PacketStorm Communications, Riedel Communications, Sencore, and TAG Video Systems for making this event a success.
Andrew Osmond, Director of Business Development for Evertz Microsystems, said “This JPEG XS interop provided a solid first step in validating the VSF TR-08 specification for JPEG XS across various vendors in the industry. This is a very important step for wider adaptation of JPEG XS and TR-08. We are pleased to have participated in this important work for the industry.”
Geoff Bowen, Chief Technology Architect for Appear, said “Appear were beyond pleased with the results from the first ‘in person’ interop of JPEG XS essences. This is essential work for accelerating customer confidence and adoption of JPEG XS ultra-low latency workflows for both managed network and ground-to-cloud contribution applications.”
For more information about JPEG XS on AWS Elemental Live encoders, AWS Elemental MediaConnect, or how AWS or AWS Partner technology and integrations can be used in live video workflows, see the AWS Elemental Live introduces JPEG XS for low-latency, visually lossless contribution to the cloud article on the AWS Media Blog.