FAQ
What's the main idea of GRACE?
In real-time video streaming, packet loss can severely impair user experience by causing video freezes or distortion. Although there are several existing solutions such as Forward Error Correction (FEC), and error concealment, these primarily enhance loss resiliency either at the sender or decoder end. GRACE introduces a novel approach by jointly optimizing both the encoder and decoder to handle various packet loss scenarios. This method uses the neural video codecs jointly trained under simulated packet losses to maintain decent quality in decoded frames despite packet losses. It enables direct decoding of loss-impacted frames without waiting for retransmissions whiile ensuring high video quality.
Why can Grace have better frame quality when loss happens?
GRACE’s encoder and decoder are trained together to handle different packet loss conditions. This joint training enables the decoder to effectively reconstruct missing data from available information, maximizing image quality under (e.g., SSIM) packet losses. Concurrently, the encoder learns to incorporate additional data that aids the decoder in recovering lost content. The joint-training approach is critical for achieving such results, and such loss-resiliency is not possible when the encoder and decoder are trained separately.
Why can GRACE achieve much smoother video playback?
Traditional real-time streaming codecs fail to decode frames when packet loss occurs, especially with dependency on previous frames (P-frames), causing subsequent frames to also be undecodable until the lost packets are retransmitted. In contrast, GRACE can decode frames impacted by loss with decent quality, eliminating the need for retransmissions and avoiding video stalls.
Does Grace require specialized congestion control algorithms?
No, GRACE is compatible with state-of-the-art congestion control algorithms used in real-time video streaming, such as Google Congestion Control (GCC). Notably, GRACE may perform even better with more aggressive congestion control algorithms.