ResGVC: Residual Coding for High-fidelity Generative Video Compression
ResGVC attacks the two core weaknesses of current generative video compression (GVC): decoder-side diffusion without explicit source guidance risks drifting away from the original video, and latent-space denoising suffers irreversible detail loss through the autoencoder bottleneck. The paper proposes to decompose the distortion between a learned codec reconstruction and the original pictures into a sequence of pixel-space residual segments, one per flow-matching denoising step, and to compress those residuals with a diffusion-transformer entropy model. By transmitting only the most fidelity-critical segments and generating the rest, ResGVC claims to advance both perceptual realism and signal fidelity, reporting >60% LPIPS and >80% PSNR BD-rate reductions over prior art.
ResGVC is a technically creative and well-motivated framework that tightly couples residual transmission with pixel-space diffusion denoising. The reformulation of codec distortion as a sequence of flow-matching residuals is novel, and the skip-mode ablations provide a principled look at bitrate allocation across steps. Still, the reported BD-rate margins—63.7% on LPIPS and 83.8% on PSNR against the previous state of the art—are extraordinarily large for this mature field, and the absence of released code together with a complex five-stage training recipe severely limits reproducibility.
The limitations analysis is incisive: existing GVC methods are rightly criticized for relying on generative priors without sufficient source constraints, and latent autoencoders are identified as a bottleneck that discards fine details. The skip-mode design is strongly supported empirically—Table 4 shows that reserving residual coding for the first two denoising steps and generating the remaining three gives the best rate-distortion trade-off. Systematic ablations against both a CNN-based entropy model and a latent-space variant further bolster the claim that a pixel-space diffusion transformer is the right architectural choice for this task.
The paper provides no code release and its five-stage training protocol is described at a coarse level, omitting iteration counts, exact learning-rate decay policies, and gradient-clipping specifics. Training on OpenVid-1M—a dataset built for text-to-video generation—introduces a potential domain mismatch with the natural-video test sets (HEVC B, UVG, MCL-JCV) that the authors do not acknowledge or analyze.
Additionally, the 83.8% PSNR BD-rate reduction against the prior state of the art is an unusually large leap in the video coding literature and warrants caution. The authors justify using YODA as the BD-rate anchor rather than VTM-17.0 because the latter’s RD curves overlap only marginally with GVC methods, but this decision weakens the reader’s ability to judge absolute performance against an established standard.
The evaluation is broad, covering LPIPS, DISTS, FID, FloLPIPS, PSNR, and MS-SSIM, and the visual comparisons suggest that ResGVC preserves sharper textures than DCVC-FM while introducing fewer hallucinations than YODA or GLC-Video. However, the numbers for DiffVC and GNVC-VD are pulled directly from their original papers rather than from a common evaluation pipeline, which limits comparison fairness. The claim of being the first to explore residual coding in GVC is plausible given the surveyed literature, but difficult to fully confirm because many cited competitors are themselves unpublished 2026 preprints.
Although the appendix details the VideoPiT architecture and the analysis/synthesis transforms, critical hyperparameters—such as the number of training iterations per stage, precise learning-rate decay, and loss-weight annealing—are missing. The full system cascades a learned video codec with a five-step pixel-space diffusion residual codec and a skip-mode schedule that must be empirically tuned, all without an accompanying codebase, making independent reproduction extremely demanding. The test-time protocol of two residual-coding steps and three residual-generation steps is backed by ablation but offered without a principled rule for adapting it to other bitrates or resolutions.
Generative video compression (GVC) methods usually employ decoder-side diffusion models to denoise reconstructed signals based on generative priors. Besides, most of these methods perform denoising in latent space rather than directly in pixel domain. In this paper, we propose ResGVC, a residual coding framework for GVC that guides denoising toward original pictures by transmitting residual signals. ResGVC first encodes the original pictures within a learned video codec, and then decomposes the distortion between the codec reconstruction and the original pictures into a sequence of residual segments, where each residual segment is acquired at each denoising step. Residual segments can further be compressed by a novel residual codec with a diffusion transformer as an entropy model. Unlike prior methods, ResGVC introduces explicit source constraints throughout denoising rather than relying on decoder-side generation alone. The diffusion-based entropy model also enables more effective distribution modeling capability based on the powerful generative priors. Moreover, denoising directly in pixel space avoids the irreversible information loss caused by latent representation bottlenecks. Experimental results show that, compared with the previous state-of-the-art method, ResGVC achieves 63.7% and 83.8% BD-rate reductions in terms of LPIPS and PSNR, respectively, while also delivering superior visual quality.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.