Towards Subject Consistency over Dynamic Subject Sets in Video Generation
As video generation scales to minute-long clips, subjects enter and leave the scene, making static-set consistency metrics inadequate. This paper introduces DynSC-Eval, a framework that tracks each subject over its entire visible lifespan and decomposes consistency into local continuity and global identity preservation via six object-level metrics. The authors further derive rewards from these metrics and use DiffusionNFT to post-train video models, demonstrating substantial consistency gains in an autonomous driving testbed and extending them to 30-second generation through curriculum learning.
The paper makes a compelling case that evaluating consistency over dynamic subject sets is essential for long-video generation. Synthetic experiments and broad model comparisons convincingly show that DynSC-Eval is more sensitive and discriminative than holistic frame-level metrics like VBench. However, the optimization experiments are confined to autonomous driving, and the paper does not analyze how detection or tracking errors affect DynSC-Eval scores, leaving general-domain robustness and pipeline reliability as open questions.
The controlled synthetic experiments provide rigorous validation: SR tracks the injected inconsistency fraction ρ with R^2 = 0.995, while VBench SC remains nearly flat. Evaluating diverse models from 5 s to 60 s demonstrates that DynSC-Eval separates models with similar VBench scores and exposes complementary local and global failure modes. The ablation further confirms that combining local and global rewards is necessary to avoid trivial solutions such as suppressing motion.
The post-training results are limited to an autonomous driving testbed, which, while practically important, does not establish that the rewards transfer cleanly to general open-domain video. The paper does not report ablations on the severe appearance mutation threshold τ or quantify how SAM2 tracking failures influence the metrics, making it unclear whether DynSC-Eval robustness holds when detectors err.
Additionally, although curriculum learning extends optimization to 30 s, absolute inconsistency remains severe, suggesting the approach mitigates rather than solves long-horizon drift. The reliance on VBench quality scores to claim preserved capabilities is also slightly at odds with the central critique of VBench's insensitivity.
The evidence supporting DynSC-Eval over VBench is strong: under joint local and global perturbations, DynSC-Eval metrics rise predictably while VBench SC is insensitive. The paper does not, however, run head-to-head comparisons against recent subject-level trackers such as T2V-CompBench under the same dynamic-subject conditions, so the margin of improvement over those methods is asserted rather than quantified.
The synthetic ablation cleanly isolates local and global perturbations, showing that local metrics barely respond to global drift and vice versa, which supports the claim of complementary coverage. The reliance on VBench quality scores to claim preserved capabilities is ironic given the paper's core argument against its insensitivity.
The authors provide detailed metric definitions, synthetic construction protocols, and training hyperparameters in the appendices, including exact learning rates and batch sizes for fine-tuning. Yet no code or precomputed data is mentioned as publicly available, and key DiffusionNFT implementation details are deferred to Appendix C, which was truncated in the provided text. Reproduction therefore requires re-implementing the full DynSC-Eval pipeline around SAM2, Grounding DINO, and DINOv3, along with sourcing the specific OpenDV splits.
We argue that as video generation extends to longer durations, subject consistency should be evaluated over \textit{dynamic subject sets}. We therefore introduce \textbf{DynSC-Eval}, an evaluation framework that dynamically tracks eligible subjects throughout their visible lifespans and measures local continuity and global identity preservation using six complementary object-level metrics, with explicit detection of inconsistency events. To validate its effectiveness, we design synthetic experiments that actively inject inconsistency events, demonstrating both the sensitivity of DynSC-Eval and the limitations of existing metrics. Evaluations of diverse models on 5s, 15s, and 60s video generation further reveal substantial subject consistency differences that are obscured by conventional metrics. Beyond evaluation, we construct rewards from DynSC-Eval and apply DiffusionNFT post-training in an autonomous-driving testbed. On 5s generation, our approach reduces the six inconsistency metrics by an average of 13.82\% for Wan-2.1-1.3B and 5.66\% for SANA-2B, with improvements also observed on the I2V model ReSim. Qualitative comparisons further demonstrate the effectiveness of our method. We then extend generation to 10s and 30s through curriculum learning and show that consistency optimization remains effective while largely preserving other capabilities.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.