Two Clocks in Diffusion MLLMs: When Answers Stabilize Before Rationales Unfold
This paper studies masked diffusion multimodal language models and shows that answer candidates often stabilize retrospectively long before their positions are permanently committed or the rationale-side canvas is filled. The authors introduce two clocks—token stabilization $T_{\mathrm{stab}}$ and commitment $T_{\mathrm{commit}}$—and measure how much of the rationale region $\mathcal{R}$ remains unwritten at stabilization. They pair this with matched-canvas visual-sensitivity ablations and a coverage-accuracy decomposition to explain why prompting changes affect overall accuracy. The analysis matters because it reveals that observing the final rationale alone is insufficient to understand when or how a diffusion model decides its answer.
The paper delivers a precise empirical characterization of generation dynamics in diffusion MLLMs, clearly separating answer stabilization, commitment, rationale unfolding, and visual grounding. Its strongest contribution is methodological: defining measurable quantities that expose the hidden order of diffusion decoding and using an exact algebraic decomposition to show that coverage changes dominate conditional accuracy changes across five prompting comparisons. The analysis is careful, well-scoped, and honest about limitations.
The distinction between retrospective stabilization of the logged candidate and permanent commitment is conceptually sharp and empirically well-executed, with 89.4–98.1% of the rationale-side canvas remaining unwritten at stabilization under the primary LaViDa condition. The matched-canvas PDM-H protocol cleanly separates visual-sensitivity peaks from answer clocks, and the symmetric decomposition $A_D - A_R = (C_D - C_R)\frac{Q_D+Q_R}{2} + (Q_D - Q_R)\frac{C_D+C_R}{2}$ gives an elegant accounting of prompting effects. Together, these measurements make abstract faithfulness questions concrete.
Sample sizes are modest (191–400 examples) and all main timing and prompting results are reported as point estimates without confidence intervals or standard errors, making it hard to judge variability. The block-length control manipulates schedule, context eligibility, coverage, and accuracy simultaneously, so attributing effects cleanly to a single cause is difficult. The coverage-accuracy decomposition is an algebraic identity, and characterizing coverage as the ‘larger component’ does not itself establish that coverage is the causal lever; confounding between instruction content and format constraints is acknowledged but not isolated. The authors also note that no code is released, which limits reproducibility.
The evidence directly supports the core claim that answers stabilize early relative to rationale commitment in the tested single-block LaViDa runs, and the block control credibly shows that changing the schedule changes the observable unwritten fraction and coverage. Comparisons to prior work are fair and specific: Li et al. and Yeom et al. are cited for early convergence and commitment order, while Thinking Diffusion and Favero et al. motivate the PDM-H visual-sensitivity readout. The paper does not claim superiority over autoregressive models, staying within its empirical scope. However, generalization beyond the three small benchmarks and three specific model configurations is not demonstrated.
Experimental settings are documented in detail: Section 4 and Appendices A–C specify decoding hyperparameters, prompt templates, data interfaces, scoring rules, and compute (eight NVIDIA H100s). However, the absence of released executables means independent reproduction would require reimplementing wrappers and trace loggers, which is a significant barrier. Small evaluation sizes (e.g., 191 for V*Bench) and the lack of reported error bars or multiple runs further reduce confidence that independent reproductions would recover the exact point estimates.
An answer candidate in a masked diffusion MLLM can stabilize while its rationale is still unfolding. We distinguish retrospective stabilization of the logged candidate from token commitment, and examine these two clocks relative to rationale generation. Analyzing our results across three visual question-answering benchmarks, we find that 89.4-98.1% of the rationale-side canvas remains unwritten at stabilization in single-block, EOS-suppressed LaViDa runs. On V*Bench, reducing block length from 128 to 8 changes this fraction from 89.4% to 1.7%, together with answer coverage and the eligible observation window. Under EOS-enabled prompting, direct instructions improve Nemotron's overall accuracy by 15.0 and 19.5 percentage points on M3CoT and ScienceQA, but reduce LaViDa/V*Bench accuracy by 11.0 points. A symmetric decomposition associates the larger absolute component of each change with coverage rather than conditional accuracy. Matched-canvas image ablations measure visual sensitivity alongside answer stabilization, separating the two temporal readouts. Together, these measurements distinguish answer stabilization, rationale unfolding, and visual sensitivity, and identify coverage as the larger component of the prompting differences.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.