Temporally-Resolved Token Attribution Reveals the Generation Dynamics of Diffusion Language Models

The axiomatic grounding is the strongest element. By instantiating the scalar map as $G_t^{(\ell)}$, the authors show that DLIG inherits IG's completeness, implementation invariance, linearity, and symmetry preservation exactly (Section 3.1). The empirical verification is equally careful: completeness is checked with hook-transparency, convergence from $m=200$ to $m=1000$, and a bit-exact partial-forward spot check across the full layer-and-step grid (Appendix A.1). The ProsQA analysis is particularly compelling—"Gold-path precision is nearly identical for successes and fails and carries no signal in a joint logistic model" (Section 6)—demonstrating that attribution localization survives even when the final answer is wrong, a genuinely surprising finding that distinguishes reading the right facts from executing the reasoning correctly.

The paper relies entirely on a single fine-tuned DiffuGPT-M (355M) checkpoint for all tasks, leaving open whether the dynamics generalize to larger, natively-trained DLMs or continuous-space and hybrid families. The underlying model's task performance is modest—WiC 63.6% overall with severe Yes-bias (90.8% same-sense vs. 36.4% different-sense), ProsQA 55.0% success, and ROCStories ROUGE-1 12.02 (Appendix A.2, Table 1)—so the interpretability findings describe a relatively weak system. Additionally, the self-generated scoring framing introduces circularity: the attribution target is the model's own emerging completion, and 22.3% of WiC steps carry zero attributable mass because no output has been committed yet (Appendix B, Table 7), making early-step dynamics unstable. While the authors frame DLIG as a hypothesis-generation tool, the absence of any causal validation—e.g., activation patching on highly attributed tokens—means the "mechanistic hypotheses" remain unverified.

The evidence supports the narrower claim that DLIG produces coherent, complete attributions and generates non-obvious hypotheses. The comparison to Nguyen's attribution-contract taxonomy and Yao et al.'s temporal causal tracing (Section 2) fairly positions DLIG as a lightweight complement to expensive interventional analyses. However, the paper does not benchmark DLIG against simpler baselines such as layer-wise attention rollout or gradient norms, so readers cannot gauge whether the computational cost of step-wise IG integration is empirically justified relative to cheaper heuristics. The fixed-target versus self-generated analysis in Appendix A.4.1 is commendably candid, noting that "on high-ROUGE stories the fixed gold target lies close to the model's actual trajectory and receives elevated gradient signal for that reason alone, independent of how much the model relied on context," which means framing choices can inflate apparent correlations.

The authors release code and model checkpoints, and hyperparameters, training splits, and prompt templates are documented in Appendix C. The main analysis uses $T=64$ denoising steps with $m=12$ Riemann points, and completeness is checked at finer resolutions (Appendix A.1). Reproduction is tractable with the DiffuGPT-M checkpoint. Blockers include the undisclosed random seed for the WiC/ProsQA test splits carved from public training data, and the ProsQA parsing pipeline that matches model generations to graph facts via fuzzy string matching involving manual templates (Appendix A.3). The implementation checks—confirming that "endpoint scores recovered through the hook match the plain forward pass to floating-point precision at every grid cell" and exact partial-forward agreement—go a long way toward ensuring correctness (Appendix A.1).