Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes
Where-OPD proposes a new on-policy self-distillation paradigm for multimodal large language models. Instead of giving the teacher zoomed-in image crops, it provides textual spatial hints that identify question-relevant visual elements and their coordinates, using procedurally generated synthetic counting scenes where object locations are known by design. The student learns to match the teacher’s behavior from the raw image and question alone, and the authors claim that training on just 3,000 such synthetic scenes transfers to broad real-world perception gains across 15 benchmarks.
The paper demonstrates that spatially-grounded textual privilege can yield broader unweighted average improvements than crop-based visual privilege, and the core ablations support the claimed importance of spatial guidance. However, the evaluation strategy raises concerns: the default thinking-disabled protocol substantially suppresses baseline performance on reasoning-sensitive tasks such as CountQA (32.00% disabled versus 46.86% with thinking enabled for the Qwen3.5-4B base model), and the mixed use of multiple-choice questions for 4B models versus open-ended questions for the 9B model complicates cross-scale comparisons. While the synthetic-to-real transfer is real on paper, the mechanism by which geometric counting scenes induce chart-reading and hallucination-reduction capabilities remains more asserted than explained.
The ablation in Table 2 is the strongest evidence: removing the hint leaves OPSD at the base-model average, while providing only the answer is weaker (71.27) than providing spatial coordinates without the count (72.35), and the full spatial hint achieves the highest average (72.94). The authors also forthrightly report that a frozen teacher outperforms an EMA-updated one, writing that "updating the teacher on synthetic images may introduce a slight domain shift, diminishing the overall gains." In addition, the per-category breakdown in Appendix B.1 demonstrates that improvements are not driven solely by counting questions, with gains on CVBench distance (+3.33), V* direct attributes (+7.83), and HallusionBench math (+27.78) categories.
The default evaluation uses thinking disabled, which severely handicaps the base model on tasks requiring intermediate reasoning; Table 3 reveals that when the best of thinking enabled or disabled is reported per cell, the base model's CountQA jumps to 46.86% and its overall average to 77.06%, shrinking the apparent margin of improvement. The training protocol is also inconsistent across scales: "Qwen3.5-4B and Qwen3-VL-4B receive a multiple-choice question, while Qwen3.5-9B receives an open-ended version."
Furthermore, the claim of broad perceptual transfer from 3,000 procedurally generated geometric scenes to document understanding, chart reasoning, and real-world hallucination benchmarks is striking but mechanistically thin. The qualitative attention maps in Figure 4 are cherry-picked, and no systematic attention analysis is reported on the full benchmark suite, making it hard to verify that spatial guidance truly induced cross-task visual-evidence alignment rather than simply regularizing the model toward more cautious, checkable reasoning.
Baseline comparisons are largely fair in that prior OPSD methods are evaluated from shared or author-provided checkpoints under an identical pipeline. Yet the framing should acknowledge that crop-based methods like Vision-OPD are explicitly designed for fine-grained zooming, so their gains on V* (+8.90) and ZoomBench (+14.56) and drops on CountQA (-10.73) are expected trade-offs rather than failures. Where-OPD achieves the highest unweighted average across all 15 benchmarks on every model, but its high-resolution gains are smaller than those of crop-specialized competitors, and it degrades slightly on HR-Bench 4K for Qwen3.5-4B (-0.63), suggesting its broader average comes partly from avoiding sharp losses on non-zoom tasks. As Vision-OPD describes its own objective, it "minimizes token-level divergence between the teacher and student next-token distributions along these rollouts," confirming the two methods target different perceptual bottlenecks.
Training hyperparameters and the procedural generation recipe are described in sufficient detail to attempt reproduction, and the authors report that training takes only 1.4–5.8 hours on 4 H100 GPUs. However, full reproducibility depends on exact scene-generation code, which is not released in the paper itself despite the GitHub project page link. A bigger risk is the evaluation pipeline: "Any response that neither stage resolves is passed to an LLM judge, which compares the response with the reference answer and returns a binary verdict," but the judge model, its prompt, and its agreement rates are not disclosed. Finally, the mixed multiple-choice versus open-ended protocols across model sizes and the non-standard default of thinking-disabled evaluation are confounders that any independent reproduction must carefully replicate.
On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a frozen or EMA version of themselves that receives privileged information. Its application to multimodal large language models (MLLMs), however, remains largely unexplored. Recent approaches use privileged visual information, such as image crops corresponding to a question, to improve fine-grained perception, but their gains are confined to tasks that benefit from such visual zooming and require either human-annotated grounding data or external teacher models. We introduce a different form of on-policy self-distillation for MLLMs that provides the teacher with textual, spatially grounded guidance identifying the visual elements relevant to a query. We use procedurally generated scenes with automatically available object identities and spatial coordinates, enabling scalable and annotation-free post-training. The teacher uses this spatial guidance to locate and integrate evidence from multiple relevant image regions, while the student learns to reproduce the resulting behavior from the image and question alone. Our approach consistently improves performance on counting, document and chart understanding benchmarks across multiple models. Importantly, although post-training uses only synthetic scenes, the resulting improvements transfer to real-world perception benchmarks, yielding a 3.23-point gain in average performance across CVBench, V*, ZoomBench, BLINK, HR-Bench, and MME-RealWorld. These results show that spatially grounded privileged information can induce broader perceptual capabilities through on-policy self-distillation, enabling substantial synthetic-to-real transfer beyond the task and data distribution used for post-training. Project page: https://github.com/sirkosophia/Where-OPD
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.