Where Do Long Captions Fail? Position-Aware Diagnosis and Reinforcement Learning for Detailed Image Captioning
Detailed image captioning is usually treated as a global coverage task, but captions are generated as ordered sequences where later tokens can degrade in different ways. This paper introduces a position-aware rubric that decomposes captions into quartile-wise supported, hallucinated, and generic content, exposing two distinct failure modes: mid-caption unsupported specificity and tail generic escape. Motivated by this diagnosis, the authors propose PAGER, a GRPO reward with separate signals for middle evidence, tail density, and length control, achieving strong pairwise wins while preserving competitive global quality metrics.
The paper makes a clear conceptual advance: long-caption failure is not monotonic degradation but a substitution pattern where the tail becomes less evidential rather than more hallucinated. The diagnostic is well-supported across six model families and a same-reference human control, and the closed-loop design—diagnosing with the same rubric used for reward construction—is methodologically elegant. Main limitations include a reference-bound verification system that conflates unreferenced correct details with generic filler, a non-representative human-evaluation panel, and CapArena win rates so lopsided that they warrant closer scrutiny of baseline parity and evaluation variance. The DCScore-F1 trade-off with CapRL also shows the gains are not metric-universal.
The quartile-wise decomposition is the strongest element. The paper shows stable mid-caption hallucination peaks ($D_{\mathrm{mid}}$ positive) and tail generic spikes ($D_{\mathrm{tail}}$ positive) across Qwen, InternVL, and LLaVA families, while dense human references show none of these patterns. Ablations confirm the reward structure is doing targeted work: removing $R_{\mathrm{mid}}$ specifically degrades $H_{Q_2}$ and $H_{Q_3}$, while removing $R_{\mathrm{tail}}$ specifically hurts $S_{Q_4}$ and raises $G_{Q_4}$, exactly as the diagnostic predicts.
The verification framework is reference-bound, meaning a visually correct detail absent from the rubric is labeled Generic, and false contradictions can arise from imperfect references; the Generic residual remains a noisy category. Human audits and preference studies relied on unpaid volunteers recruited from the authors’ personal circle, which the paper admits is "not a representative panel of users" and carries unavoidable familiarity bias. The main CapArena comparison pools three very different baselines into a $\approx$96% win rate, yet DCScore-F1 still favors CapRL and the base-model CAPTURE margin is tiny, suggesting dominance is not absolute. Finally, the primary 4B result appears to come from a single training run; multi-seed replication in Appendix J.7 is limited to a 100-step short-budget transfer at larger scales.
The evidence supports the core diagnostic claims: positional effects are consistent across six models, confidence intervals exclude zero, and the same-reference control rules out artifacts. Comparisons to related work are fair and accurate. CapRL optimizes caption utility via VQA accuracy without position-specific signals, and PoSh localizes errors through scene graphs but does not model quartile composition or split rewards by region. Where the evidence strains is in attributing all improvement to the reward design alone: the filtered PixMo curriculum and content-aware length penalty likely contribute meaningfully to the CapArena margins, yet their isolated effects are not fully disentangled.
Judge prompts, extractor templates, and hyperparameters are documented in Appendices A and D, which aids reproduction for well-resourced labs. However, no code release is mentioned, the filtered PixMo curriculum depends on unstated quality-filter thresholds, and training requires roughly two days on 32 A100 GPUs. Using Qwen3-8B as the primary reward judge for a Qwen3-VL-4B policy raises potential self-reward bias, though the authors partially mitigate this with DeepSeek-V3.2 validation and human-majority labels ($\kappa=.89/.92$).
Detailed image captioning is usually framed as an amount problem: a model should mention more visual facts while avoiding hallucination. However, autoregressive VLMs generate captions as ordered sequences, and later generations may be produced under weaker or noisier visual grounding. We therefore argue that long-form captioning should be studied not only by what facts are mentioned, but also by where they appear. We introduce a position-aware rubric representation that decomposes dense human references into atomic visual claims, verifies generated captions against those claims, and induces a quartile-wise composition of supported, hallucinated, and generic content. This lens reveals two qualitatively different failures in long captions: models tend to produce unsupported specificity in the middle, while the final portion often escapes into vague, non-evidential language rather than simply accumulating more hallucination. A same-reference human control does not exhibit these patterns; additional human audits examine the visual validity of the induced labels. Motivated by this diagnosis, we propose PAGER (Position-Aware Grounded Evidence Reinforcement), a GRPO training objective with separate signals for supported fine-grained middle evidence, evidence-bearing tail continuation, claim coverage, and content-aware length control. The result is a closed loop: the same ordered claim representation identifies where long captions fail and supplies the position-specific reward needed to repair those failures. Experiments on detailed-caption benchmarks show that PAGER improves position-wise grounding and pairwise caption preference while preserving competitive global caption quality.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.