A Matched-Budget Audit Framework for Recaptioned Image-Text Supervision Distributions
This paper attacks a growing bottleneck in text-to-image training: how to audit billion-scale recaptioned corpora before spending millions of GPU-hours on downstream training. It proposes a matched-budget audit framework that fixes a text budget $B=64$ and profiles any supervision distribution $D_{\pi,V_c,C}$ across five axes—prompt-side coverage, image-conditioned faithfulness, and caption-surface health—using controllable basic units (CBUs) as a common claim currency. The work matters because it disentangles corpus quality from model-training confounders and releases both a ~490M-caption corpus and a reusable artifact bundle.
The paper presents a conceptually clean and practically valuable framework for pre-training corpus audit. The five-axis profile and CBU abstraction are well-motivated, and the release of ~490M captions with full artifact bundles is a strong community contribution. That said, the evaluation is bounded by a single captioner family (Qwen3.5-35B), heavy reliance on VLM judges with known failure modes, and an absence of downstream T2I experiments that would validate whether higher CBU yield actually improves generation. The human-verification sample is also tiny relative to corpus scale.
Fixing a lexical budget $B=64$ is the strongest design choice: it isolates policy and captioner effects from the trivial length confound that makes longer captions look richer. The same-captioner ablation is compelling; with $V_c$ held constant, switching from a plain detailed-caption instruction to the proposed prompt-register policy raises supported CBU per caption by +3.0 to +3.6. The surface-health diagnostics—opener rate, prefix mass, and distinct-3—are cheap, interpretable, and clearly expose describe-style frame concentration in reference corpora.
First, the audit deliberately stops before T2I training, so the central thesis—that better CBU profiles yield better generators—remains unsupported. The authors note that 99.8–99.99% of their captions exceed the 77-token CLIP context, which means any downstream comparison would be dominated by encoder truncation and augmentation policy rather than corpus quality alone.
Second, the risk metric $\rho$ conflates captioner hallucination, claim-extractor error, and judge inaccuracy into a single scalar. The two Judges agree least on count and text-rendering claims (85.1% and 87.6%), and the human-verification sample covers only 137 claims, providing weak anchoring for a 490M-image corpus.
The evidence supports the internal claims well. The seven paired comparisons hold source rows fixed, isolating the caption surface, and the cross-corpus results are directionally consistent across both Qwen and Gemma Judges. However, the comparison against the short tag-style surface CC12M-Qwen3-VL† reveals a tension the paper does not resolve: that surface achieves the highest CBU per 100 lex and the lowest risk, yet the authors advocate for longer, denser captions without demonstrating that their chosen frontier point is superior for T2I training. Prompt-pool support comparisons use public community prompt sets, which is fair, though the pools are limited to natural-language prompts and omit tag-style distributions.
Reproducibility is a clear strength. The authors release the full corpus (~490M image identities), code repository, lexical tokenizer, CBU schema, judge schemas, prompt pools, request templates, and Croissant/RAI metadata. Computational costs are transparently reported at ≈12.7k H200 GPU-hours for caption generation plus roughly $10^2$ GPU-hours for sampled audits. The framework is corpus-agnostic by design, so third parties can apply it to smaller targets without regenerating the full release. Hyperparameters for greedy/schema-constrained decoding are documented in Appendix H, though temperature and top-p are not stated in the main text.
Recaptioned image-text corpora are now standard for text-to-image (T2I) training, with vision--language model (VLM) captioners replacing sparse alt-text by dense descriptions. A recaptioned corpus is a supervision distribution induced by a documented captioning policy ($\pi$), captioner ($V_c$), and source corpus ($C$). Length-correlated proxies miss caption-register artifacts and downstream T2I benchmarks entangle the corpus with training choices, so this distribution is hard to audit at corpus scale. We introduce a reusable matched-budget audit framework for recaptioned supervision distributions $D_{\pi,V_c,C}$: at a fixed text budget of $B = 64$ it reports a five-axis profile spanning prompt-side coverage, image-conditioned faithfulness, and caption-surface health, with claimed controllable basic units (CBU) as the common claim unit. We instantiate the framework on seven paired comparisons over five public source corpora. Across the four cross-corpus pairs, the released surface raises supported CBU per caption by $+3.39$ to $+6.36$ under both Qwen and Gemma Judges, and on CC12M the same framework exposes a long-vs-dense frontier that is consistent across both judges and four budgets. We release the audited multi-source recap corpus ($\approx$ 490M) together with the audit-artifact bundle.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.