Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models
The paper challenges the widespread assumption that reinforcement learning (RL) invents the reflection marker "wait" in reasoning models. Using historical-linguistics-inspired "grammaticalization" tracking, the authors trace the word through 120 open-model checkpoints and find it almost always arrives from training data—mid-training, fine-tuning on reasoning traces, or reasoning-rich pretraining—while zero-RL alone rarely coins it.
The central claim is well-supported but not absolute: zero-RL from bases that lack "wait" almost never creates the marker. Across 18 zero-RL runs on 9 Qwen2.5, Mistral and Qwen3 bases, accuracy rises by a median of 42.9 MATH-500 points while the marker stays below 0.05 per 1k tokens in 17/18 runs. The paper's takeaway is direct: "Zero-RL rarely installs wait. In all but one of 18 runs from a base without it, RL raises accuracy while the marker stays below 0.05 per 1k tokens." The grammaticalization framework (frequency, fixation, erosion, bonding, usage drift) plus an injected-error alarm test ($d'$ and $c$) is a genuinely novel methodology, though several preregistered predictions failed and the evidence is observational rather than causal.
The label-free analytical toolkit is the paper's strongest contribution. The authors import grammaticalization theory (Bybee, Lehmann, Hopper & Traugott) to construct indices—frequency per 1k tokens, sentence-initial fixation $\mathrm{SI}_m(t)$, orthographic form erosion, chaining, and usage-drift via Jensen–Shannon divergence—that require no LLM-as-a-judge or human behaviour labels. The injected-error alarm test is also elegant: by comparing $d'(t) = z(\mathrm{HR}) - z(\mathrm{FA})$ and $c(t) = -\frac{1}{2}[z(\mathrm{HR}) + z(\mathrm{FA})]$ across checkpoints, they show that under OLMo-3 RL-Zero the marker becomes "a more sensitive but more liberal alarm." The scale (120 endpoints) and statistical care (cluster bootstrap over problems, Holm correction, Poisson intervals) are commendable.
The evidence is observational, not causal. The compared lineages differ in data and recipe alongside the marker, and each open checkpoint series is a single training run, so intervals capture only evaluation sampling variance, not training variance. The most damaging empirical challenge is General-Reasoner-Qwen3-4B, which the authors flag as "the exception": this zero-RL run goes from <<0.01 to 0.29 wait per 1k tokens, breaking the 17/18 pattern. They hypothesize the marker was "latent" in the base, but this is post-hoc. Additionally, "Several predictions we registered did not hold," including their preregistered prediction that the marker would "bleach" (lose its error signal). Instead, the injected-error test shows $d'$ rises and the marker remains linked to errors. The scope is also narrow: all RL runs use outcome rewards on English mathematics; frontier scale, process rewards, and other domains are untested.
The evidence supports the headline claim that training data seeds the marker while RL reshapes it, but the single exception and observational design prevent a watertight conclusion. Comparisons to related work are fair and precise: the authors correctly distinguish their contribution from prior studies that locate reflective *behaviour* before RL (Liu et al., Gandhi et al., Essential AI) by explicitly tracking the *lexical form* "wait" across open lineages. They also effectively criticize the use of LLM judges for behavioural labelling, showing one judge reversed the human-labelled trend despite moderate item-level agreement ($\kappa=0.64$). Their argument that "marker frequency alone does not indicate RL-induced reasoning" is a useful caution to the growing literature on chain-of-thought monitorability.
The study uses only publicly released checkpoints and standard benchmarks (MATH-500, AIME, GSM8K), which strongly aids reproducibility. Hyperparameters and sampling protocols are documented in appendices B and D. However, the authors state their code and provenance map are hosted at an anonymous link (https://anonymous.4open.science/r/wait-provenance/), which is inaccessible outside the review process and will block immediate independent reproduction until a permanent release. All reported intervals are 95% cluster-bootstrap over problems, but because training trajectories are single runs, the uncertainty does not include training variance—a significant caveat for anyone attempting to replicate the exact RL trajectories.
Reinforcement learning (RL) is credited with teaching reasoning models to write {wait} before reconsidering a step, yet large RL accuracy gains do not require the word. This view can confuse what RL discovers with what a model inherits from its training data. We trace the word through open models' training histories, reading it with label-free indices of frequency, position, form and usage and an injected-error test, all computed from sampled outputs. We find that zero-RL almost never coins the marker: on bases that lack the word, RL raises MATH-500 accuracy substantially while {wait} stays near absent. It arrives with training data: mid-training in OLMo-3, and fine-tuning on reasoning traces installs it in most models whose base lacks it. Where the base already uses it, both released RL-Zero recipes (OLMo-3, MiMo) multiply it, whereas KL-free GRPO from the same base does not; OLMo-3's recipe wears down its form and makes it a more sensitive but more liberal alarm; RL after supervised fine-tuning barely changes it, and our GRPO recipe erases a marker seeded by fine-tuning. Finally, we show that marker frequency alone does not indicate RL-induced reasoning and caution against LLM-as-a-judge revision metrics, as automated judges can reverse human-labelled trends.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.