Persistent Depth Ordering amid Shifting Block-Bypass Responses in Language Model Pretraining
Most interpretability studies ablate or probe a single training checkpoint, leaving it unclear whether observed depth-dependent sensitivities are stable or transient. This paper tracks whole-block identity bypass across five released pretraining trajectories and eleven model-domain combinations to ask what persists and what shifts. The core finding is that the relative depth ordering of block sensitivities remains recognizable even as the magnitudes redistribute, and that the intervention pathway itself evolves differently across models and perturbation scales. The work matters because it cautions against treating single-checkpoint ablation profiles as static descriptions of a model's internal organization.
The paper offers a carefully executed, large-scale longitudinal characterization of layer bypass responses, combining broad profiling with controlled matched-perturbation experiments. Its central descriptive claim, that depth ordering persists while individual block responses change, is well supported by the data. However, the study is necessarily phenomenological: whole-block bypass is a blunt instrument, the evaluation is restricted to teacher-forced next-token metrics on small fixed contexts, and independent replication is concentrated in a single small model. As the authors note, the results "do not identify semantic circuits, causal shares, targeted compensation, or safe joint pruning" (Section 5), so readers should treat the findings as structured phenomenology rather than mechanistic explanation.
The strongest support comes from the breadth and consistency of the profile data. Across all combinations, adjacent-checkpoint Spearman correlations are high (median $\rho = 0.906$), and even endpoint correlations remain positive ($\rho = 0.380$ to $0.785$), showing that "depth ordering can remain recognizable even while individual block responses change substantially" (Introduction). The cross-domain transfer analysis is also compelling: "A template constructed from two domains correlates with the held-out third domain at a median 0.944" (Section 4.2), suggesting that redistribution patterns are not domain-specific artifacts. The controlled Pythia and OLMo-2 experiments add value by separating local perturbation magnitude from downstream response, revealing that these components can move in opposite directions.
The intervention design is coarse and limits mechanistic insight. Whole-block identity bypass removes an entire residual update, so the study "does not isolate semantic circuits, and separate bypass costs do not establish safe joint pruning" (Section 5). The evaluation is narrow: "The experiments measure next-token response on fixed teacher-forced text, rather than mathematical/code correctness or free-running generation" (Section 5), and the broad profiles rely on only 18 windows of 257 tokens per domain.
Replication is thin for larger models: "The broad comparisons cover single released trajectories; independent-run replication is concentrated in Pythia-160M" (Section 5). Finally, the preregistered direction-control test for targeted downstream compensation failed: "Neither checkpoint meets the preregistered joint criterion for targeted natural-direction compensation" (Appendix F), weakening the claim that the intervention pathway has been finely characterized.
The statistical evidence matches the descriptive claims. The Spearman trajectories, held-out window concentration, and cross-domain correlations are reported with bootstrap intervals and appropriately cautious language. Comparisons to related work are fair: the authors distinguish their longitudinal focus from prior single-checkpoint deletion studies such as Gromov et al., 2025 and Men et al., 2025. However, the leap from observed correlation patterns to the claim that "the intervention pathway also develops during training" (Section 5) is correlational; no causal manipulation of the training process is performed, and the authors correctly flag that "These controlled tests show how different components evolve within the intervention pathway; they leave the causes of the broad cross-domain pattern open" (Section 5).
Reproducibility is partially but not fully satisfied. The release includes "the complete released-profile table, the 35 exported window-count arrays, focal preregistrations and processed summaries, figure and table code" (Reproducibility statement). However, "full focal acquisition and token-level model replay require the retained raw experiment records and acquisition code, which are outside this summary export" (Reproducibility statement). Moreover, the Pythia continuous-output experiment had reproducibility issues: "The initial cross-environment Pythia evaluation did not reproduce historical integer disagreement counts exactly" (Appendix D), requiring a protocol amendment with a representation-derived ambiguity bound. OLMo historical counts reproduced exactly. While checkpoint identities, precision, and text construction are documented, the absence of a complete environment lockfile for historical comparisons and the tiny context sample (18 windows) would block exact independent reproduction.
Layer interventions are widely used to probe the internal organization of language models, yet most analyses examine a single training checkpoint even though model representations and computations evolve throughout pretraining. This leaves open which depth-dependent intervention responses reflect persistent organization and which are transient consequences of training. We study this question using single-block identity bypass on fixed teacher-forced contexts across five released trajectories and 11 model-domain combinations. We find that block-bypass responses retain recognizable depth ordering while their magnitudes redistribute: nearby checkpoints preserve stronger rank correspondence than distant ones, and large changes concentrate at positions that recur across text samples and transfer across evaluation domains. Controlled experiments further show that changes in the natural bypass effect cannot be reduced to a single downstream sensitivity: in replicated Pythia runs, local missing-update magnitude grows while the pooled matched downstream response decreases, whereas OLMo-2 7B exhibits a different balance. These matched responses also depend on perturbation strength and direction, without identifying targeted compensation. Together, our results show that longitudinal layer sensitivity is structured but not static, and that single-checkpoint intervention responses should be interpreted in the context of how the underlying perturbation pathway evolves during training.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.