Harness Policy Distillation: Aligning Agent Policies with Runtime Semantics

The paper's decomposition of policy mismatch into supervision mismatch and runtime-state mismatch is conceptually sharp and usefully separates what HPD fixes from what prior methods address. The three-view objective—Harness-Replay (T), Injected (I), and Runtime (R)—is well derived via KL chain rule (Appendix F) and the component ablations in Table 2 show that verified teacher targets alone (HPD-T) already beat the strongest baseline OPD, while adding I/R together yields further gains. The cross-architecture and cross-harness adaptations in Figure 4 support portability. The training details in Appendix C and open-sourced artifacts (code and model weights) are commendable.

Several baseline results are anomalous and deserve scrutiny. SFT collapses task success from 3.09% to 2.06% and final-answer F1 from 26.82% to 2.40% despite improving reference-path metrics, which looks more like a training failure or severe hyperparameter mismatch than a pure distribution-shift effect. If SFT was not tuned fairly, the comparison landscape changes. More importantly, staged training of T followed by R catastrophically drops TS from 20.58% to 7.03%, indicating extreme sensitivity to optimization dynamics that the paper does not fully explain. The BFCL v4 transfer in Section 4.3 rests on only 92 cases, which is underpowered for strong generalization claims. Finally, the exact-match verification gate plus cosine-similarity teacher selection may overfit to reference syntax rather than semantics.

The evidence supports the core claim that harness-verified action targets improve over unverified teacher distillation: HPD-T (20.58% TS) beats OPD (15.61% TS) despite using the same 8B frozen base model, whereas OPD uses a privileged 32B teacher. This is a strong result. However, comparisons are not always apples-to-apples: OPD's 32B teacher is computationally stronger but lacks the harness control document, while HPD gains come from the intersection of verification and runtime annotation. The five-seed paired analysis with Holm-corrected p-values (Appendix C.5) adds statistical rigor. The metrics are well chosen—separating Action-Acc, Step-SR, RC, Exact, and TS—but the paper could more tightly link each metric to its corresponding mismatch type. The near-parity with DeepSeek V4 Flash on TS (22.98% vs 22.30%) is interesting but should not be over-interpreted as HPD matching frontier models, given differences in model scale and training.

Reproduction is partially feasible but blocked by several factors. The training hyperparameters are exhaustively documented (Appendix C, Table 4: AdamW, constant 2e-6 LR, batch size 128, LoRA on Qwen3-8B, 8x A800s), and the authors release code and model weights. However, the harness-specific components—AutoProbe, the serializer $\Omega_H$, the transition kernel $T_H$, the observation adapter $\operatorname{Observe}_H$, the compiler $\Phi_H$, the verifier $V_{H,b}$, and the retry logic—are complex and must be reproduced exactly. The paper warns that applying HPD to a new harness requires reconstructing these from scratch. The evaluation uses proprietary TRAJECT-Bench tasks with replayed tool-output queues; independent reproduction requires access to this benchmark. Only five seeds are reported for variance estimation, and no standard deviations are given for the main Table 1 results. The teacher construction retry procedure (up to 5 attempts) and the $K=2$ candidate sampling with temperature 1 also need exact replication to reproduce the training targets.