Cross-Benchmark Transfer from RL on Agentic Coding Tasks
The paper asks whether reinforcement learning on a small set of expert-built coding tasks can teach a generalizable ability to finish agentic work, rather than merely overfitting to a benchmark format. The authors post-train Kimi K2.7 Code with one epoch of GSPO on a rank-32 LoRA adapter using 1,700 tasks graded by hidden tests and pass-to-pass regressions, then measure transfer to six external benchmarks spanning three agent harnesses, multiple domains, and horizons up to multi-hour builds. The results matter because they probe whether RLVR on narrow, high-quality data yields broad agentic competence or just domain tuning.
The paper makes a plausible but not fully airtight case for transferable completion habits. Pass@1 improves on every benchmark, and the pooled significance is strong ($p<0.001$), yet several individual gains are statistically marginal: SWE-Bench Pro ($p=0.059$), DeepSWE against the public baseline ($p=0.055$), and SWE-Marathon ($p=0.089$) all fail conventional thresholds. The qualitative trajectory analysis is illuminating but necessarily selective, and without controlled ablations the leap from correlation to the claimed mechanism—general engineering habits rather than benchmark-bleed—remains interpretive.
The paired trajectory analysis is the strongest element. The authors show quantitatively that base-model failures are near-misses on DeepSWE—the median failed run still passes 86% of target tests—and then map paired trajectories onto four failure modes (lost requirements, narrow testing, silent regressions, weak ground truth) that the trained model avoids. They also tie each behavioral shift to a specific property of the training reward, such as the regression gate enforcing zero reward whenever a pass-to-pass test fails.
First, inference variance undermines a headline result: the in-house DeepSWE base run scored 26.5% against a public baseline of 31.0%, so the reported +12.4 pp gain relies on the more favorable public number while the trajectory analysis uses the lower in-house run. Second, the training run is very short—only 108 rollout steps and 216 optimizer updates on 1,700 tasks—yet produces large gains; without ablations on dataset size, reward design, or supervised warm-up, it is unclear what to attribute the effect to. Third, the claim that transfer reflects general habits rather than domain knowledge is asserted but not experimentally isolated, for example by probing zero-shot transfer to unseen tool schemas or languages.
The quantitative evidence supports weaker correlational claims better than the strong causal claims the paper prefers. The trajectories are vivid, but they are drawn only from tasks the trained model newly solves, creating a selection bias toward readable successes. Comparisons to related work are descriptive rather than empirical: there is no direct baseline using the same compute and model trained on mined SWE-bench data or with a binary reward, so the benefit of 'expert-built, individually gradable requirements' remains a hypothesis. The authors themselves acknowledge the interpretive leap, concluding that 'what transferred was not domain knowledge ... but a way of finishing work'—a plausible inference, not a demonstrated one.
Experimental detail is unusually thorough: hyperparameters (GSPO with $\varepsilon_{\text{low}}$, $\varepsilon_{\text{high}}$, no KL penalty, rank-32 LoRA with $\alpha=32$, Adam with decoupled weight decay), sampling ($top$-$p$ 0.95, 65,536 tokens per turn, 262,144 context), and infrastructure (64 H200s, Miles, SGLang, Megatron-LM with expert parallelism) are all listed. The short training duration should make replication cheap in principle. The blocking issue is data access: the 1,700 expert-built tasks from Surge AI are not released, and neither is the exact harness configuration or code. Independent reproduction is therefore impossible without the proprietary dataset.
Coding agents often fail in the last mile: they build most of a feature but drop a requirement, test only the cases their implementation already handles, break behavior that was supposed to stay intact, or validate against an unchecked assumption. We ask whether reinforcement learning (RL) on expert-built agentic coding tasks closes this gap, and whether what the agent learns transfers beyond the training distribution. We post-train Kimi K2.7 Code, a 1T-parameter (32B active) open-weight mixture-of-experts model, with RL alone on 1,700 tasks: 1,000 repository tasks graded by hidden fail-to-pass tests and by pass-to-pass tests of existing behavior, and 700 terminal tasks graded by expert-written hidden verifiers. The reward is the fraction of target checks passed and drops to zero if any pass-to-pass test fails. One epoch of GSPO on a rank-32 LoRA adapter improves pass@1 on each of the six external benchmarks we evaluated, across three agent harnesses: SWE-Bench Pro (60.1 to 64.8), DeepSWE (31.0 to 43.4), Terminal-Bench 2.1 (67.4 to 82.0), Terminal-Bench 3 (1.4 to 12.1), Terminal-Bench 4 (0.0 to 7.6), and SWE-Marathon (5.0 to 25.0). Pooled over the five independent task sets (Terminal-Bench 4 revises Terminal-Bench 3), the improvement is significant (p < 0.001), and it remains significant on the three sets released after the training data was collected (p = 0.004); the model also improves under both harnesses never used in training. Median trajectories on DeepSWE and Terminal-Bench 3 are 24-35% shorter in agent steps. The base model's failed DeepSWE runs are mostly near-misses, and on the tasks the trained model newly solves, paired trajectories show it avoiding each of the four failure modes above.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.