Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents
This paper identifies progress stagnation—unproductive reasoning-action loops—as a critical failure mode when distilling multi-turn agent capabilities from large teachers into small language models via supervised fine-tuning. To mitigate it, the authors propose Progress-Aware Distillation (ProD), which measures the stagnation gap between student-generated and teacher trajectories and encodes that discrepancy into an adaptive margin within a SimPO-style preference objective. The contribution matters because prior agent distillation improves tool-use accuracy yet collapses on longer horizons, directly undermining practical deployment of efficient small-model agents.
The paper delivers a compelling empirical diagnosis and a pragmatic training-time remedy. The stagnation analysis across eight SLMs is thorough, and the adaptive-margin formulation $f(\tilde{\tau},\tau)$ is intuitive and well-motivated. Still, the work never explains why SFT mechanistically induces stagnation, and its reliance on an undisclosed judge model and unreleased code weakens confidence in the headline stagnation metrics.
The systematic stagnation evaluation is a standout strength. The authors define three complementary scores—lexical, semantic, and judge-based—and show they are highly correlated ($r>0.92$), lending robustness to the finding that SFT-distilled agents loop severely. The adaptive margin $f(\tilde{\tau},\tau) = c_0(\alpha \max\{0,\Delta S\} + (1-\alpha)\mathbb{I}_{\text{Ans}(\tilde{\tau})\neq\text{Ans}(\tau)})$ is simple yet effective, and the ablation in Table 6 demonstrates that balancing stagnation severity with answer correctness yields the best results.
The paper describes stagnation extensively but does not pinpoint the causal mechanism by which SFT produces it—whether from exposure bias, imitation of surface-level patterns, or insufficient credit assignment across long contexts. Without this diagnosis, it is unclear whether ProD is treating a symptom or a root cause. A second concern is evaluation brittleness: final-answer accuracy is judged by GPT-4.1-mini, and the LLM-as-judge stagnation metric relies on a model identified as gpt-5.4-2026-03-05 that is not publicly available, which may systematically favor teacher-like trajectories.
The authors acknowledge the first limitation: "Our current stagnation scores do not explicitly distinguish these cases." Moreover, the preference-style objective is closely related to SimPO and DPO with a per-instance margin; while effective, the incremental algorithmic novelty is modest.
The experimental evidence strongly supports the core claim that ProD mitigates stagnation: across eight SLMs, ProD improves average in-domain accuracy by 4.7% over SFT and reduces average turns and tokens by 11.6% and 20.7%, respectively (Table 4). Comparisons to repetition-penalty decoding and concise-trajectory SFT confirm that trajectory-level intervention is necessary. However, the paper does not compare against RL-based distillation or test-time search, which are plausible alternatives for improving exploration. The comparison to on-policy DPO and SimPO in Appendix J shows ProD ahead, but the margins are modest (1–4 points), suggesting the adaptive margin is helpful rather than transformative.
The authors provide extensive hyperparameters in Appendix B.1—LoRA rank 64, learning rates, batch size 32, and fixed values $\beta=2.5$, $\lambda=1$, $\alpha=0.5$—which is commendable. Yet the code and data are withheld until acceptance, and the LLM-as-judge stagnation evaluation depends on a non-public model. Reproducing the full pipeline requires access to the teacher API (GPT-4.1), proprietary search tools, and significant compute (single A100/H200). The use of greedy decoding capped at 10 turns is clearly stated, but exact prompts for trajectory generation and the search-tool integration details remain underspecified.
While LLM-based agent systems have demonstrated remarkable proficiency in reasoning and solving complex real-world tasks, they also come with substantial token consumption and inference latency, limiting their practical deployment. To enhance the efficiency of agent systems, recent efforts have focused on transferring agentic capabilities from large-scale models to small language models (SLMs) via supervised fine-tuning-based distillation. Nevertheless, as interaction trajectories grow longer, involving extended reasoning-action chains and extensive environmental feedback, distilled student agents tend to experience progress stagnation: becoming trapped in unproductive loops characterized by persistent failed strategies and difficulty making meaningful progress. Our systematic analysis shows that this phenomenon consistently occurs across diverse SLMs and task domains. To address this issue, we propose Progress-Aware Distillation (ProD), which explicitly detects and penalizes stagnation in student-generated trajectories. ProD encodes the stagnation gap between student and teacher into an adaptive margin for dynamically penalizing possible stagnation. By iteratively applying this procedure, ProD enables student agents to progressively approximate teacher behavior. Extensive experiments involving 8 SLMs, ranging from 0.6B to 8B parameters, on both in-domain and out-of-domain benchmarks, demonstrate that ProD substantially mitigates progress stagnation while enhancing both the success rates and task-completion efficiency of distilled SLM agents.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.