Agentic Knowledge-Model Co-Evolution via Consistency Regularization
LLM agents improve by distilling experience into external knowledge, but different analyses of the same trajectories can produce guidance with wildly different utility. This paper introduces SPLICE, a framework that adapts consistency regularization to knowledge evolution: it perturbs the textual instructions that transform experience into skills or memories, replays tasks under each resulting view, and updates knowledge only where utility gains are consistent across views. A co-evolution loop interleaves this knowledge evolution with GRPO and gated contrastive self-distillation to internalize the guidance. The core idea matters because current agents can be misled by plausible-looking but ineffective skills, and SPLICE offers a principled filter for generalizable knowledge.
The paper presents a sharp, well-motivated analogy between prediction consistency under parameter perturbations and utility consistency under instruction perturbations, backing it with strong cross-domain empirical gains and formal propositions. The SPLICE framework is conceptually cohesive and consistently outperforms both training-free and training-based baselines. However, the evaluation is limited to two sizes of a single model family, the asynchronous protocol relaxes key theoretical assumptions, and the heavy reliance on an unspecified external optimizer hurts reproducibility. Overall, it is a solid and creative contribution to agentic self-improvement.
The central mechanism is strongly validated by ablations: replacing the proposed common-gain selection with a simple mean-gain baseline causes WebShop accuracy to drop by up to 13.8 points and validation accuracy to improve in only 29% of rounds versus 67.86% for common gain. Empirically, the gains are reliable across math, coding, spreadsheets, and web shopping: the training-free variant already beats every knowledge-evolution baseline on every benchmark, and the full co-evolution variant exceeds the strongest training baseline by 3.0–4.6 average accuracy points while cutting output tokens per correct solution by up to 44%. These results suggest that consistency across evolution views captures a genuine signal rather than simply scaling computation.
The framework depends on an unspecified external LLM serving as the knowledge optimizer, yet the paper never names the model or provides its prompts, making independent reproduction difficult. More fundamentally, the asynchronous co-evolution procedure admits a divergence between theory and practice: Proposition 1 assumes a fixed model, but Appendix B.4 notes that during asynchronous evolution the replay model can be newer than the experience-generating model, introducing a shift $b_i$ such that "model improvement ($b_i > 0$) admits tasks whose gains come from training rather than from knowledge." Although validation mitigates this, the theoretical guarantee no longer directly applies.
Additionally, experiments are confined to two sizes of Qwen3.5, leaving generalization to frontier-scale or other families unverified.
The comparisons are generally thorough. The authors pit SPLICE against a mix of training-free (ReasoningBank, ACE, EvoSkill, SkillOpt) and training-based (GRPO, SkillRL, ReSkill) baselines, and they directly test whether the gains are merely from extra computation by comparing Best branch and Merge-all ablations. The results in Figure 4 show that "Best branch performs worst, suggesting that relying on a single view discards useful evidence, while the evidence merged by Merge-all can be redundant or distracting." However, the contrastive self-distillation component is not independently ablated in the main text, so it is unclear how much of the Co-Evo improvement stems from the consistency-regularized knowledge itself versus the novel distillation objective. The observation that "skill-augmented baselines do not always improve over GRPO" also raises the possibility that some baselines were handicapped by poor skill quality rather than weaker policy training.
Reproducing this work would be difficult. The paper mentions no code release, and the knowledge optimizer is described only as "an external LLM" without revealing the model identity, API version, or system prompts used to generate evolution views and textual revisions $\delta_j$. While Appendix A provides hyperparameters such as $N = 3$ evolution views, learning rate $1 \times 10^{-6}$, and group sizes, the core knowledge optimizer prompts, the exact task replay infrastructure, and the token-level gating implementation are omitted. The reliance on public benchmarks (LiveMath, LiveCode, SpreadSheet, WebShop) is helpful, but a third party would need to reconstruct the entire multi-view replay and validation pipeline from scratch.
LLM agents can evolve by learning from accumulated experience. Existing approaches extract reusable external knowledge, such as skills and memories, from experience to guide subsequent task solving, with some further internalizing this knowledge through model training. However, different ways of analyzing and transforming the same experience can yield knowledge with substantially different utility, even when the resulting guidance appears reasonable. Inspired by consistency regularization, we identify consistent utility gains across different ways of analyzing the same experience, rather than relying on any single way of analyzing it. We introduce SPLICE, a framework for consistency-regularized knowledge-model co-evolution. It constructs evolution views by perturbing how the same experience is analyzed, producing different knowledge candidates. Knowledge updates are then guided by common gain, which accounts for utility gains and their consistency across views. During model evolution, self-distillation internalizes the evolved knowledge alongside reinforcement learning, using knowledge associated with consistent utility degradation as a contrastive reference. The two processes interleave, with model updates driving further knowledge evolution. Experiments across diverse domains show that SPLICE outperforms knowledge evolution and model training baselines by up to 7.1 and 4.6 points in average accuracy, while its evolved knowledge reduces output tokens per correct solution by up to 44% and transfers across model sizes and families.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.