When Non-Adversarial Noise Hurts Learning: Benchmarking Skill Self-Evolution in Agents
LLM agents learn reusable skills from experience, but real-world noise can corrupt the rules they distill. This paper introduces RSE-Bench and RSEClaw, a benchmark and framework for evaluating skill self-evolution under ordinary, non-adversarial noise across four entry points: instructions, evidence, execution records, and outcome bindings. The central finding is that noisy learning retains only $25.5\%$–$42.1\%$ of the aggregate gains from clean evolution, and noise confined to learning records erodes performance as much as input-side perturbations. The work matters because it shows that benign deployment imperfections persist in agent memory and propagate to later tasks.
The paper makes a timely and methodologically sound contribution by isolating how non-adversarial noise inside the learning loop degrades evolved skills. Its core finding—that record and binding noise erode gains as much as input-side noise—is empirically supported and practically important. However, the evaluation covers only three methods and two backbones, and some clean-learning baselines underperform, which tempers the strength of the noisy-learning conclusions.
The benchmark design is a clear strength. The paper carefully separates learning-side from evaluation-side noise, enforces task-preserving constraints, and evaluates on held-out tasks with frozen skill stores. The RSEClaw framework decouples methods from execution and noise injection through a fixed interface, enabling fair comparison. The process diagnostics (input exposure rate, candidate-to-clean similarity) are valuable additions that go beyond aggregate scores.
Two limitations stand out. First, the study evaluates only three self-evolution methods, and clean evolution itself is unstable: on GPT-5.2, ACE’s clean evolution on SpreadsheetBench ends below the initial agent ($40.83$ vs. $45.00$), making noisy-learning losses harder to interpret. Second, binding noise (N-Bind) reaches optimizer inputs for only $15\%$–$21\%$ of episodes, so claims about its impact are based on limited actual exposure. The adaptation analysis—whether learning under noise helps deployment under the same noise—is restricted to a single method and benchmark, weakening its generality.
Comparisons to related work are fair and verified. The paper accurately distinguishes RSE-Bench from prior benchmarks: AgentNoiseBench evaluates static execution-time robustness without skill updates, RethinkSkill probes frozen skills after clean evolution without learning-loop noise, and SkillMisevo-Bench focuses on adversarial safety rather than ordinary noise. The claim that this is the first task-preserving benchmark injecting non-adversarial noise into the skill self-evolution loop appears correct based on these checks. The $25.5\%$–$42.1\%$ retention figure, however, aggregates across methods with very different clean-learning gains and standard deviations, so it should be read as a directional summary rather than a precise universal rate.
Reproducibility is generally strong: the benchmark data, noise generators, and framework are released, and run-time noise modules are deterministic given a recorded seed. Appendix D provides detailed hyperparameters and method configurations. The main barrier is the reliance on specific closed models (GPT-5.2, GPT-5.5, DeepSeek-V4.1-Flash) whose availability is not guaranteed to other researchers; while the noisy task releases are shared, re-running the full evolution pipeline requires access to the exact model versions used.
Large language model agents increasingly improve by distilling experience into reusable skills, but errors in that experience can persist in learned rules and affect later tasks. Existing evaluations largely examine clean learning or perturbations encountered after skills are frozen, leaving robustness during self-evolution insufficiently understood. We introduce RSE-Bench, to our knowledge the first benchmark for evaluating skill self-evolution under ordinary, non-adversarial noise. It spans four domains and six benchmarks, with controlled perturbations to instructions, environmental evidence, execution records, and outcome--record pairings that preserve the underlying tasks and native scoring rules. The protocol separates noise during learning from input-side noise at evaluation, testing frozen skill stores on held-out tasks. We also introduce \textsc{RSEClaw}, a method-agnostic framework supporting consistent noise injection and process diagnostics of actual exposure and skill updates. Across three methods and two backbones, noisy learning retains only $25.5\%$--$42.1\%$ of the aggregate gain achieved by clean self-evolution, depending on the noise type. Perturbations confined to execution records and outcome--record pairings cause average losses comparable to input-side noise, despite leaving the original executions and evaluations intact. Diagnostics further identify differences in candidate skill text that coexist with small changes in mean held-out performance.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.