AgenticEvolve: Evolutionary Program Search with Persistent Language Model Agents that Design and Run Experiments
AgenticEvolve replaces the one-shot LLM variation operator in evolutionary program search with a persistent agent that edits code, runs self-designed experiments, and commits candidates for formal evaluation. By letting the agent test hypotheses locally between expensive evaluations, the framework aims to improve sample efficiency and final program quality on tasks ranging from online bin packing to symbolic regression. The authors report gains over a matched No-analysis ablation and over OpenEvolve, while noting that cost trade-offs depend heavily on model pricing.
The paper makes a clear empirical case that persistent LLM agents with a local experimentation tool improve program quality, but the evidence is weakened by a tiny sample in symbolic regression (one run per problem), inconsistent evaluation protocols across tasks, and cost advantages that vanish for models with expensive cached-input pricing. As the authors acknowledge, "Each symbolic regression method ran once per problem, so run-to-run variation is not measured" (Section VII). Moreover, "The heuristic runs predate the feedback restriction of Section III-D, so for all three methods the feedback of a failed evaluation included the truncated output and traceback" (Section IV-C), which means the heuristic and regression experiments were not conducted under identical conditions.
The matched No-analysis ablation is methodologically strong: it removes only the run_analysis tool while keeping the persistent conversation, archive, migration, and budget intact, isolating the effect of experimentation more cleanly than prior work. The trajectory in Section V-D gives a concrete illustration of how the agent estimates bin waste, builds a placement rule, and optimizes it locally before committing. The authors summarize that "AgenticEvolve achieves the best mean final quality on both hidden evaluation instances and unseen test instances with every model, reaching the baselines' final quality with a fraction of their formal evaluations" (Section I, Contributions), and the design note that "No-analysis removes run_analysis from the available tools, disables access to analysis.py and removes the corresponding instructions from the system prompt. The difference between the two configurations therefore measures the contribution of the agent's experiments" (Section III-D) makes the causal claim precise.
The symbolic regression experiment is statistically under-powered. With a single run per method per problem, structure-recovery counts in Table IV could easily reflect chance, and the authors concede that "Each symbolic regression method ran once per problem, so run-to-run variation is not measured" (Section VII).
Cost comparisons are also less favorable than the sample-efficiency narrative suggests: AgenticEvolve is several times more expensive than No-analysis on every task, and with GLM and Qwen its persistent conversation incurs high cache-read costs that erase any cost lead over OpenEvolve. Protocol differences further muddy the comparisons—the heuristic runs retained richer failure feedback—and the authors note that in the final restricted runs, "agents still attempted to access hidden data through analysis scripts and candidate programs" (Section VII), relying on audit rather than sandboxing.
On heuristic design, the evidence supports higher sample efficiency: AgenticEvolve reaches OpenEvolve's final quality with as few as three evaluations versus 180, and beats both No-analysis and OpenEvolve on hidden and unseen instances with every model. In symbolic regression it recovers eleven of twelve equations exactly or with negligible extra terms, versus six for No-analysis and six for SR-Scientist, at roughly comparable cost. Yet the comparison to OpenEvolve required substantial setting tweaks—migration interval, population size, and feature rescaling—that the authors document but which could still bias results. Likewise, "LLM-SR costs seventeen times as much as AgenticEvolve in symbolic regression" (Section VII), but this gap is driven largely by DeepSeek's long reasoning outputs and the lack of prompt caching in LLM-SR's implementation, not purely by algorithmic design.
The authors release source code, frozen framework snapshots, run records, and evaluation-access audits on GitHub and Zenodo, which is commendable. Hyperparameters, budgets, prompts, and model settings are reported in detail. However, independent reproduction faces two obstacles. First, the hidden heuristic evaluation instances and the exact evaluator state are not publicly redistributed, so replicating the reported scores depends on the authors' released evaluator. Second, and more seriously, the framework "uses no operating-system sandbox" and relies on "protocol-level access restrictions" (Section III-D), with compliance "checked by audit" (Section VII). Without OS-level isolation, reproducing the security environment exactly is difficult, and a run could silently access hidden data if the audit misses a path.
Evolutionary program search with large language models keeps the structure of an evolutionary algorithm and uses the model as its variation operator. Typically, the model generates each candidate program in a single request from parent programs and their scores. We present AgenticEvolve, an evolutionary program search framework in which each island is run by a persistent large language model agent. Between evaluations, the agent designs and runs experiments on the candidate program. To measure the contribution of experimentation, we compare AgenticEvolve with No-analysis, an otherwise identical configuration with experiments disabled. On two heuristic design tasks with three large language models, AgenticEvolve produces better programs than No-analysis and OpenEvolve on both hidden and unseen instances. It reaches the baselines' final quality with far fewer evaluations and comparable cost. On twelve equation discovery problems, it recovers the ground-truth structure of more equations than No-analysis and existing language-model and genetic programming baselines.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.