ADAPT for Science: A Multi-Agent System for Self-Evolving Scientific Optimization
Scientific optimization is expensive and constrained, yet the search policy is usually fixed ahead of time. ADAPT treats the complete policy $\pi=(W,O)$—where to start and how to move—as the object of optimization, using historical designs for target-aware initialization and multi-agent LLM evolution to adapt the procedure itself. Evaluated on eight held-out gravitational-wave detector topologies under matched budgets, it reduces mean loss by 54.9% versus the best classical optimizer and by 62.2% versus the best agentic baseline. The work is compelling within its domain but leaves open questions about generality and reproduction.
The paper presents a strong single-domain demonstration with compelling empirical margins, but it stops short of establishing the method as a general scientific optimization system. The core idea—optimizing the full search policy $\pi=(W,O)$ rather than only the scientific parameters—is well formulated and the ablations support its decomposition into selection and optimizer evolution. However, the evaluation is limited to one simulator and one domain, and reproducibility is hampered by undisclosed code, data URLs, and reliance on future-generation proprietary models. The held-out gains are real within this scope, yet the broader claim of continuous self-evolving scientific optimization remains a promise rather than a proven property.
The decomposition of search into where to search (initialization and budget allocation) and how to search (optimizer evolution) is principled, and the ablation in Table 3 shows both stages contribute meaningfully: removing evolved selection raises mean loss to $0.423$, while removing evolved optimization raises it to $0.256$. The discovered selection policy is physically interpretable—it progressively screens 64 retrieved candidates down to 8, then 2, then 1, with a protected exploratory branch—and the evolved optimizer transparently modifies Adam with time-dependent scaling $s(r)=0.05+0.95r^2$, finite-state rejection, and shorter first-moment memory $\beta_1=0.7$. The eight held-out topologies constitute a proper held-out test because the paper explicitly excludes them from the experience library and all strategy development: 'Held-out targets and their outcomes are never returned to either evolver.'
I see two major concerns. First, the domain coverage is narrow: every experiment uses the same differentiable GW-detector simulator, and the paper itself acknowledges that 'the results therefore establish the effectiveness of the framework in one challenging scientific domain, but do not yet demonstrate universality across scientific disciplines.' Second, the impressive margins over agentic baselines partly reflect ADAPT's exclusive access to 15,409 historical designs; the general-purpose baselines are not modified to use this library, so the comparison measures a full system against components rather than isolating the procedural evolution itself. The No-Prompt ablation is also under-specified in the main text—the reader learns only that it 'removes the science-specific guidance while retaining the corresponding agentic evolution pipeline,' without knowing which prompts or priors were dropped. In addition, the composite optimizer score $S=w_q S_{quality}+w_a S_{anytime}+w_f S_{feasibility}+w_i S_{improvement}$ is introduced without any explanation of how the weights are set, leaving the meta-objective effectively undocumented in the main paper.
The evidence supports the within-domain claims but not the broader framing. The matched wall-clock budget and the strict held-out protocol are strengths: 'All comparisons use matched random seeds, hardware, numerical precision, and the same total wall-clock budget,' and held-out targets are excluded from retrieval and evolution. Classical optimizers and agentic program-evolution baselines are both covered, with ADAPT achieving the lowest loss on all eight topologies. That said, the strongest baseline (NAAdam at $0.452$) is not domain-tuned, whereas ADAPT exploits a large historical bank and an evolved selection pipeline. A more informative comparison would be ADAPT versus the same pipeline with retrieval but without evolution, which the 'Direct+GEPA' and 'Fixed+Adam' ablations approximate but do not fully replace.
Reproducibility is the weakest part of the submission. There is no code repository, no public link to the historical design library, and no downloadable simulator in the main text—only a promise that 'the supplementary material provides ... implementation settings.' The method depends on future-generation proprietary APIs (GPT-6 Astra and Claude-fable-5.1) whose weights, prompts, and behavior are not guaranteed to be stable or accessible. Because the selection policy and optimizer program are outputs of a stochastic LLM-driven search, independent reproduction would require not only the exact prompts and evolution logs but also deterministic access to the same models at the same version—a condition the paper does not describe.
Scientific discovery often requires identifying a small number of high-performing designs in vast, constrained search spaces. This is difficult when evaluations are costly, feasible solutions are sparse, parameters are strongly coupled, and physical constraints are strict. Recent AI for AI methods can automatically improve algorithms and optimization procedures. However, general purpose approaches are not designed around scientific priors, physical constraints, and limited evaluation budgets. We introduce ADAPT for Science, a multi-agent framework that treats scientific optimization as two coupled decisions: where to search and how to search. ADAPT uses historical designs to identify promising and diverse initial conditions, progressively concentrates computation on favorable search directions, and evolves the optimization procedure from execution feedback. We evaluate ADAPT on the design of gravitational wave detectors using a differentiable physics simulator and eight unseen detector topologies. Under matched scientific evaluation budgets, ADAPT achieves a mean loss of 0.204 +/- 0.101, reducing loss by 54.9% relative to the best classical optimizer and by 62.2% relative to the best general purpose agentic evolution baseline. ADAPT achieves the lowest loss on all unseen topologies among the evaluated baselines. ADAPT provides a step toward scientific optimization systems that continuously learn from prior experience and their own search outcomes.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.