LAGO: Language Agents with Reusable Growing Options
Long-horizon tasks like real-world software engineering expose the brittleness of monolithic agent loops, where a single model must simultaneously decompose, execute, and verify within one trajectory. LAGO addresses this by treating specialized language agents as temporally extended options—encapsulating role specifications, intra-option policies, termination conditions, and verifiers—that a meta-agent can retrieve from a persistent library or spawn on demand. The core proposition is that this hierarchical scaffold turns task decomposition into a reusable, growing component that requires no weight updates, and the authors report 58.6% on the full 500-task SWE-bench Verified with a frozen Qwen3.5-35B-A3B.
The paper offers a conceptually clean hierarchical framework and a methodologically sound controlled comparison against re-run baselines under identical frozen weights. However, its central causal claims about the value of a growing library of reusable agent-options are underpowered: the head-to-head scaffold comparison rests on a 150-task subset where the authors themselves note a roughly ±8 percentage-point uncertainty, and the 50-task component ablation is too small to isolate library content from generic decomposition scaffolding. The authors are admirably transparent about these limitations, but the result is evidence for practicality rather than a rigorous demonstration of mechanism.
The formalization of agent-options as four-tuples $o=(\mathrm{spec}_o, \pi_o, \beta_o, v_o)$ is precise and directly adapts the hierarchical RL options framework to language agents. The experimental protocol is the paper’s strongest feature: instead of assembling an uncontrolled leaderboard, the authors re-run mini-SWE-agent and Terminus 2 end-to-end on the same frozen Qwen3.5-35B-A3B policy, temperature, and task-ID set, producing an apples-to-apples comparison of delivered scaffolds. On the full 500-task SWE-bench Verified, the same frozen model resolves 58.6% under the official Docker evaluator, which is a respectable absolute result for a training-free system.
The analysis is severely limited by sample size and confounds. The decisive scaffold comparison in Table 1 uses a 150-task subset; at that scale, the authors correctly caution that only sizable gaps are meaningful because differences carry roughly ±8 percentage points of binomial uncertainty. The component ablation in Table 2 shrinks to 50 tasks, making its four- to six-task swings nearly uninterpretable. Crucially, the paper does not equalize context windows or inference budgets across scaffolds, so LAGO’s gain may partly reflect higher token expenditure rather than architectural superiority.
Cross-model transfer is weak. When the executor is fixed to DeepSeek-V3.2, the fine-tuned Qwen3-30B meta-agent improves over its base variant by only 6 points on SWE-bench Pro (40% vs 34%) and shows no gain on BrowseComp; when Qwen3-30B serves as executor, absolute performance collapses to 4–6%. The library ablation also conflates mechanisms: disabling the library removes the hierarchical routing structure, so the six-task drop does not cleanly isolate reusable skill from the mere fact of decomposition.
The evidence supports the claim that LAGO is a practical scaffold, but overstates the case for causal impact of the growing library. The full 500-task result (58.6%) and high library reuse rate (84.4%) demonstrate activity, yet the paper itself admits that reuse metrics measure retrieval activity, not causal value. Comparisons to related work are fair: the authors correctly position LAGO against static multi-agent frameworks (MetaGPT, AutoGen) and self-evolving skill libraries (Voyager, CODESKILL), emphasizing that LAGO stores complete specialized agents with verifiers rather than raw prompts or code snippets. The options framing is well-attributed to Sutton et al. and Bacon et al.
Reproducibility is mixed. On the positive side, the authors specify the frozen model (Qwen3.5-35B-A3B), temperature 0, task IDs (stratified, seed 42), and the official Docker evaluator. On the negative side, no code repository or complete hyperparameter list is provided in the text: the admission thresholds $\theta_{\mathrm{task}}$ and $\theta_{\mathrm{nov}}$, the deduplication similarity function, and all prompts are omitted. The seed library is named (agent_library_seed350) but not released, and the Qwen3-30B SFT dataset and QLoRA configuration are only summarized in Appendix D. An independent team would need substantial implementation detail to reproduce the results.
Long-horizon tasks expose a weakness of monolithic tool-use loops: one agent must decompose, execute, and verify a task within a single trajectory. We introduce LAGO, which treats specialized language agents as reusable temporally extended options. A meta-agent retrieves options from a persistent library or spawns new ones, dispatches sub-tasks, verifies results, and admits successful novel options. The library grows across tasks without updating model weights. On 500 SWE-bench Verified tasks, frozen Qwen3.5-35B-A3B in both roles resolves 58.6% under single-attempt evaluation with the official Docker evaluator. On a fixed 150-task subset, we re-run mini-SWE-agent, Terminus 2, and LAGO with the same policy, temperature, task IDs, and evaluator; their resolve rates are 49.3%, 57.3%, and 60.3%, respectively. Cross-model experiments provide evidence consistent with a role for decomposition quality. A 50-task component ablation finds four fewer resolved tasks without decomposition, two fewer with shuffled reused content, and six fewer without the library. These results support reusable agent-options as a practical abstraction for task streams, while leaving component-level causality task- and model-dependent.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.