TreeFlex: Agents Navigating Their Own Search Trees
Standard ReAct agents carry every failed attempt and off-path observation in one linear context, which eventually buries the evidence they need. TreeFlex solves this by saving intermediate states as nodes in a search tree and letting the same language model choose where to resume and what to do next, operating in two roles—Reflection for global navigation and Execution for local work—trained end-to-end with a tree-aware adaptation of GRPO. The paper shows that this design improves accuracy on deep research and software engineering benchmarks, especially on harder tasks, though it comes with higher inference costs and small-scale training.
TreeFlex demonstrates that letting a single model manage its own frontier—choosing which saved state to resume and what instruction to issue—can outperform flat and folded-context baselines on two agent benchmarks. The RL-trained 4B model achieves 48.8% Mean@8 on BrowseComp-Plus, 7.3 points above flat ReAct and 6.0 above FoldAgent. However, the gains are tempered by substantially higher per-rollout compute, small training scales, and single-run configurations that leave training variance uncharacterized.
The tree-aware RL formulation is the strongest conceptual contribution. By treating each complete search as one GRPO rollout and training each generated token only on the exact context it was sampled from, TreeFlex avoids inventing tree-shaped attention masks and keeps the credit-assignment clean. The dual-role design is also elegant: the same model switches between a global Reflection view of node cards and a local Execution view of a single restored path, so the harness need only save and restore states.
The main concern is that TreeFlex uses far more inference budget than the baselines—about 2.8 times the generated tokens of FoldAgent and 6.8 times those of flat ReAct per rollout on BrowseComp-Plus—so the accuracy gains are not compute-matched. The authors show that a single TreeFlex rollout still beats majority-voting over multiple flat samples at comparable retrieval budgets, but the sequential latency and token costs are substantial and unpenalized in the reported metrics.
Additionally, the experiments are small and underpowered: one training run per configuration, a 4B model trained on only 616 questions, and a 9B model on at most 250 SWE tasks, with the authors explicitly noting that training variance is unknown.
On BrowseComp-Plus, the evidence supports the headline comparisons, and the paper responsibly notes that repeated flat sampling does not close the Pass@8 gap. The difficulty-bin analysis is convincing: TreeFlex gains 9.5 points on hard questions where flat ReAct and FoldAgent struggle most. On SWE-bench Verified, however, the comparison is more nuanced. TreeFlex's base model already leads FoldAgent by 19.5 points before any RL, suggesting much of the gain comes from the architectural harness rather than the learned policy. A hand-written step-wise delegation prompt brings FoldAgent within 1.6 points of TreeFlex's base model, and RL on that prompt actually degrades performance because of a train-test mismatch in how submissions are graded, weakening the claim that the tree-search policy is uniquely responsible for the SWE improvements.
The paper provides detailed hyperparameters, prompts, and pseudocode in the appendices, which aids reproduction, but it does not release code, data, or the harness implementation. Key blockers include the absence of a public artifact link, the reliance on custom per-action out-of-scope and precision-first judges whose exact prompts are long and model-dependent, and the use of several unreleased or future-dated dependencies such as FoldAgent and BrowseComp-Plus. Without the exact harness and judge implementations, independent reproduction of the reported numbers will be difficult.
A ReAct-style agent accumulates observations, intermediate results, and failed attempts in a single context. As this context grows, stale and off-path information consumes its limited capacity and obscures relevant evidence, making focused reasoning increasingly difficult. We introduce TreeFlex, a framework in which the agent's own policy decides where to search. The agent records each intermediate state, with a short summary, as a node in a search tree maintained by the harness. The policy is a single language model that acts in two roles. In the Reflection role, it has a global view of the tree through these summaries and picks a node to resume, together with an instruction. In the Execution role, the harness restores the path that led to that node, and the model carries out the instruction with the usual task tools until it saves a new node or proposes a solution, which Reflection then approves or sends back to another node. Through these choices, the policy decides whether to deepen the current line, branch from or revisit an earlier state, verify a proposal, or stop, without a fixed traversal order or a score for each node. We train navigation, task actions, checkpointing, and proposal review jointly with a tree-aware adaptation of GRPO, using task-level outcome rewards augmented by process rewards. After small-scale RL of a 4B model, with the same active-context limit for all methods, TreeFlex reaches 48.8% accuracy on a challenging deep-research benchmark, versus 41.5% for flat ReAct and 42.8% for FoldAgent, and its Pass@8 is about ten points higher than either. On SWE-bench Verified, with all three systems trained with RL from the same 9B model, TreeFlex resolves 54.9% of tasks, versus 43.3% for FoldAgent and 45.2% for its step-wise variant.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.