YouRA: A Persistent-State Architecture for Evidence-Traceable Autonomous Research Agents
YouRA tackles a structural reliability problem in end-to-end research agents: manuscript claims drift away from executed experiments because hypotheses, failure histories, and evidence pointers are buried in transient dialogue context rather than kept as persistent, verifiable state. The architecture introduces a YAML-based Verification State Architecture (VSA), an Independent Controller that separates control from execution (using GPT-5.2), and Stateful Reflection with three-level failure routing (bounded repair, redesign, or reset). Evaluation on MLR-Bench's 10-task end-to-end subset reports the highest Overall scores across matched backbones, complemented by hallucination and data-provenance diagnostics, though non-significant differences on the latter versus the strongest baseline temper the reliability claims.
The architecture is conceptually well-motivated and the ablation evidence is informative, but the empirical claims are constrained by small sample size, unmatched tool budgets, and non-significant hallucination-count differences against the strongest baseline. YouRA improves MLR-Bench Overall over both MLR-Agent and AI Scientist V2 (Table 12: $+0.85$, $p=0.006$ and $+0.64$, $p=0.044$), yet the Limitations section explicitly notes that "The systems compared in our experiments operate with different tool configurations and token budgets, so the reported scores reflect performance under each system's resource conditions." Moreover, on human-validated hallucination counts YouRA is not statistically distinguishable from AI Scientist V2: "On these corrected counts, MLR-Agent is significantly worse than both other systems, while YouRA and AI Scientist V2 are not statistically distinguishable." This weakens the claim that persistent state alone is uniquely responsible for reliability gains.
The VSA's YAML-based state externalization is an elegant, concrete solution to context truncation and hallucination in long-horizon pipelines, and the gate-driven progression invariant (MUST_WORK/SHOULD_WORK) forces explicit verification before claims propagate downstream. The ablation design is particularly strong: "Removing either core-state component, the VSA or the Independent Controller, also pushes YouRA's Overall below both baselines on Sonnet 4.5 and Opus 4.5." A subtle but important strength is that upstream ideation is not where YouRA wins: "MLR-Agent scores significantly higher on Overall at the Idea stage," which means the end-to-end advantage genuinely emerges during execution, verification, and writing rather than from better starting ideas. The data-provenance diagnostic showing 27/30 real-data-based papers versus 11/30 for MLR-Agent is also a notable evaluation advance.
The 10-task end-to-end subset is small for drawing robust conclusions about open-ended research, and the 4-judge panel produces only 40 verdicts per pairwise comparison. A more serious issue is that the systems compared use different tool stacks and token budgets, which the authors acknowledge but cannot disentangle from architectural effects; notably, the w/o MCP ablation removes "tool-grounded execution and structured access to memory and external artifacts," suggesting MCP access itself explains much of the real-data advantage. The hallucination diagnostic is valuable, yet the critical comparison to AI Scientist V2 on this metric is non-significant, and the moderate precision of 75.2% means roughly one in four flagged instances are false positives. The ablations are also confounded: removing the Independent Controller replaces GPT-5.2 control with "simple artifact checks," so the drop reflects both lost architecture and lost model capability. Finally, evaluation relies on LLM judges with known limitations: "Input truncation due to the judge LLMs' context length limits may disadvantage papers with extensive code, result files, or logs."
The primary evidence consists of MLR-Bench end-to-end rubric scores, order-controlled pairwise preferences, component ablations, an automated hallucination diagnostic with human validation, and a data-provenance classification. The paired sign-flip permutation tests over 30 matched cells are statistically appropriate. Comparisons to related work are generally fair, though the paper explicitly flags that the comparison to AI Scientist V2 is not tool-normalized. The distinction between union and intersection hallucination counts is methodologically careful. One gap is that ablations support each component's contribution but do not rank them: "Comparing the drop magnitudes pairwise does not turn up statistically reliable differences among the components." Another tension is that the hall-ucination diagnostic's overall precision is 75.2% with only moderate Cohen's $\kappa = 0.54$, so automated flag counts are noisy signals.
Code, VSA schemas, prompts, generated papers, and evaluation scripts are released at the stated GitHub URL. However, independent reproduction faces serious barriers: the pipeline routes through multiple commercial APIs (Anthropic, OpenAI, Google, xAI) and "random seeds are not used as a reproducibility guarantee for the controller/evaluation layer because it routes through multiple commercial LLM API paths... that do not provide a uniform deterministic replay guarantee across models and backend updates." Additionally, "All experiments use the backbone LLM's default sampling parameters," meaning temperature and top-p are provider-controlled. The tool stack includes five MCP-facing services plus Claude Code, and costs or token budgets are not reported, leaving resource requirements opaque.
End-to-end research agents can now produce complete scientific papers, yet manuscript claims often diverge from executed experiments. This gap is structural: research state, failure histories, and claim-evidence alignment are not maintained as persistent, verifiable state across long-horizon pipelines. We present YouRA (Your Research Agent), an architecture for stateful, evidence-traceable autonomous research. YouRA preserves research state, execution evidence, and failure history across the research trajectory by integrating three components: a Verification State Architecture (VSA) that tracks hypotheses, gates, and evidence pointers; an Independent Controller that turns state and reflection records into lifecycle, recovery, and debate/review control while separating control from execution; and Stateful Reflection that logs failures as structured lessons and routes recovery through bounded repair, redesign, or reset. On MLR-Bench's predefined ten-task end-to-end subset, YouRA improves over both MLR-Agent and AI Scientist V2 on scalar Overall across all three matched backbones. An automated diagnostic using MLR-Bench's hallucination taxonomy reports intersection/union counts for four fact-based failure types, and data-provenance diagnostic shows more real-data-based outputs. Ablating each of the four components (the VSA, the Independent Controller, MCP tool access, and reflection-guided recovery) supports their separable contributions. Removing either core-state component drops YouRA below the full system. Code: https://github.com/PrayPrey/Your-Research-Agent.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.