IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems
IQuest-Q1 is a 320-billion-parameter sparse Mixture-of-Experts foundation model (15 billion activated) optimized for agentic command-line interaction. It merges Stable SplitNorm, the Muon optimizer, and $\mu$P parameterization within a three-stage pipeline—progressive pre-training, context-extending mid-training, and multi-scaffold post-training using reinforcement learning and multi-teacher distillation—to advance open-weight CLI agent capabilities. While the paper integrates several validated training techniques and releases model weights openly, benchmark results reveal it still trails frontier proprietary systems on the hardest coding and reasoning tasks.
The paper is a thorough but incremental technical report on a large open-weight model. Its architectural and scaling analyses are well executed, yet headline benchmark results are sobering: IQuest-Q1 trails Claude Opus 5 and GPT-5.6 Sol on key software-engineering tasks such as SWE-Bench Pro (57.5 versus 79.2) and Terminal-Bench 3.0 (17.9 versus 42.7). The 'human-on-the-loop recursive self-improvement' claim rests on a single anecdote rather than systematic evidence. As the authors themselves concede, 'Expert-level reasoning remains challenging: HLE scores of 39.2 without tools and 55.7 with tools indicate room for improvement in both settings.'
The controlled scaling analysis and post-training engineering are the paper’s strongest parts. Using identical training data, the authors show that 'IQuest-Q1 exhibits a 1.93× higher scaling efficiency than the baseline, requiring 1/1.93≈52% of the training compute to reach the same training loss,' consistent with independent Muon literature showing '~2× computational efficiency compared to AdamW with compute optimal training.' The Critical Model Size framework—defining the scale where $\eta_{\mathrm{opt}}(N_{\mathrm{CMS}}) = \eta_{\mathrm{safe}}(N_{\mathrm{CMS}})$—is a clean conceptual extension of critical-batch-size analysis, and the multi-layered anti–reward-hacking safeguards reflect commendable engineering rigor.
Benchmark performance is notably behind frontier models: IQuest-Q1 lags Claude Opus 5 by more than 20 points on SWE-Bench Pro and trails significantly on Terminal-Bench 3.0 and 4.0. Many competitor scores in Table 2 are marked with asterisks, denoting 'from our own testing,' which raises the risk of harness mismatch and evaluation bias rather than fair comparison. No ablation ties architectural innovations—Stable SplitNorm, Muon, or $\mu$P—to downstream task gains; the 1.93× efficiency claim rests solely on training-loss curves.\n\nThe recursive self-improvement framing is overstated. Section 6.2 offers a single anecdote where the model diagnosed tool-call failures in one task, and the authors admit that 'This iteration demonstrates the potential of iterative model development with IQuest-Q1, while also highlighting the importance of human supervision.' That is assisted debugging, not systematic recursive self-improvement.
Evidence for the headline agentic claims is mixed. While CyberGym (84.5) and SkillsBench (74.1) are competitive, the model struggles on expert reasoning—'HLE scores of 39.2 without tools and 55.7 with tools indicate room for improvement in both settings'—and on the hardest terminal benchmarks. The comparison to DeepSeek-V4-Flash-0731 rather than the stronger Pro variant may flatter results, and the asterisked competitor scores ('Results with * are from our own testing') undermine confidence because harness parity cannot be independently verified. Reproducing the exact evaluation setup—including harness versions and time limits up to 12 hours—would be required to validate these numbers.
The open release of model weights is a genuine positive. However, independent training reproduction is blocked by insufficient specifics: the 29T-token corpus is described only at a high level ('web pages, public GitHub code, books, academic papers, news, and domain-specific materials'), exact data-mixture ratios and synthesis code are undisclosed, and the model-merging recipe uses 'prescribed weights and global $\ell_2$-norm calibration' without defining those weights or the reference checkpoint selection logic. The proprietary IQuest-CLIBench further prevents full external verification, as the authors note it is 'our proprietary built-in benchmark for evaluating human-agent collaboration and user experience.'
We present IQuest-Q1, a sparse Mixture-of-Experts (MoE) agentic foundation model with 320B total parameters (15B activated). IQuest-Q1 is developed through a three-stage process: (1) stable and efficient pre-training, combining Stable SplitNorm, Muon, and muP with a two-stage curriculum whose second stage increases the coding-data mixture, preserving a wide stability margin between safe and optimal learning rates; (2) fine-grained mid-training, expanding the data mixture across domains and formats and progressively extending the context; and (3) multi-scaffold with multi-version post-training, combining SFT, domain-specialized RL, and MOPD across various scaffolds to integrate complementary training signals and strengthen agentic coding capabilities. Furthermore, model merging combines different experts and checkpoints from different stages while mitigating conflicts among their parameter updates. The resulting IQuest-Q1 demonstrates capabilities in both coding and general tasks. We release IQuest-Q1 as an open-source model to support community research and development.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.