Harnessing Language Models as Optimization Modeling Agents
This paper tackles the challenge of translating natural-language optimization problems into correct, executable solver programs using frozen LLMs. It introduces OptiCycle, a lightweight inference harness that iterates code generation, execution, and repair in a persistent Python state, and OptiState, which adds an explicit five-element revisable formulation. Across nine benchmarks spanning 2,634 problems and 11 Qwen variants, execution-guided revision produces broad accuracy gains and outperforms heavier multi-agent workflows and post-trained specialist models without any parameter updates, though the extra formulation structure helps only some backbones.
The paper makes a strong, well-controlled empirical argument that inference-time execution loops can dominate both task-specific architectures and additional training for optimization modeling. By fixing the backbone and grading protocol while varying only the harness, the authors obtain credible gains and are careful to note where their claims stop. The work is practically motivated and methodologically transparent about its limitations.
The unified harness abstraction cleanly separates model capabilities from inference procedure, and the experimental design keeps model weights, benchmarks, and grading constant. The gains from executable interaction are large and consistent: OptiCycle raises macro accuracy by up to 49.46 percentage points over direct generation and achieves 65.21% on Qwen3.5-9B, exceeding the strongest competing baseline by 4.29 points while using fewer tokens. The cost analysis adds practical value, showing OptiCycle consumes 11.77k tokens per trajectory versus 14.54k for Reflexion and 34.38k for AutoFormulation.
The value of OptiState’s explicit formulation state is erratic and not well explained: Qwen3-14B, Qwen3-8B, and Qwen3.6-27B show little to no macro gain over OptiCycle, and some backbones lose accuracy on individual benchmarks like ComplexOR and OptMath. The authors describe this as model-dependent but offer no mechanistic diagnosis. The evaluation metric is also narrow—correctness is judged solely by objective-value agreement or infeasible/unbounded status, ignoring constraint satisfaction, decision-vector feasibility, and formulation equivalence.
The comparisons further mix method-specific inference budgets rather than matched steps or tokens. The authors concede that matched-budget ablations are still needed to isolate the value of individual components, which weakens cost-adjusted conclusions.
The core evidence—that executable interaction beats direct generation and six adapted agent baselines under shared backbones—is solid and clearly tabulated. However, the comparison with post-trained specialist models in Figure 4 mixes different families and sizes, making it illustrative rather than rigorous. The claim that formulation state is selectively beneficial is fair given the mixed per-benchmark results, yet the paper does not ablate the initial formulation call from the revisable workspace itself, so it is unclear which sub-component drives the gains on the subset of models that do improve.
Sampling hyperparameters, model lists, and solver environments are documented, but the exact system prompts, feedback summarization rules, state-editing instructions, and controller logic are not released in the manuscript. The authors note that configuration revisions differ across model generations, implying that prompts were hand-tuned per family. Without code or templates, independent reproduction would require substantial guesswork. On the positive side, the authors report missing scores, timeout exceptions, and scoring rules transparently.
Large language models (LLMs) can translate natural-language optimization problems into solver programs, but reliably producing correct formulations and implementations remains challenging. Existing approaches improve optimization modeling through agentic workflows or task-specific post-training. This raises a practical question: can a lightweight harness using a single model role achieve competitive performance without additional training? We introduce OPTICYCLE and OPTISTATE, two lightweight inference harnesses for optimization modeling, and evaluate them on nine benchmarks comprising 2,634 problems using 11 Qwen variants spanning 0.6B-27B parameters. Our results show that execution-guided revision consistently improves macro accuracy over direct generation for all evaluated models, with gains of up to 49.46 percentage points. Despite their simplicity, both harnesses outperform six adapted agentic workflows in same-model comparisons and four evaluated post-trained optimization models at comparable model sizes, without additional training. Explicit formulation state provides further but model-dependent improvements, suggesting that additional structure is not universally beneficial.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.