TRACE: Tackling Real-World Resource Assignment Problems via Agentic Heuristic Design
TRACE targets dynamic resource assignment in industrial systems where processing times, energy profiles, and queue capacities are unknown a priori. While existing LLM-based Automatic Heuristic Design learns only from scalar fitness scores, TRACE couples an evolutionary loop with an agentic workflow that distills noisy system logs into insights and executable surrogate models. This feedback turns blind trial-and-error into data-driven design, yielding heuristics that cut cost by over an order of magnitude relative to prior AHD methods and reduce energy by 65% against industry standards without sacrificing reliability.
TRACE makes a strong case that scalar-feedback AHD is insufficient when the full system model is hidden from the LLM. The architectural separation between a frontier Reasoner and an open-weight Coder/Evolution loop is sensible, and the empirical gains—especially the roughly $36\times$ cost reduction over EoH in the $N=50$ cloud scenario and the $25\%$ energy reduction over EoH in the 5G vRAN case—are large and consistent across runs. However, the abstract claims evaluation on "up to 5050 heterogeneous servers" while the experiments only report $N=10$ and $N=50$, undermining a key scalability narrative. Other limitations include a reliance on LLM judges for auditability and the absence of a public code release.
The core insight—that scalar fitness reveals what works but not why in partially specified systems—is sharply formulated and directly addressed. The agentic workflow elegantly circumvents feeding massive raw logs to an LLM by having a Coder generate schema-specific extraction code, and the ablation showing that Coder failure sharply degrades performance while swapping the Reasoner model does not confirms that the gains are architectural rather than due to a stronger backbone. The inclusion of a measurement-driven 5G vRAN scenario built from industrial testbed data also elevates the work above purely synthetic benchmarks.
A glaring inconsistency exists between the abstract’s claim of evaluation on "up to 5050 heterogeneous servers" and the actual experiments, which only evaluate $N=10$ and $N=50$ nodes (Table 1). This unsupported leap damages the scalability narrative. The auditability study, while interesting, relies entirely on LLM-as-a-judge scoring of a single heuristic per method, which risks circularity when the judges evaluate LLM-generated code; although a three-panel design mitigates some bias, it remains a weak substitute for human operator validation. Additionally, the 5G evaluation compares TRACE only against EoH among AHD baselines, omitting HSEvo—the best baseline in the smaller cloud scenario—so the 5G claim of superiority over SoTA AHD is narrower than advertised.
The evidence strongly supports TRACE’s dominance over scalar-feedback AHD in the reported settings. The authors correctly note that prior frameworks such as EoH rely exclusively on scalar fitness, which the EoH paper confirms by evaluating only the quality of each generated heuristic on problem instances. In the cloud benchmark, the distribution over 10 runs shows that in Scenario 2 every TRACE run beats every baseline run, and the gap widens as the number of processors grows. The 5G vRAN scenario is grounded by empirical lookup tables from an industrial testbed and operational traffic traces, lending more external validity than stylized combinatorial benchmarks. However, the comparison to Oracle-Time and Oracle-Queue is uninformative because these oracles use privileged information unavailable to any practical scheduler, and the absence of conventional human-designed heuristics or DRL policies in the 5G comparison leaves a conspicuous gap in the baseline landscape.
The paper reports detailed hyperparameters ($E=20$, $I=150$, $E_{\max}=5$, $M_{\max}=10$), the full generation prompts (Appendix H), and specific LLM versions (Qwen3-30B-A3B for generation, Gemini-3-Pro for reasoning), which aids reproduction. The underlying 5G dataset is public via Zenodo. However, no source code repository is mentioned, the sandbox environment for executing generated Coder artifacts is only loosely specified, and the random seeds for the 10 independent runs are not provided. Consequently, independent reproduction would require reimplementing both the simulator and the agentic workflow from scratch.
Dynamic resource assignment, the real-time allocation of task streams to heterogeneous processing nodes, is the backbone of modern computing infrastructure. While learning-based schedulers excel in research, industrial deployments still rely on hand-written rules that operators can read, audit, and execute within tight latency budgets. LLM-based Automatic Heuristic Design (AHD) promises to automate writing such rules. However, existing AHD frameworks were developed for combinatorial problems fully specified to the LLM, and they learn only from a scalar fitness score. In real systems, the behaviour that determines a good heuristic, such as processor speeds or power consumption, is unknown a priori: the score reveals which heuristic performs better, but not why. This missing information is recorded in the system logs that every evaluation produces. Exploiting it is non-trivial: logs are massive and noisy, the relevant signals depend on the objective, and their content and format vary across hardware and software stacks, so they can neither be fed to an LLM as is nor processed by a fixed parser. We propose TRACE, which couples an evolutionary AHD loop with an agentic knowledge-extraction workflow. A Reasoner agent analyzes the log schema in light of the objective and formulates hypotheses about the system dynamics; a Coder agent writes and executes schema-specific code to test them, producing insights or executable tools for the evolved heuristics. We evaluate TRACE on a synthetic cloud benchmark and a 5G vRAN scenario built from industrial testbed measurements and operational traffic traces. TRACE consistently outperforms state-of-the-art AHD methods in resource assignment problems and yields more auditable heuristics at under 2% overhead.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.