STEER: Reducing Inference Cost in Relational Foundation Models through Semantically Informed Sampling
STEER attacks the high inference cost of relational foundation models by shrinking their graph-sampled context using schema semantics instead of random dropping. The approach prompts an LLM once per task to rank foreign-key edges into relevance tiers, then probabilistically steers neighborhood traversal during sampling. Evaluated on RelBench across three state-of-the-art RFMs, it cuts average context size by about 40% while maintaining or slightly improving accuracy, offering a practical, model-agnostic optimization.
STEER is a well-motivated and cleanly executed idea that replaces uninformed graph traversal with a lightweight, reusable LLM-based relevance filter. The paper shows consistent context reduction across both transformer- and GNN-based relational foundation models, and the amortized cost of two LLM API calls per task is clearly justified. However, the work stays within a narrow experimental scope—only eight databases on one benchmark—and leaves several design choices, such as the fixed default probability profile and the optional column filter, insufficiently justified for broader claims of generality.
The core claim holds: semantic guidance substantially outperforms naive context reduction. The comparison to naive pruning is devastating for random dropping—naive 50% pruning causes severe accuracy degradation, while STEER achieves comparable or larger size reductions without loss. The probabilistic filtering design is also elegant; as the authors note, relevance is rarely binary, and probabilistic edge inclusion naturally handles gradations while hedging against LLM ranking errors. Implementation across three heterogeneous samplers supports the claim that STEER is model-agnostic.
A primary concern is the reliance on a universal, fixed probability profile mapping tiers to inclusion probabilities ($\pi(\textit{Very Strong})=1.0$, $\pi(\textit{Strong})=0.6$, $\pi(\textit{Moderate})=0.2$, $\pi(\textit{Weak})=0.0$). The authors admit this is not tuned per task, and while an ablation shows task-specific tuning can help, it is described as costly; this leaves the default profile essentially an unprincipled constant set that happens to work on the benchmark, with no guarantee of transfer. The optional column filter further undermines broad claims: it improves sample size but produces mixed accuracy outcomes, including a 3.11 percentage point drop in RT-J regression $R^2$ and mixed results elsewhere.\n\nThe evaluation is also limited to 24 tasks across 8 databases on RelBench, with no analysis of how schema complexity, graph diameter, or multi-hop relevance affects the LLM ranker. Finally, the paper relies heavily on future-dated citations (2025–2026) for foundational RFMs and benchmarks, suggesting the experimental setting may not yet be fully stabilized.
The evidence adequately supports the central claim that STEER reduces context without sacrificing accuracy, but comparisons to related work are mostly conceptual rather than empirical. The paper positions STEER against naive pruning and full-context inference, yet it does not compare against simpler heuristics such as degree-based or frequency-based edge selection, nor against learned task-specific samplers. The related work discussion cites prior GNN sampling methods but does not benchmark against them, arguing that STEER suits the zero-shot RFM setting. This is fair, but without runtime or monetary cost comparisons to alternative compression strategies, the practical advantage is partially asserted rather than measured. The per-task probability tuning ablation (Tables 3 and 4) also reveals high variance: tuning sometimes helps dramatically but can also severely hurt generalization, which somewhat weakens the endorsement of the default profile as universally robust.
Reproducibility is relatively strong. The authors release source code, data, and model checkpoints at a public GitHub repository, and they document hardware (8x NVIDIA H100), model configurations (context length 1024 for RT and RT-J, fanout 20 and depth 2 for Griffin), and evaluation budgets (1280 items per task). However, reproducing the full results requires access to frontier LLM APIs at high reasoning effort, which incurs monetary cost and introduces potential API drift. The LLM outputs themselves—the tiered edge rankings—can vary with model versions, and no inter-annotator stability or prompt sensitivity analysis is provided beyond a limited prompting-strategy ablation. Additionally, since RT and RT-J checkpoints are not standard commercial artifacts and the Griffin checkpoint was obtained via personal communication, independent reproduction depends on the continued availability of these specific non-standard model weights.
Relational foundation models (RFMs) are pretrained once on a collection of relational databases and prediction tasks, and then applied zero-shot to previously unseen databases and tasks. To make a prediction for a target row, an RFM samples a neighborhood of rows linked to that row through foreign keys and uses this neighborhood as its inference context. Lowering inference cost is an important goal for any foundation model, and for RFMs this cost grows with the size of the context. The simplest ways to shrink the context is to drop some of the sampled rows, but this ignores the semantics of the database schema, so it is as likely to discard informative rows as uninformative ones. We propose STEER, a sampling approach that shrinks the inference context by concentrating it on the tables most relevant to the prediction task at hand. STEER obtains relevance information by prompting a large language model to rank the foreign-key edges of the database schema into relevance tiers for the given task, and then maps each tier to a probability of following that edge during traversal. Because the ranking uses only the schema, it is computed once per task and reused across all subsequent predictions, amortizing its cost. We evaluate STEER on three state-of-the-art RFMs (RT, RT-J, and Griffin) and show that it reduces inference context size by about 40% on average while maintaining, and in some cases improving, prediction accuracy.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.