RelICL: Training-free Relational Learning with Tabular Foundation Models
Relational learning with tabular foundation models currently relies on Deep Feature Synthesis (DFS), which flattens multi-table schemas into single tables via column-wise aggregations along meta-paths. This paper identifies two critical limitations of DFS—feature explosion and interaction blindness—and proposes RelICL, a training-free method that instead propagates row embeddings through the schema graph using the foundation model itself. By iteratively pooling and fusing embeddings, RelICL keeps the TFM-agnostic benefits of DFS while scaling to complex schemas and capturing cross-column interactions. The experimental study on 21 RelBench tasks shows RelICL matches state-of-the-art DFS approaches, though it does not dominate them outright.
The paper presents a conceptually clean and practically motivated alternative to DFS for in-context relational learning. The core idea of reusing a tabular foundation model's encoder for both intermediate feature extraction and final prediction is elegant, and the theoretical propositions formally establish inherent limitations of DFS-based flattening. However, the empirical evaluation, while showing parity with strong DFS baselines, does not demonstrate a clear universal win on real-world benchmarks, leaving the practical urgency of solving interaction blindness and feature explosion somewhat muted for typical schema depths. The contributions are solid but incremental because the method trades one set of approximations for another without a decisive empirical advantage on standard tasks.
The theoretical analysis of DFS limitations is precise and compelling. Propositions 1 and 2 constructively prove that bounded-depth DFS can fail when predictive signals are distant or depend on column interactions, and the synthetic experiments validate this unambiguously: RDBLearn drops to random chance on both tasks while RelICL solves them perfectly. The method's design is also commendably simple and modular, requiring no training and allowing any TFM that produces row embeddings to be plugged in as a backbone, which directly supports the paper's claim of model agnosticism.
The primary concern is the limited empirical separation on real-world benchmarks. Despite identifying severe theoretical flaws in DFS, RelICL achieves only rough parity with TabPFN-Rel on RelBench and often trails trained models like RelGNN or KumoRFM-2, suggesting that pathological cases may be rare in practice or that heuristic DFS pruning already suffices for typical depth bounds.
A second concern is the fragility of Assumption 1. The paper shows that replacing features by row embeddings can significantly hurt performance on some tables, yet it provides no quantitative analysis of how often this failure mode occurs during multi-hop propagation in RelBench, where row embeddings are fed back through the encoder repeatedly.
The evidence strongly supports the conceptual claims: RelICL avoids feature explosion and interaction blindness by design, and the synthetic tasks in Tables 1 and 2 provide unambiguous confirmation. However, comparisons to related work on RelBench are confounded. The authors rely on published baseline numbers that were not reproduced and correctly note that wins may not be statistically significant. Moreover, TabPFN-Rel includes orthogonal feature engineering that boosts its results, and no ablation isolates the contribution of RelICL's embedding propagation from the ensemble and hyperparameter selection strategy used in the reported configuration.
Reproducibility is mixed. The authors release source code and report all 11 hyperparameters with their search spaces and selected values, which is commendable. Yet the default configuration is an ensemble over multiple settings found via random search on a 12-task subset, consuming 121 GPU-hours and 1200 trials. This makes independent reproduction costly and raises questions about whether reported gains stem from architectural innovation or from extensive tuning and ensembling. Additionally, reliance on specific TFM checkpoints without guaranteed long-term availability remains a practical barrier.
Tabular foundation models achieve state-of-the-art performance on single-table tasks without any training. Recent work suggests that they are also well-suited for relational learning via deep feature synthesis (DFS), which flattens a relational schema into a single table by adding aggregates of the other tables' columns as features. This approach is appealing because it directly benefits from improvements to or customization of the underlying tabular foundation model. In this paper, we identify two key problems with DFS: feature explosion and interaction blindness. The first problem arises because the number of DFS features grows quickly as the schema becomes more complex, limiting scalability and performance. The second problem arises because column-wise aggregates do not account for feature interactions, limiting performance. We propose and explore an alternative method termed RelICL, which keeps the benefits of DFS but alleviates these two problems. At its heart, RelICL propagates and fuses information step by step through the schema graph, using the same tabular foundation model that is eventually used for prediction to do so. In our experimental study using RelBench tasks, RelICL was on par with the strongest approach based on deep feature synthesis.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.