Riemannian Geometry Speaks Louder Than Words: From Graph Foundation Model to Next-Generation Graph Intelligence
This paper proposes Riemannian Foundation Model (RFM), a vision for unifying graph learning through Riemannian geometry rather than GNN message-passing or LLM serialization. The authors argue that graphs are discrete analogs of manifolds, and that concepts like vector bundles, curvature, and parallel transport provide the proper toolkit for universal graph modeling—enabling both structural inference and generation in a way that current Euclidean GNNs and tokenized LLMs cannot achieve.
This is an ambitious position paper ("blue sky idea") that attempts to reframe graph foundation modeling around differential geometry rather than incremental architectural improvements. The critique of existing approaches is incisive: GNNs are correctly identified as crippled by oversmoothing and fixed Euclidean biases, while LLM-based methods are aptly criticized for fragmenting topology into path-dependent traversals. However, the RFM proposal remains largely at the conceptual level, with vague operationalization of how discrete graphs map to continuous manifolds in practice, and insufficient distinction from prior non-Euclidean graph learning work (e.g., hyperbolic GNNs) beyond philosophical appeals to "intrinsic" versus "extrinsic" geometry.
The diagnosis of current limitations is compelling. The observation that graph serialization for LLMs offers only "fragmented, path-dependent views" while "fall[ing] short of capturing high-order structural patterns" accurately captures why token-based approaches struggle with topological reasoning. The mathematical toolkit invoked—vector bundles for heterogeneous modalities, parallel transport for knowledge transfer, and curvature for measuring inference consistency—is theoretically sound and genuinely appropriate for graph data. The research agenda sketched in Section 4, progressing from universal structural understanding to Riemannian LLM integration, provides a coherent roadmap for the community.
The central conceptual distinction—RFM studies "intrinsic" geometry while prior work uses "extrinsic" representation spaces—is philosophically appealing but computationally underspecified. The paper does not clarify how RFM differs algorithmically from existing hyperbolic or spherical GNNs (Liu et al., 2019; Sun et al., 2021) beyond terminology. The claim that RFM "recovers the manifold underlying graphs" is hand-wavy: given a finite discrete graph, what is the metric, how is curvature computed locally, and what is the connection's Christoffel symbol construction? Without these details, the framework risks being unfalsifiable. Table 1 claims graph generation is absent from state-of-the-art GFMs, but models like GRAVER (Yuan et al., 2025a) and others with generative components are listed in that same table, creating confusion. Finally, as a pure vision paper, it offers no experiments, algorithms, or complexity analysis here—though it cites the authors' prior RiemannGFM work (Sun et al., 2025a) for implementation.
The comparison to geometric deep learning (Bronstein et al., 2017) and non-Euclidean graph learning is fair but incomplete. The paper acknowledges these approaches but claims they focus on "extrinsic representation spaces" whereas RFM studies "intrinsic" properties. This distinction is subtle: hyperbolic GNNs also learn curved embeddings that reflect graph hierarchies. The evidence that LLM-based methods fail to capture structure is anecdotal rather than empirical (no citation of specific failure modes), and the capability matrix in Table 1 lacks citations or methodology for how capabilities were assessed. The paper's strongest comparative point—integrating LLMs via vector bundles with Euclidean fibers—is theoretically elegant but unvalidated.
As a "blue sky" vision paper, reproducibility in the traditional sense is not applicable. However, the paper fails to provide even conceptual algorithms, pseudocode, or architectural specifications that would allow others to implement RFM. Key missing details include: (1) how discrete graph adjacency maps to a Riemannian metric $g_{ij}$, (2) how parallel transport is computed between non-adjacent nodes, (3) the dimensionality and topology of the base manifold, and (4) training objectives for recovering the manifold structure. The authors reference their prior WWW 2025 paper (RiemannGFM) for implementation details, suggesting this paper is a conceptual companion rather than a technical contribution. No code, data, or hyperparameters are provided here.
Graphs provide a natural description of the complex relationships among objects, and play a pivotal role in communications, transportation, social computing, the life sciences, etc. Currently, there is strong agreement that Graph Foundation Models (GFMs) are essential for advancing graph learning, yet considerable disagreement persists on how to build a powerful, general-purpose GFM analogous to Large Language Models (LLMs). Graph Neural Networks (GNNs) exhibit limitations in memory retention and principled interpretability when confronted with multi-domain pretraining and adaptation. The challenge of graph serialization hinders the direct application of LLMs, as the words struggle to capture the structural complexity and diversity inherent in graphs. In contrast, Riemannian geometry offers an elegant mathematical framework for modeling structures, while remaining compatible with graph semantic learning, even with LLMs. In this paper, we argue that, for graphs, Riemannian geometry speaks louder than words, and lay out the foundational principles for GFM. Reimagining with Riemannian geometry, we introduce a blue sky idea-Riemannian Foundation Model (RFM)-that opens a new pathway for capturing complex structural patterns and uncovering cross-domain generalities. RFM emphasizes intrinsic graph geometry and embodies endogenous capacities for structural inference and generation, moving beyond mere representation-space switching. Accordingly, we outline a progressive agenda that begins with universal structural understanding through intrinsic geometry, and then rebuilds LLM with a Riemannian engine for general-purpose graph modeling and beyond. Thus, RFM enables a paradigm shift from designing graph models to solving graph-structured applications with RFM agents, unlocking the next-generation graph intelligence.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.