Outperforming 1000 Euclidean Layers With Hyperbolic Goal-Conditioned RL
Goal-conditioned RL in complex mazes suffers from a geometric mismatch: Euclidean embedding distances model straight-line proximity, yet walls can make actual route distances highly tree-like and deceptive. This paper introduces Hyperbolic CRL, a drop-in replacement of the critic's Euclidean distance with a hyperbolic geodesic distance on the hyperboloid. Its central finding is that this geometric alignment allows a depth-64 hyperbolic network to outperform a 1024-layer Euclidean baseline on hard humanoid maze tasks while training more than $12\times$ faster and using roughly $13\times$ fewer parameters.
The paper presents a compelling and well-motivated case that representation geometry can substitute for extreme depth in goal-conditioned RL. The geometric analysis in Section 4 elegantly links Gromov $\delta$-hyperbolicity of maze environments to the advantage of hyperbolic embeddings, and the empirical gains on hard navigation tasks are dramatic. The authors candidly acknowledge limitations, notably that their default curvature $c=0.5$ is not optimal for all environments. Still, the headline comparisons rely on pitting a depth-64 hyperbolic agent against a depth-1024 Euclidean baseline evaluated primarily on select tasks, which slightly narrows the scope of the claimed universal efficiency advantage.
The strongest contributions are the principled geometric case study and the behavioral analysis. Section 4 directly connects the increasing tree-likeness of route distances in Ant U5-Maze to the hyperboloid's structure, and gradient-alignment experiments show that Euclidean critics direct agents toward walls while hyperbolic critics do not. The behavioral evidence is equally intuitive: the authors observe that "The Euclidean agent takes a straight-line shortcut: it runs at the wall, vaults over it, lands in the goal region, and falls, which ends the episode. The hyperbolic agent instead walks around the corner and stands at the goal." These qualitative and quantitative results align tightly with the paper's core thesis.
A primary concern is that the default curvature magnitude $c=0.5$ is not universally beneficial. The sensitivity analysis in Section 5.4 shows that "on Humanoid, hyperbolic agents improve with depth like Euclidean agents, but remain well below them from depth 16 on," with the deficit attributed to the fixed curvature. Because Humanoid lacks walls, this is expected, yet it undermines the claim that hyperbolic CRL is an unqualified drop-in improvement across all benchmarks without per-environment tuning. Second, the paper's framing of Euclidean shortcut behavior as a failure partly repurposes a capability reported by Wang et al. (2025) as a flaw, and the offline gains on OGBench, while positive, are relative improvements on low baselines that leave absolute success rates on the hardest mazes modest. Finally, the symmetric geodesic distance cannot represent asymmetric one-way transitions, a limitation the authors note but do not resolve.
The evidence strongly supports the maze-specific claims. Hyperbolic critics achieve higher Spearman correlation with route distance than Euclidean critics, and in the route-ordering test they correctly rank goals with similar straight-line but different route distances far above chance; the authors state that "This aligns with our previous analysis of Figure 2(c), showing that hyperbolic critics order such goals by route distance more reliably than Euclidean critics," which underpins the improved stitching results. However, comparisons to prior work require caution: the authors explicitly note that "Our Humanoid results are not directly comparable to Figure 1 of Wang et al. (2025)" due to a Brax timestep discrepancy, and the headline speedup metric is a depth-vs-depth comparison rather than an iso-parameter controlled ablation.
Reproducibility is aided by extensive hyperparameter tables and architecture schematics in Appendix D, and the authors state that "our code is included in the submission and will be made publicly available upon acceptance." Still, independent reproduction may be complicated by reliance on a custom library (hyperbolix) for JAX hyperbolic operations and by the subtle Brax timestep issue that breaks comparability with Wang et al. on the Humanoid task. Moreover, the wall-clock speedup and memory claims hinge on comparing depths of 64 versus 512 and 1024, so they do not isolate the runtime cost of the hyperbolic head alone; at equal depth the hyperbolic head only modestly lowers throughput by about 6.27%.
Scaling self-supervised contrastive reinforcement learning (CRL) to 1000-layer Euclidean neural networks has enabled state-of-the-art goal-reaching performance, yet wall-clock time scales linearly with depth. We find that changing the geometry achieves stronger performance with far shallower networks. This matters in complex navigation environments, where walls can make route distance deviate substantially from Euclidean distance. In a U-shaped maze, we show that the longer the route, the more it deviates from Euclidean distance. Hyperbolic CRL addresses this mismatch by replacing Euclidean distances in CRL's contrastive objective with geodesic distances in hyperbolic space. On hard navigation tasks, inductive bias beats scale: depth 64 hyperbolic agents outperform 1024-layer-deep Euclidean networks while training more than $12 \times$ faster and using about $13 \times$ fewer parameters. We observe that hyperbolic agents behave qualitatively better in hard humanoid mazes, where they follow corridors to reach goals, whereas Euclidean agents try straight-line shortcuts over walls. Our results show that matching the representation geometry to the task can substitute for substantial network depth. Code is at https://github.com/Probabilistic-and-Interactive-ML/HypGCRL .
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.