Higher-Order Molecular Grammars for Generative and Foundation Models in Chemistry
This paper introduces Higher-order Grammar Representation (HGR), a method that lifts molecular graphs into combinatorial complexes and parses them into compact sequences of context-free production rules, making higher-order topology such as ring systems tractable for standard sequence models. To reduce benchmark bias toward simple rings, the authors also construct RingDiv, a ring-enriched benchmark of 1.18 million molecules. HGR is instantiated in generative models (HGR-VAE and HGR-LDF) and a foundation model (HGR-FM), with claims of 100% validity by construction and state-of-the-art distributional alignment and transfer performance.
HGR is a conceptually clean and empirically strong contribution to molecular representation learning. By serializing combinatorial complexes as grammar rule sequences, the authors obtain exact reconstruction together with 100% RDKit-valid generation, while achieving leading distributional alignment on several benchmarks and the highest mean AUC on seven MoleculeNet tasks under both probing and fine-tuning. The paper is dense and the scope of guarantees is carefully circumscribed, though questions remain about scalability to billion-scale corpora and whether the gains are specific to the grammar formulation or partly attributable to the induced hierarchical tokenization.
The validity guarantee and the sublinear corpus growth are well-supported. The ablation isolating higher-order tokens is informative, showing that removing them degrades pretraining dynamics and downstream transfer. The ring-diversity critique of existing benchmarks is backed by systematic indices (RDI and QRCI), and HGR-LDF's FCD leadership on RingDiv300k is convincing.
Several limitations warrant attention. First, the upfront lifting cost and rule-corpus growth, while sublinear, have not been demonstrated at billion-molecule industrial scales. Second, generation remains biased at the motif level; the acknowledged ether enrichment under HGR-VAE shows the grammar alone does not eliminate all distributional artifacts. Third, the 100% validity guarantee explicitly excludes stereochemistry and synthetic accessibility: the rules encode only recorded local structures, so global chemical plausibility beyond valency is not assured. Fourth, the foundation-model comparisons focus on graph-pretraining baselines and do not include recent large-scale chemical language models, leaving open the generality of the transfer gains.
Additionally, the marginal gain from the attention bias in HGR-FM is smaller than that from the higher-order tokens themselves, raising the question of whether much of the benefit could be captured by a simpler hierarchical tokenization without the full composite attention machinery.
The generation comparisons span diffusion, grammar, and fragment-based baselines across five benchmarks. HGR's validity and FCD leadership on GuacaMol and RingDiv300k are clear, and the size-resolved JS divergences support fidelity on large, ring-rich molecules. However, the evidence is not uniformly favorable: HGR-LDF trails diffusion baselines on the GuacaMol KL score, and the structural metrics (NSPDK MMD, curvature) are largely relegated to the supplement. The argument that conventional benchmarks are saturated is only partially mitigating, because HGR is still evaluated on those benchmarks where it does not uniformly dominate.
Code is available via GitHub and RingDiv is promised at publication. Model configurations, optimisers, and training hyperparameters are detailed in Sec. 3.4. The grammar induction and lifting algorithms are given in supplementary pseudocode, which should allow independent implementation. Results are reported over three random seeds, though main-text tables show means only and standard deviations are deferred to the supplementary results. A remaining blocking factor is the sensitivity of the motif-vocabulary construction (frequency thresholds, merge criteria), which is not fully ablated.
Molecular learning models are strongly shaped by their underlying representations. Yet standard sequential and graph formalisms struggle to explicitly encode higher-order topology, such as ring systems and recurring motifs. Existing higher-order representations can capture these structures directly, but they are often computationally demanding and difficult to decode into valid molecules. Here, we introduce Higher-order Grammar Representation (HGR), a principled, topology-aware framework that lifts molecules to combinatorial complexes and parses each complex into a compact sequence of production rules under a context-free higher-order grammar. By serialising higher-order topology into rule sequences, HGR makes these structures directly compatible with standard sequence models, avoiding the computational overhead of explicit higher-order encodings while preserving topological expressiveness. To reduce benchmark bias towards simple ring systems, we construct RingDiv, a ring-enriched benchmark containing 1.18 million molecules, including the curated RingDiv300k subset, and introduce the ring diversity index (RDI) to quantify ring-system coverage. In molecular generation, HGR-based models uniquely combine 100% validity by construction with leading distributional alignment, ranking first in FCD on all five generation benchmarks. In representation learning, HGR-FM achieves the highest mean AUC across seven MoleculeNet benchmarks under both transfer protocols, improving on the strongest baseline by 8.3 and 3.3 AUC points under probing and full fine-tuning, respectively. Collectively, these results establish HGR as an efficient higher-order representation for molecular generation and transferable representation learning.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.