Predicting Unseen Scientific Interactions: Conditional Flow Models Hallucinate Non-Additive Effects
Conditional generative models are increasingly used to predict biological and scientific responses under unseen experimental conditions. This paper investigates whether flow-matching models accurately recover non-additive interactions between factors or instead hallucinate them when extrapolating beyond the training support. The authors propose decomposing interaction errors into recovery and hallucinated components and find that aggregate fidelity metrics often conceal failures to learn interactions, while successful extrapolation depends critically on whether condition structure continues smoothly from observed to held-out regions.
The paper presents a timely and methodologically clean investigation showing that conditional flow models often fail to recover non-additive interactions under extrapolation, and that standard endpoint errors can hide these failures. The decomposition of interaction error into recovery ($\alpha$) and hallucination ($\beta$) is elegant and empirically useful across four scientific domains. However, the framing of inductive-bias-driven extrapolation as uniquely problematic 'hallucination' is slightly overstated, and the restriction to mean-based two-factor contrasts limits the generality of the conclusions.
The proposed geometric decomposition $\mathcal{E}_I^2 = (1-\alpha)^2 + \beta^2$ cleanly separates interaction recovery from hallucinated structure and is a genuine methodological contribution. The controlled PPG simulator experiments are a standout: by tuning the interaction strength $\gamma$ down to zero while holding main effects fixed, the authors provide causal evidence that models generate non-additive structure even in fully additive regimes. The finding that endpoint-error and interaction-error learning curves decouple---with checkpoint selection based on endpoint error choosing models whose interaction predictions are already degrading---is practically important and persuasively demonstrated.
The work is limited to mean-based contrasts of two factors, leaving open whether distributional or higher-order interaction failures follow the same patterns. A more subtle concern arises in the molecular conformer experiments, where $\alpha$ actually increases as training support becomes sparser; the authors trace this to exaggerated between-family scaling rather than improved within-family recovery, yet a reader relying on $\alpha$ alone could be misled. Finally, in underdetermined regimes the 'hallucination' label feels overstated: when held-out interactions are not identified by the training support, any model prediction necessarily reflects inductive bias, and castigating flow models for non-zero $\beta$ without comparing against deterministic regression baselines in all domains leaves the unique role of generative architectures unclear.
The evidence strongly supports the central warning that aggregate fidelity metrics can conceal interaction failures, with clear cross-domain demonstrations in a synthetic simulator, drug screens, perturbation-seq, and molecular conformers. Related work is cited fairly, and the distinction between smoothness-identified, relationally-identified, and underdetermined regimes provides a useful taxonomy. That said, the paper does not always isolate whether these are peculiarities of flow matching or of the broader supervised-learning setup; for instance, dose-response recovery is noted to be nearly identical to a deterministic regression baseline, yet similar controlled comparisons are absent in the other three domains.
Reproducibility is reasonably strong: code is publicly linked, and Appendix Table 5 provides model architectures, hyperparameters, optimizer settings, and inference details for all four domains. The PPG simulator is deterministic given a fixed photon-path bank, which aids exact replication. However, the nested hold-out splits in the PPG grid and the molecular conformer 'chemical squares' rely on specific geometric and graph-matching criteria that are described but not exhaustively codified in the main text, and training runs involve up to 1M steps across multiple seeds, placing non-trivial compute requirements on would-be replicators.
Conditional generative models are increasingly used for scientific prediction beyond the direct training support. Yet scientific questions often concern whether one intervention changes the effect of another. Such interactions can be of primary interest while explaining little of the total variation, allowing models to appear accurate overall without recovering the relevant scientific effect. It remains unclear how these models learn interactions and when these successfully extrapolate to unseen conditions. We therefore study interaction recovery in flow-matching models using a physiological time-series simulator, drug dose-response data, combinatorial cell perturbations, and molecular conformer generations, where we separate recovery from hallucinated interaction structure. In the controlled setting, individual effects are learned much earlier than interactions and models hallucinate interactions where none exist. Across settings, we find qualitatively different generalization problems, with successful prediction depending on whether condition structure continues from observed to held-out targets. When it does, interactions are recoverable. If not, models can generate plausible responses while missing the interaction. We furthermore find that hallucinations increase as targets become less identifiable, and that successful interaction recovery in output space can fail under nonlinear readouts. Finally, we show that aggregate fidelity metrics can conceal whether models learn and extrapolate interactions.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.