MorphingNet: Symbolic Regression Where the Network Is the Formula
MorphingNet attacks the scalability problem in symbolic regression by turning formula search into continuous network training. Its units are pole-free rational functions and complex exponential-logarithm shapes whose parameters morph smoothly between elementary forms, so the network itself is a closed-form formula at every step. A post-training structure-recovery step rewrites the learned expression into compact elementary notation. The paper reports state-of-the-art SRBench accuracy, exact recovery of most ground-truth formulas, and GPU speedups of two orders of magnitude over genetic programming on large data.
MorphingNet makes a strong case for continuous-structure symbolic regression. Its trainable rational and complex exponential-logarithm units elegantly remove the need for a fixed operator library during optimization, and the SRBench gains are real. Yet the structure-recovery stage is brittle and order-sensitive, the C-unit domain is only sampled rather than certified, and the comparison with KAN understates recent hybrid extensions. The paper is a solid step toward scalable SR, but users should treat the recovered formulas as candidates rather than definitive discoveries.
The mathematical design of the morphing units is genuinely novel. The rational unit $\varphi(z) = P(z)/(1+Q(z)^2)$ avoids poles while preserving Pade-like approximation power, and the complex unit $J(z) = (e^{q L_\beta(z)} - 1)/q$ smoothly interpolates between exponentials, sinusoids, and power laws without discrete switches. The proximal LM training with ADMM is an excellent fit for the small-parameter regime, returning exact zeros so that units can be pruned cleanly. On SRBench the rational network achieves $R^2 > 0.999$ on 97.0% of noise-free problems, a clear margin over the 85-88% of the best published methods.
The most serious concern is the heuristic fragility of structure recovery. Because the trained network is long and redundant, the greedy rewrite is the only path to readability, and the authors concede it is 'sensitive to the last digits of its inputs.' The C unit also carries an uncertified domain risk: the logarithmic branch requires $1+\beta z > 0$, but this is only checked on 256 sampled probes, not verified over the whole input box; about 3% of confluent formulas fail finiteness on the test set.
At the highest noise level the rational margin over Operon shrinks to two points (70% vs 68%), within the split-to-split variance, so the claim of universal noise-level dominance is technically true but practically thin.
The SRBench evaluation follows the standard protocol and the speed comparison with Operon is fair in absolute terms, though the orders-of-magnitude GPU speedup is expected given the fundamental difference between dense least-squares optimization and discrete evolutionary search. The comparison with KAN is less balanced. The authors describe KAN as a 'spline soup' whose exported formula 'loses accuracy' because symbolic matching is approximate. While the original KAN does rely on post-hoc spline-to-symbol fitting, KAN 2.0 introduced MultKAN with explicit multiplication nodes and kanpiler for embedding prior symbolic knowledge; the paper does not benchmark against these newer KAN capabilities and instead compares against baseline KAN 2.0 formulas that use default spline grids.
The training algorithm and hyperparameters are documented in enough detail for reproduction, but several barriers remain. The structure-recovery pipeline is a multi-stage heuristic involving Unit, Merge, Quadratic, Delete, Swap, and Template rewrites with joint LM refits, and the authors note that 'a different execution order can end at a nearby form.' The custom GPU kernel implementing Ozaki's scheme for exact Gram-matrix accumulation is highly specialized. Most importantly, the paper makes no mention of code release or an accompanying repository, which is the largest practical obstacle to independent verification of the SRBench and recovery statistics.
We introduce MorphingNet, a network for symbolic regression (SR) whose units change their function continuously. Instead of searching over expressions built from a fixed operator library, MorphingNet trains the form of each function as a parameter, so SR becomes one continuous optimization problem. Its units are either pole-free rational functions or complex exponential--logarithm functions, which move between exponentials, sinusoids, damped oscillations, power laws and logarithms as their parameters change. The network is a readable formula at every step of training, and no operator library is needed to train it; a structure-recovery step afterwards rewrites the trained formula into short elementary expressions. Training is dense linear algebra, so it is GPU-friendly and scales to large data. MorphingNet achieves state-of-the-art accuracy on SRBench, recovers most generating formulas on synthetic benchmarks, and on large data is up to two orders of magnitude faster than the strongest search method. By turning the search for a formula into the training of a network, MorphingNet offers a new paradigm for SR and for scientific discovery from data.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.