Your paper timeline
Scroll AI takes the way you would scroll a great paper aggregator: quick signal first, deeper critique when something earns your attention, and challenges when a claim feels off.
1579 papers
Trending mixes fresh papers with community signal.
0
stat.MLcs.AIcs.LG Yujie Zhu, Charles A. Hepburn, Matthew Thorpe et al. · Oct 1, 2026

This survey systematises the sprawling literature at the intersection of optimal transport and reinforcement learning through a four-question taxonomy: what role OT plays, which distributions are compared, which formulation is used, and how temporal structure is handled. By foregrounding the geometry-aware nature of transport-based discrepancies, the authors explain why OT can outperform $f$-divergences when learner and expert distributions overlap weakly—a common scenario in imitation learning, offline RL and robust control. The result is a broad methodological map spanning occupancy matching, trust regions, world-model learning and multi-agent settings.

Reinforcement learning (RL) algorithms frequently compare probability distributions, such as state visitation distributions induced by policies and experts, action distributions from learned policies and offline datasets, or transition distributions from learned models and environments. However, commonly used divergences may become ineffective when these distributions overlap weakly, which is frequently encountered in imitation learning, offline RL, and deployment under distribution shift. Optimal transport (OT) offers an alternative by measuring the cost of \emph{moving} probability mass from one distribution to another under a ground cost that encodes task geometry. This survey covers how OT is used inside RL objectives and algorithms. For each method, we identify: the role OT plays, the distributions compared, the OT formulation used, and the treatment of temporal structure. Beyond categorising existing methods, we discuss the motivations behind different OT choices, practical considerations such as cost design and computational challenges, and highlight open problems including scalable trajectory-level transport, principled handling of mass mismatch, and theoretical analysis for OT-regularised RL.
0
cs.AIcs.CLcs.LG Sidi Chang, Peiying Zhu · Oct 1, 2026

The paper argues that attributing model behavior to synthetic training data requires first recording how each item was generated, because a waveform-label pair alone erases upstream history. The authors introduce a structured provenance substrate that binds scenario seeds, generated content, audio waveforms, fact checklists, quality signals, and review lineage into an immutable manifest. They instantiate this in a five-pass audit of 113 synthetic Japanese care-handoff speech assets totaling 1.552 hours, finding that while versioned, seed-disjoint manifests support some downstream attribution research, exact upstream tracing is blocked by floating generator aliases and missing per-item TTS stamps. The work reframes data attribution as a two-stage problem: establish a typed, versioned generation path $z \longrightarrow x \longrightarrow m \longrightarrow r \longrightarrow b$ before attempting causal or contributive claims.

Attributing model behavior to synthetic training data requires knowing what produced each training item before estimating what that item caused. A waveform-label pair does not preserve this knowledge. We propose a generation-provenance substrate in which a synthetic research object binds source specification, generated content, waveform, target, fact requirements, quality signals, review lineage, and immutable manifest identity. Producer and selection mechanism determine evidentiary meaning; storage location and variable name do not. We audit this substrate in a private Japanese care-handoff pipeline. A 113-asset review population contains 1.552 hours of synthetic speech across six scenario families; all items have linked audio, transcripts, candidate notes, and fact checklists, but human evidence is selective and source-specific. Two faithful-only manifests are scenario-seed-disjoint and immutably versioned, while exact upstream attribution remains blocked by floating generator aliases, missing per-clip TTS and code stamps, and an unversioned checking prompt. We argue that generation provenance is necessary but not sufficient for behavior attribution: it defines the candidate causal graph and audit units, whereas contributive attribution still requires frozen training runs and intervention or influence evidence. The paper contributes a compact provenance contract, an audit protocol, and a bounded case study for synthetic-data attribution; controlled research access may be offered, but we do not claim causal training-data attribution, clinical validity, or unrestricted public release.
0
cs.LG Junhyuk Huh, Seoungbin Bae, Dabeen Lee · Oct 1, 2026

This paper studies causal logistic bandits with counterfactual fairness constraints, where the learner observes only factual rewards and must infer counterfactual effects through shared logistic parameters. The core contribution is establishing minimax optimal regret-violation tradeoffs under a weak pooled coverage condition, showing that a target-specific information scale $V_\star$ governs the estimation difficulty and yielding matching $\widetilde{\Theta}(T^{2/3})$ upper and lower bounds.

We study causal logistic bandits with counterfactual fairness constraints. The causal structure is given through known factual and counterfactual feature maps that share an unknown logistic reward parameter, but the learner observes only factual rewards. Consequently, the directions determining counterfactual feasibility need not be identifiable from the available feedback. The closest prior analyses either omit a coverage condition or impose a comparatively strong one, and do not establish matching lower bounds. We first show that some coverage condition is necessary: without a coverage-type restriction, factually indistinguishable environments with different optimal fair actions force $\Omega(T)$ expected joint loss. Under a weaker full-rank condition on the factual covariance pooled across actions, we identify a target-specific information scale $V_\star$ that measures the difficulty of estimating rewards and counterfactual effects from factual feedback. We construct worst-case families satisfying this condition on which every policy incurs expected joint loss $\Omega\left(\left[V_\star\min\{\log K,d\}\right]^{1/3}T^{2/3}\right)$. We also give an explore--then--exploit procedure tuned using $V_\star$ and an adaptive algorithm that does not require its value. Both algorithms achieve $\max\{R_T,V_T\}=\widetilde{O}\left(\left[V_\star\min\{\log K,d\}\right]^{1/3}T^{2/3}+\kappa d/\sigma_0^2\right)$, where $R_T$ is regret relative to the best fair action and $V_T$ denotes the cumulative stage-wise positive violations. Thus the upper and lower bounds match in their leading dependence on $T$, $V_\star$, and $\min\{\log K,d\}$, up to logarithmic factors.
0
cs.CVcs.LG Ziqi Jiang, Zhenqi He, Long Chen · Oct 1, 2026

Flow Matching (FM) suffers from trajectory crossings that create high-frequency velocity targets and amplify numerical integration errors during few-step sampling. This paper proposes CoFlow, a framework that injects a repulsive contrastive drift into the forward SDE to push training trajectories away from negative samples, thereby smoothing the learned velocity field. The authors derive a simulation-free stochastic interpolant, prove that the transformation preserves the endpoint distributions when boundary terms vanish, and show consistent improvements on ImageNet 256x256 at 20 sampling steps without extra training cost.

Trajectory crossing remains a critical bottleneck in Flow Matching (FM), and previous works typically view these crossings from a theoretical optimization perspective causing velocity averaging. They attempt to address it indirectly by post-hoc distillation or endpoint coupling, without explicitly regulating the intermediate trajectories. In this paper, we introduce a new network learning perspective: crossing points inherently induce large local Lipschitz constants in the target velocity field, leading to two drawbacks. First, high Lipschitz constants correspond to high-frequency signals in the velocity field that neural networks struggle to fit due to spectral bias. Second, they also imply drastic velocity variations, leading to severe numerical integration errors in few-step inference. To alleviate this, we propose CoFlow, a framework that introduces the contrastive learning paradigm into FM to explicitly repel trajectories during training, thereby lowering the local Lipschitz constants of the velocity field. Specifically, we formulate CoFlow from a Stochastic Differential Equation (SDE) perspective by injecting a repulsive drift term. This drift actively guides the forward process of positive samples away from negative trajectories, effectively reducing the local Lipschitz constant. Furthermore, we derive an equivalent stochastic interpolant formulation from this SDE, providing a simple and tractable design space to control the influence of negative samples. Extensive experiments on ImageNet 256x256 demonstrate that CoFlow significantly reduces FID compared to standard FM in few-step inference (e.g., 20 steps), with no added training overhead. The code can be accessed at: https://github.com/HKUST-LongGroup/CoFlow
0
cs.LG Alessandro Montenegro, Riccardo Venturelli, Marco Mussi et al. · Oct 1, 2026

PPO is the workhorse of on-policy deep RL, yet it discards collected data after just a few updates, leaving substantial sample efficiency on the table. This paper asks whether reusing recent off-policy samples within a multiple importance weighting framework can improve PPO without changing its core mechanics. The authors propose two variants—uniform ($\omega$PPO-U) and balance-heuristic-corrected ($\omega$PPO-BH)—derive policy improvement bounds that account for stale advantages, and empirically show that moderate windowed reuse boosts final performance while excessive reuse can backfire.

Among on-policy deep reinforcement learning methods, Proximal Policy Optimization (PPO) has become the de facto standard, due to its consistently strong empirical performance across diverse application domains. However, on-policy methods are inherently sample inefficient: fresh data collected under the current policy is used for just a few updates before being discarded. Off-policy methods avoid this inefficiency via experience replay, achieving notable sample efficiency gains, but at the cost of training instabilities or extensive tuning. This motivated the rise of hybrid strategies that augment PPO with off-policy data reuse. Existing sample-reuse variants of PPO demonstrated improved sample efficiency over vanilla PPO, yet a systematic study of when reuse helps, in which scenarios, and to what extent remains missing. In this work, we study the effectiveness of sample reuse in PPO by instantiating two variants within a multiple importance weighting framework. Both retain the core PPO mechanics, reusing only samples from a window of recent iterations, thereby isolating the effect of data reuse from other factors. The variants, termed wPPO-U and wPPO-BH, employ vanilla importance weights or balance-heuristic-corrected ones, respectively. For both, we derive policy improvement lower bounds providing theoretical grounding for their respective losses. We use them to empirically study when and how data reuse improves sample efficiency or final performance of PPO across continuous control tasks.
0
cs.LG Parthiv Chatterjee, Kashish Kanjaria, Vashisth Purani et al. · Oct 1, 2026

User interactions across movies, news, and dialogue all supply evidence that can update a user model, even though their native vocabularies differ. This paper proposes an action-on-item schema that maps heterogeneous behaviors into a shared event interface, enabling one history encoder — PerTIDE — to learn reusable state updates for both predictive ranking and personalized headline generation. The authors support the idea with invariance and stability theory, plus controlled transfer experiments showing that a frozen source-trained core retains a $15.23$ MRR advantage over a random core, suggesting that compatible histories can share a single update mechanism.

A user's movie, news, and dialogue histories differ in their native actions and outputs, yet each interaction supplies evidence that can update user memory. We study whether these histories can train one reusable update mechanism. An action-on-item schema pairs a mapped interaction role with a content embedding, allowing shared update parameters to operate on separate user states. We establish invariance to native relabeling, bounded state changes under item-embedding perturbations, and a pooled-training bound under explicit compatibility conditions. The Multi-Timescale State Hypothesis (MTSH) specifies how this evidence enters, persists, and is consumed; PerTIDE implements it with action gating, three state-space traces, fusion, and command-conditioned readout. On PENS, the same history encoder supports both next-news prediction and personalized headline generation. In a controlled PENS-to-MovieLens experiment, a frozen source-trained core exceeds an identically structured random core by 15.23 MRR points after fitting the same target consumer. On MIND, PerTIDE retains a 4.12-point MRR advantage over a same-input three-branch state-space control. Action, readout, and trace interventions identify complementary contributions to these gains. Together, the theory and experiments support learning history updates across compatible sources and reusing them through predictive and generative consumers.
0
cs.LGcs.GTcs.MA S. Rasoul Etesami · Oct 1, 2026

This work tackles decentralized learning in multi-player stochastic games where each agent's state transitions depend only on its own state and action, but rewards are arbitrarily coupled. The core idea is to run independent online mirror descent in the dual space of local occupancy measures, using asynchronous dyadic confidence sets to handle unknown transition kernels without synchronized episodes. This matters because the guarantees scale with local state-action dimensions and mixing times rather than the joint product space, offering the first fully online, uncoordinated framework for this setting.

We consider stochastic games with independent controlled chains and unknown transition kernels, where players observe only their local states and realized payoffs. We develop a fully online, decentralized, and uncoordinated mirror-descent algorithm that operates in the dual space of occupancy measures for approximating stationary Nash equilibrium (NE) policies. The algorithm uses a single transition/reward sample at every primitive time step, relies only on local information, and requires neither coverage of the joint state space nor synchronized episodes. Under uniform-ergodicity and finite-coverage assumptions, we show that, with high probability, the time-averaged fixed-comparator regret decays at the canonical $O(T^{-1/2})$ rate, up to logarithmic factors and polynomial dependence on the game parameters. In particular, the complexity depends on the cover times of the individual local state spaces rather than the product state space, avoiding exponential dependence on the number of players and the sizes of the joint state and action spaces. The resulting finite-time regret bound further yields an approximate coarse-correlated-equilibrium guarantee, which is natural for arbitrary reward functions since computing a stationary $\epsilon$-NE is PPAD-hard in this setting. Under an additional global variational-stability condition, we show that the same fully online algorithm converges asymptotically in the last iterate to a stationary $\epsilon$-NE. Our results provide a fully online and scalable learning framework for stochastic games with unknown independent chains. The algorithm can also be viewed as a primal-dual framework for Markov games that exploits the independence and local structure of the players' controlled transition chains.
0
cs.CLcs.AIcs.LG Darpan Aswal, C\'eline Hudelot · Oct 1, 2026

Diffusion language models generate text through iterative denoising rather than left-to-right prediction, making attribution a spatiotemporal problem: support for any output token is distributed across both network depth and timesteps. This paper introduces Diffusion Layer Integrated Gradients (DLIG), which extends Integrated Gradients to score a DLM's commitment to its completion at any chosen layer and denoising step, decomposing it into per-token contributions. It matters because existing token-attribution tools ignore the denoising trajectory entirely, while interventional alternatives are computationally expensive.

This work presents Diffusion Layer Integrated Gradients (DLIG), a token attribution method for diffusion language models (DLMs) that extends Integrated Gradients (IG~\cite{sundararajan2017axiomatic}) to arbitrary layers and denoising steps. DLIG attributes a DLM's progressive commitment to a self-generated or fixed completion for an input prompt. We establish direct correspondences between DLIG and the IG axioms of completeness, implementation invariance, linearity, and symmetry preservation. As a lightweight complement to interventional analysis, DLIG provides an inexpensive first check of mechanistic hypotheses across the denoising trajectory. We demonstrate this on word-sense disambiguation, multi-hop graph reasoning, and sentence infilling, revealing how DLMs draw on inputs across positions, layers, and denoising steps.
0
cs.LG Michael Hauri, Peter Buttaroni, Fabian A. Mikulasch et al. · Oct 1, 2026

The paper tackles the problem of learning task-agnostic world models for planning in latent space. It introduces CTWM, a reconstruction-free method that combines residual prediction with a log-determinant regularizer to recover the scaled Laplacian representation, so that latent L2 distances correspond to environment commute-time distances. This matters because it enables zero-shot goal-reaching without reward, and empirically CTWM matches or beats the LeWM baseline while using half the parameters.

World models allow agents to plan in latent space by choosing a sequence of actions that most reduces the distance to a given goal state. Thus, planning can benefit from latent representations whose distances mirror commute-times in the environment. The spectral embedding space of the graph Laplacian provides such a representation, if it obeys a specific eigenvalue-dependent scaling. Unfortunately, instantiating the graph Laplacian is intractable in large, continuous environments. Self-supervised learning offers a natural route to such commute-time-preserving embeddings at scale. However, here we show that existing methods, which commonly encourage isotropic representations to prevent representational collapse, tend to degrade the "correct" eigenvalue-dependent scaling, leading to an inaccurate representation of commute times. To address this problem, we introduce Commute-Time-Preserving World Models (CTWMs), combining a latent displacement predictor and a log-determinant regularizer that prevents collapse, which provably recover the correctly scaled Laplacian representation under reversible deterministic dynamics and at the predictor's fixed point. In numerical simulations, CTWM matches or outperforms LeWM, a task-agnostic baseline, on several complex, continuous goal-reaching benchmarks, while using half the parameters.
0
cs.LGcs.DS Anthony Dugois, Vincent Fagnon, Giorgio Lucarelli · Oct 1, 2026

We study non-clairvoyant single-machine scheduling to minimize the sum of completion times, a problem for which no constant-competitive algorithm is possible in the pure setting. The authors introduce a robustness framework that leverages a classifier's confusion matrix to partition jobs into predicted classes, characterizing uncertainty as internal permutations rather than numerical scenarios. This structural information yields an oblivious non-adaptive Min-Cost algorithm that is simultaneously optimal for multiple criteria, alongside new adaptive and randomized extensions.

We study the classical single-machine scheduling problem of minimizing the sum of completion times of jobs in a non-clairvoyant setting, where the processing time of each job remains unknown until its completion. This is a hard problem for which no constant competitive algorithm is possible. Inspired by robust optimization and learning-augmented algorithms, we introduce a novel robustness framework that leverages structural information provided by a classification model to overcome this limitation. Specifically, we assume that jobs are partitioned into classes and we have access to the confusion matrix of the classifier, whose entry $(k,\ell)$ indicates the number of jobs predicted to belong to class~$k$ but that actually belong to class~$\ell$. In this manner, we are able to characterize uncertainty as a set of permutations within each predicted class, rather than as a collection of discrete numerical scenarios, avoiding the computational difficulty of classical robust metrics, such as Min-Max and Min-Max Regret. In addition to these worst-case metrics, we also consider the expected objective over all scenarios. We first propose an optimal non-adaptive strategy that is oblivious with respect to all three robust criteria. We then investigate adaptive and randomized algorithms, showing that they can outperform the optimal non-adaptive strategy when the matrix exhibits particular structural properties.
0
cs.LGnlin.CD Hiroto Tamura, Gouhei Tanaka · Oct 1, 2026

This paper addresses the challenge of learning minimal, interpretable piecewise-linear representations of unknown nonlinear dynamical systems from time-series data. The authors propose training a higher-capacity almost-linear RNN first, then hierarchically reducing it via a fixed-point-guided procedure that linearizes redundant ReLU units while preserving distinct fixed-point symbols, and finally retraining the compact model from the parent initialization. The central insight is that redundant nonlinear capacity can act as a structural and optimization scaffold for discovering reliable minimal models, which matters because directly training at the theoretical minimal capacity often fails catastrophically.

Understanding a nonlinear dynamical system from time series requires not only reproducing its trajectories, but also identifying a simple representation that preserves its essential dynamical structure. Almost-linear recurrent neural networks (AL-RNNs) are piecewise-linear RNNs in which only a subset of units use ReLU nonlinearities, so that nonlinear capacity is explicitly controlled by the number of ReLU units. Their activation patterns define linear regions, represented as symbols, whose observed transitions form a symbolic transition graph. However, directly training AL-RNNs with few ReLU units to realize minimal dynamical representations can be unreliable. We ask whether an AL-RNN with more ReLU units can instead be trained first and systematically reduced to a minimal dynamical representation. We introduce a fixed-point-guided hierarchical reduction procedure that progressively linearizes selected ReLU units, merging neighboring linear regions and graph nodes while preserving distinct symbols containing fixed points (FPs). The resulting reduction tree defines a hierarchy of progressively simpler candidates. Each reduced candidate is initialized from the parent parameters and retrained under guidance from the parent dynamics. We also prove that reproducing $Q$ distinct fixed points requires at least $Q$ FP-containing symbols, providing a certificate of symbol-level minimality when this bound is attained. On the 3-scroll Chua system, direct training with the theoretical minimum of three ReLU units achieves high-fidelity minimal realizations in only 20% of seeds, whereas our learn-reduce-retrain strategy increases the seed-macro success rate to approximately 71% at the same final nonlinear capacity. These results show that redundant nonlinear capacity can serve as a scaffold for discovering and realizing minimal dynamical representations.
0
cs.LGstat.ML Kotaro Kamiya, Joel Nicholls · Oct 1, 2026

Kamiya and Nicholls introduce the Clifford Sheaf Neural Network (CSNN), a geometric-graph architecture that places Clifford-algebra stalks on cellular sheaves and transports multivector features with learned restriction maps. The core advance is replacing versor conjugation with a normalized reversion sandwich $s_c(x)=c x \tilde{c}/q(c)$, yielding a sheaf Laplacian that stays positive semidefinite by construction while mixing feature grades. Despite a careful theoretical map of the design space, the paper contains no empirical experiments, leaving its practical value for graph-level regression entirely unproven.

We introduce the Clifford Sheaf Neural Network (CSNN), an equivariant sheaf neural network for geometric graphs that places a Clifford algebra on each stalk of a cellular sheaf and transports multivector features along edges. The canonical choice of restriction map for sheaves with algebra-valued stalks is algebra homomorphism. Adding the constraint of equivariance, the naive choice becomes versor conjugation. However, versor conjugation is expressively weak, so we drop algebra homomorphism and arrive at the K-term sandwich. The resulting sheaf Laplacian is positive semidefinite by construction, needs no versor constraint, and still mixes grades. Our main contribution characterizes the resulting family of restriction maps along three axes: which grades a map couples, how much of the endomorphism space it reaches, and how well it is conditioned. The K-term sandwich spans half of the endomorphism space, and in Cl(3, 0, 0) it corresponds to the maps that commute with the central pseudoscalar. The number of terms controls expressivity. CSNN is the reversion member, a first-order model by construction and the grade-mixing corner of this family, developed as a sheaf construction for graph-level equivariant regression.
0
cs.SIcs.LG Alexandra Dache, Manon Rustin, Arnaud Vandaele et al. · Oct 1, 2026

This paper tackles community detection in multilayer networks where connectivity patterns and node degrees vary across layers. The proposed method, mFROST, jointly factorizes each layer via orthogonal symmetric nonnegative matrix trifactorization $A_l \approx D_l V \theta_l (D_l V)^\top$, enforcing shared community assignments $V$ while allowing layer-specific degree corrections $D_l$ and community interactions $\theta_l$. This flexibility matters because real-world multiplex networks often exhibit heterogeneous node sociability and diverse block structures that simpler shared-factor models miss.

Multilayer networks allow the modeling of interactions between the same entities across different contexts, such as temporal observations, varying settings, or interactions of different types. The goal of community detection in multilayer networks is to identify groups of nodes exhibiting similar connectivity patterns, which may vary across layers. We propose a method based on a joint nonnegative symmetric matrix trifactorization for community detection in multilayer networks, where each graph is approximated by a nonnegative symmetric matrix trifactorization. Our approach enforces constraints on the factor matrices so that communities are disjoint and shared across layers, while allowing each layer to have its own connectivity patterns and node degrees. This flexibility enables the model to capture both local and global structural variations across layers. We also develop an algorithm to efficiently solve this problem. We evaluate multilayer community detection methods using the multilayer degree-corrected stochastic block model (MDCBM), a flexible framework for generating realistic multilayer graphs with heterogeneous degrees and varying connectivity patterns. Experiments show that our method reliably detects communities across diverse regimes, whereas existing state-of-the-art approaches are often limited by restrictive structural assumptions.
0
cs.LGcs.AImath.ST Hanna Malet, Gabriel Turinici · Oct 1, 2026

This paper studies model collapse in diffusion models under a fixed-budget data regime, where synthetic data accumulates but each generation trains on a constant-size subsample from the growing pool. Unlike prior work on total replacement (rapid collapse) or unbounded accumulation (collapse avoided), the authors argue that fixed budgets yield partial, feature-selective collapse: "fragile" features with large eigenvalues in a linearized parameter dynamics model decay quickly, while "robust" features with small eigenvalues persist for exponentially many generations. They support this with theory (stochastic recursion giving per-feature halving times) and experiments on 2D spirals, MNIST, Fashion-MNIST, and CIFAR-10.

Model collapse arises when generative models are trained on synthetic data produced by earlier models. The phenomenon has attracted considerable attention because of its societal and technical implications. However, previous studies have reached seemingly contradictory conclusions: replacing real data with synthetic data causes collapse (Shumailov et al.), yet accumulating real data alongside synthetic data can prevent it. For diffusion models, we study an intermediate regime typical of finite-budget pipelines: all past datasets and the real data are kept, but each new model is trained on a fixed-size sample from this growing pool, so the real fraction vanishes without any data being removed. Experiments on a 2D spiral dataset as well as the image benchmarks (MNIST, Fashion-MNIST, and CIFAR-10) show that replacement protocol degrades dataset rapidly as in the literature, whereas the fixed budget degrades only partially, sparing some features. A linear-response model of the multi-generational parameter dynamics, analyzed by stochastic recursion, confirms that the two protocols differ: some features will be fragile and lost within a few generations for both protocols, while some will be robust and preserved over practically unbounded horizons under the fixed budget protocol.
0
cs.LG Pascal Janetzky, Yuxin Wang, Michael Klar et al. · Oct 1, 2026

Neural architecture search is bottlenecked by the cost of obtaining ground-truth performance labels. ppNAS addresses this by fusing a small set of labeled architectures with a large set of zero-cost proxies via prediction-powered inference. The core insight is to construct pairwise ranking supervision from both sources and debias systematic discrepancies using a rectified PPI objective, yielding state-of-the-art results under tight evaluation budgets.

Evaluating candidate architectures in neural architecture search (NAS) faces an inherent trade-off: on the one hand, reliable performance labels are limited because training and evaluating architectures is costly; on the other hand, zero-cost proxies (ZCPs) are cheap to compute at large scale but can be noisy. Yet, how to effectively combine these two sources of supervision remains unclear. In this paper, we propose PPNAS, a novel prediction-powered inference (PPI) approach for NAS. PPNAS fuses (1) a small set of architectures with observed performance labels and (2) a large set of architectures with ZCP information. To combine these two sources of supervision, PPNAS exploits the ordinal information provided by ZCPs to construct additional pairwise ranking supervision, while PPI debiases systematic discrepancies between ZCP-based and true performance rankings. We evaluate PPNAS in end-to-end predictor-based NAS, where it achieves state-of-the-art under limited evaluation budgets. To the best of our knowledge, PPNAS is the first prediction-powered approach for label-efficient NAS.
0
cs.LGcs.SYeess.SY Simon Heilig, Jens P\"uttschneider, Mohammad Itani et al. · Oct 1, 2026

Standard stable port-Hamiltonian neural networks (s-PHNNs) enforce a single global minimum in the learned Hamiltonian, limiting them to systems with exactly one attractor. The authors overcome this by parametrizing the Hamiltonian as a product of Bregman divergences generated by a single fully input-convex neural network (FICNN), yielding a model---the ms-PHNN---that can represent multiple asymptotically stable equilibria while retaining local stability certificates by construction. This matters because many physical systems, from buckled beams to magnetic pendulums, exhibit multi-stable energy landscapes that existing guaranteed-stable neural ODEs cannot capture.

Stable port-Hamiltonian neural networks certify asymptotic stability by construction. Yet, their Hamiltonian is a global Lyapunov function with a single global minimum, so they can represent only dynamic systems with {one} attractor. We demonstrate that this excludes even simple systems with energy landscapes forming a double well, and we overcome the restriction by parametrising the Hamiltonian as a {product} of Bregman divergences generated by one input-convex network. We prove that the resulting model is locally Lyapunov stable, that the coexistence of stable equilibria forces additional non-asymptotically-stable equilibria to exist, that all equilibria lie in a bounded region, and under a hyperbolicity assumption that almost-everywhere stability holds. On three systems our approach is able to recover the energy surface characteristics and improve the convergence speed by 1.8$\times$-8.5$\times$.
0
cs.LG Jingyao Zhang, Yuxuan Li, Lu Han et al. · Oct 1, 2026

Standard variational information bottleneck (VIB) regularizes representations by penalizing a single global mutual information $I(Z;X)$, which inherently treats all encoded information as equally valuable. This paper identifies a structural limitation: under a label-induced partition $C=\pi(Y)$, the chain rule gives $I(Z;X)=I(Z;C)+I(Z;X\mid C)$, so scalar compression conflates condition-level structure with residual within-condition variation. The authors propose a dual-bottleneck objective combining a standard KL term for global capacity and a conditional KL term that targets within-condition information, instantiated with a simplex-structured conditional prior. The result is a principled regularizer that integrates into existing pipelines and shows the clearest empirical benefits in low-data classification and medical-image segmentation, exactly where nuisance-like residual variation can dominate.

Standard information bottleneck (IB) regularization constrains representations via a single scalar I(Z;X), implicitlytreating all information as homogeneous. However, a single global compression control couples label-relevant structurewith residual within-condition variation, rather than regulating their allocation independently, allowing nuisanceinformation to persist in learned representations. For example, in medical imaging applications, residual variation oftenstems from acquisition conditions, background factors, or subject-specific appearance. This issue becomes particularlypronounced in data-limited settings, where models tend to overfit such variation, hindering generalization. While existingregularization methods can stabilize training, control capacity, or shape representation geometry, they do not explicitlyseparate nuisance-like variation from task-supporting structure. To address this limitation, we revisit IB from a structuredperspective based on a label-induced partition, where condition-level structure and within-condition information playdistinct roles. This leads to a dual-bottleneck formulation: a standard KL term controls global information capacity, while aconditional KL term targets within-condition information. We show that the conditional KL admits an exact decompositioninto a within-condition information term and a prior-mismatch term, explaining its alignment with the design objective.With a simplex-structured conditional prior, the method provides controllable latent geometry and integrates seamlesslyinto existing pipelines. Experiments on classification and segmentation show the clearest gains in low-data classificationand consistent improvements across dense prediction benchmarks.
0
cs.LGcond-mat.dis-nn Qiuliang Liu, Liming Wu, Qi Li et al. · Oct 1, 2026

Disorder is ubiquitous in functional crystals—nearly half of ICSD entries show some form of disorder—yet most generative models assume perfectly ordered lattices with deterministic site occupations. EP-Flow attacks formula-conditioned disordered crystal structure prediction without site-level annotations by representing occupancies as an Occupancy Distribution Matrix (ODM) that lives on a formula-dependent transportation polytope. It proposes a flow-matching model that canonicalizes heterogeneous polytopes into a shared double-centered space via log-domain double centering, learns a marginal-preserving velocity field, and recovers valid occupancies with a differentiable Sinkhorn inverse map, jointly generating fractional coordinates and lattice parameters. If the absolute performance can be scaled, this formalism could become a standard way to handle substitutional, vacancy, and interstitial disorder in generative materials design.

Generative models have made rapid progress in ordered crystal structure prediction, yet many functional materials are intrinsically disordered, with substitutional mixing, vacancies, or interstitial species controlling their properties. Existing crystal generators either assume deterministic site occupations or require site-level disorder annotations, which are often unavailable when the chemical formula is the primary input. We formulate disordered crystal structure prediction through an Occupancy Distribution Matrix (ODM), a continuous site-by-species representation that unifies ordered crystals, solid solutions, vacancy disorder, and interstitial occupancy. A valid ODM must satisfy coupled site-wise occupancy, mass-conservation, and non-negativity constraints, placing each sample on a formula-dependent transportation polytope. We propose Entropic Polytope Flow (EP-Flow), a marginal-constrained flow matching framework that canonicalizes heterogeneous polytopes into a shared double-centered space, learns a marginal-preserving flow, and recovers feasible occupancies through a Sinkhorn inverse map. By jointly generating occupancies, fractional coordinates, and lattice parameters, EP-Flow achieves state-of-the-art performance on formula-conditioned disordered CSP benchmarks derived from COD and MPDS, substantially outperforming adapted ordered-crystal generators. Analyses further show that EP-Flow recovers sparse and chemically meaningful local disorder patterns rather than merely matching global composition statistics.
0
cs.LG Santiago Florido Gomez, St\'ephane Rivaud · Oct 1, 2026

Neural scaling laws favor ever-larger models, but deciding when a network actually needs more capacity remains ad hoc. CAGE-NAS proposes a theoretically grounded rule: keep the architecture fixed only while the current parameter space can certify a valid functional gradient descent step; otherwise, expand the structure in a function-preserving way. By coupling growth decisions to the geometry of the empirical tangent space, the method unifies training and architecture adaptation under a single convergence certificate. On a controlled synthetic regression task, it discovers compact three-layer MLPs that sit in the extreme top tail of an exhaustive post-hoc search, though the evaluation remains limited to a toy benchmark.

The progressive growth of neural networks requires deciding when the current representation remains sufficient for optimization and when it should be expanded. CAGE-NAS formulates this decision in function space through an admissibility criterion on approximations of the functional gradient. As long as a representation enables a certified Functional Gradient Descent step, the architecture remains fixed; when the criterion fails, a function-preserving expansion is applied and the resulting representation is evaluated again. As the main instance, we study the family induced by the tangent space, using a regularized projection of the functional gradient. In a controlled setting with exact certification, CAGE-NAS produces architectures positioned above the 99.8th performance percentile by held-out RMSE among all admissible alternatives within the same parameter budget, without enumerating them during the growth trajectory.
0
cs.ROcs.LG Giulio Schiavi, Andrei Cramariuc, Michael Pantic et al. · Oct 1, 2026

Standard grasp synthesis systems freeze their parameters after offline training and struggle with unfamiliar objects. This paper presents a continual-learning framework that replaces the fixed parametric scorer with a memory-based non-parametric model in a learned embedding space, augmented by recalling and transferring user demonstrations to new scenes via geometric registration. Instead of gradient-based finetuning, the robot adapts by adding grasp outcomes and demonstrations to structured memories, which enables fast, lightweight updates during real-world deployment with limited forgetting.

Most current grasp synthesis systems are trained offline and remain fixed during deployment. While this works well when deployment conditions resemble the training data, performance can degrade when robots encounter conditions they have not seen before, such as unfamiliar objects. In this work, we present a continual-learning framework for single-view 6-DoF grasp synthesis for a parallel-jaw gripper in cluttered scenes. Rather than finetuning a large parametric model, our method adapts through memory in a learned embedding space: grasp outcomes update future grasp scores, while optional user demonstrations are recalled and transferred to new scenes as additional candidate grasps. We evaluate our method in simulation and in extensive real-world experiments comprising over 1500 grasp trials. We show that our method matches the performance of existing 6-DoF grasping baselines even before adaptation, improves online on unseen objects from categories absent or underrepresented during training, and supports long-horizon continual learning with limited forgetting. In real-world experiments, our method reaches over 90\% success rates on several challenging object categories after only 50 online grasp attempts. Videos and code at https://giuschio.github.io/cl_grasping/.