Your paper timeline
Scroll AI takes the way you would scroll a great paper aggregator: quick signal first, deeper critique when something earns your attention, and challenges when a claim feels off.
797 papers in cs.LG
Trending mixes fresh papers with community signal.
0
stat.MLcs.LG BaoZhi Chu, Ben Weinkove · Oct 14, 2026

What is the least width required for a deep neural network to universally approximate continuous functions from $\mathbb{R}^n$ to $\mathbb{R}^m$? This paper attacks the minimal-width problem by importing tools from differential topology, proving that whenever real projective space $\mathbb{RP}^{n-1}$ smoothly immerses into $\mathbb{R}^n$, any network with a continuous monotone activation must have width at least $n+\min\{n,m\}$ to approximate arbitrary maps. This yields the exact ReLU minimal width $n+m$ for the special dimensions $n\in\{2,3,4,7,8\}$ with $m\le n$, and exposes a surprising link between universal approximation and the immersibility of projective spaces.

We investigate the minimal width of deep neural networks that can approximate all continuous functions from $\mathbb{R}^n$ to $\mathbb{R}^m$ in the topology of uniform convergence on compact subsets. We show that whenever the real projective space $\mathbb{RP}^{n-1}$ admits a smooth immersion into $\mathbb{R}^n$, the minimal width is at least $n+\min\{n,m\}$, assuming that the activation function is continuous and monotone. As a consequence, the minimal width of ReLU networks is $n+m$ whenever $n$ is equal to 2, 3, 4, 7 or 8 and $1\le m \le n$, extending results of Hanin-Sellke and Hwang. This suggests an intriguing connection between the approximation theory of neural networks and differential topology.
0
cs.LGstat.ML Dwaipayan Haldar, Indranil Patra, Azhar Yousuf et al. · Oct 14, 2026

Rotation-based post-training quantization for LLMs currently forces a choice between cheap data-agnostic transforms like Hadamard and costly learned rotations that require iterative optimization. SpectrumQuant introduces a closed-form, covariance-aware orthogonal transform derived from a first-principles objective on the Stiefel manifold: by aligning activations with their eigenbasis and applying a Hadamard or butterfly mixing component, it equalizes per-coordinate variances to minimize quantization error. The result is a method that cuts calibration costs by more than $20\times$ relative to learned baselines while preserving competitive accuracy on Llama and Mistral models.

Orthogonal transforms have emerged as an effective tool for low-bit quantization of large language models (LLMs), as they reshape activation distributions without changing the underlying model function. Existing approaches typically rely either on fixed data-agnostic transforms, such as Hadamard rotations, or on calibration-time optimization of learned rotations. We study rotation selection from first principles and show that maximizing covariance leads to spectral alignment and provides a principled criterion for constructing quantization-friendly orthogonal transforms. Based on this insight, we propose SpectrumQuant, a closed-form rotation method that estimates activation covariance from calibration data and uses it to construct a covariance-aware orthogonal transform for post-training quantization. Our formulation characterizes quantization-friendly rotations as compositions of alignment and mixing components, while recovering Hadamard matrices as a special case. Interestingly, a class of butterfly matrices belongs to this category. In the realm of LLM quantization, SpectrumQuant bridges the gap between data-agnostic structured rotations and learned rotations, providing an effective and efficient data-aware alternative for post-training quantization. Experiments on Llama and Mistral models show that SpectrumQuant is competitive with state-of-the-art rotation strategies while requiring less than one-third of the GPU memory and significantly less time than learned rotations approaches like SpinQuant and DartQuant.
0
cs.LG Hanyao Guo, Mo Cui, Zihan Tan et al. · Oct 14, 2026

ChemArgus tackles a real blind spot in chemistry evaluation: endpoint-only scores hide where reasoning chains break. The benchmark enforces fine-grained rubric scoring across three controlled protocols that share a single scoring unit, so gaps between full-problem, sequential-carry, and oracle-scaffolded results isolate integration, propagation, and local-reasoning failures. The design is principled, yet the paper never states the dataset size and evaluates models lacking citations or independently verifiable endpoints, leaving the empirical results impossible to validate.

Chemistry evaluation has moved past multiple-choice probes toward open-ended expert problems, yet the benchmarks that define the current frontier still score answer endpoints alone. Such scoring conflates memorized facts with stepwise derivation, treats visual and textual inputs as interchangeable, and confounds long-horizon planning with stepwise execution. These blind spots prevent any diagnosis of where the reasoning chain breaks in a model. We present ChemArgus, a competition-level chemistry benchmark drawn from chemistry competition material, textbooks, and research papers, whose scoring replaces the answer endpoint with three measurement properties. Fine-grained scoring grades every subquestion point by point with partial credit, so a score lands at the level of the step rather than the outcome. Multimodal capability is priced on every problem containing a real image, paired with a controlled text description, so visual understanding is measured against text on identical tasks. Diagnostic attribution reads paired metrics that share one scoring unit and differ only in context construction, so a gap between two scores isolates a single cause, namely integration, propagation, or local reasoning. GPT-5.6-Sol scores 23.2% on unassisted whole-problem solving against 28.8% and 30.4% under the two sequential metrics, a gap measuring the integration burden. Through rubric partial credit and these cause-isolating score gaps, ChemArgus shows not only whether a model answers chemistry correctly but where its reasoning chain fails, so that every score it reports doubles as a diagnosis, a readout of the chain rather than a verdict, aiming to guide language models step by step on their path toward chemistry expertise, one corrected step at a time, each step priced by the rubric, point by point.
0
cs.DCcs.LG Daijin Hu · Oct 14, 2026

JointServe targets multi-model LLM agent serving on memory-constrained GPUs, where heterogeneous expert weights and ever-growing KV caches physically contend for HBM bytes. It unifies weight shards and KV blocks as competing residency objects under one budget, exploits the agent's own tool-call stream to prefetch destination experts during caller generation gaps, and streams only incremental layer groups when a swap occurs. The work is motivated by the architectural shift from single-model tool use to planner-orchestrated pools of heterogeneous experts—a pattern visible in SWE-bench, $\tau$-bench, and OSWorld traces—and targets the private and edge deployments where full co-residency is arithmetic impossible.

LLM agents are evolving from a single model calling tools to planners that orchestrate multiple expert models whose capabilities live in distinct weights. This shift breaks a hidden assumption of today's serving systems: that the working set of models fits in GPU memory. On the memory-constrained GPUs typical of private and edge deployments, naive whole-model swapping serializes weight transfer with computation and, without host-backed KV restoration, discards session state at a switch. We present JointServe, a serving system that treats weight shards and KV blocks as residency objects competing for one unified memory budget. JointServe exploits three structural properties of agent workloads (directed calls, gapped execution, and dual-state residency) to prepare models before they are called: an intent signal parsed from the caller's own output stream triggers prefetch during generation, partial k* residency reduces switching to incremental layers, and a preemptive transfer scheduler protects live-session KV with a guaranteed bandwidth share. A carve-out control step adjusts the weight/KV allocation as workload demands change. Evaluations show that JointServe reduces session completion time by 21.7% over the strongest whole-model swap baseline and stays usable in a crash regime where every whole-model swap baseline fails to finish (a 46-fold gap). On replayed SWE-bench, tau-bench, and OSWorld traces, it cuts session completion time by up to 49% and lifts SLO attainment from 2.8-24.5% to 76.8-100%.
0
cs.LGcs.SYeess.SY Luka Todorovic, Kaushal Gianchandani, Sai Ravela · Oct 14, 2026

Neural emulation need not require learning a substitute for known physics. This paper investigates PolyNets that compile continuous-time polynomial dynamics and discrete-time RK4 integration into a single tensor graph, benchmarking the approach on the 1D Korteweg–de Vries equation against classical CPU/GPU solvers and three trained neural models. The central finding is that when governing coefficients are known, the resulting network requires no training yet preserves numerical fidelity while achieving competitive GPU throughput.

Neural emulation need not require learning a substitute for known physics. We investigate PolyNets that couple a continuous-time neural representation of polynomial dynamics with a simple recurrent network for discrete-time integration. When the governing coefficients are known, the resulting tensor graph requires no training and preserves the prescribed numerical scheme while enabling efficient hardware execution and parallel simulation. Using the one-dimensional Korteweg-de Vries equation, we compare PolyNet with CPU and GPU finite-difference Runge-Kutta solvers and three trained neural models: a state-map emulator, a physics-informed predictor, and a learned-tendency integrator. Analytical soliton solutions provide a reference for amplitude and phase errors across grid resolutions, 10^6 dispersively scaled time steps, ensembles of up to 10^4 trajectories, and varying initial amplitudes. PolyNet achieves accuracy comparable to the numerical solvers, including at amplitudes outside the learned models' training conditions. It combines this fidelity with high GPU throughput, approximately constant over the tested resolution range, and favorable ensemble scaling.
0
cs.SEcs.LG Rakhshanda Jabeen, Morgan Ericsson, Jonas Nordqvist et al. · Oct 14, 2026

Architectural drift is expensive to detect when the mapping between source files and architectural modules is missing or outdated. This paper frames code-to-architecture mapping as semi-supervised node classification over a directed multi-relational dependency graph, expanding a small labeled seed via iterative self-training. By holding features, training loops, and classifier heads fixed across GCN, GAT, and a graph-free MLP, the authors isolate whether dependency structure genuinely improves mapping accuracy.

Architectural technical debt arises when a system's implementation drifts from its inten ded modular decomposition, while the mapping between source files and architectural modu les is missing, incomplete, or costly to maintain manually. We formulate code-to-archite cture mapping as semi-supervised node classification over a directed multi-relational de pendency graph, with files as nodes, typed dependencies as edges, and lexical features d erived from code identifiers and directory layout. Starting from a small labeled sample, an iterative self-training procedure expands the mapping to the remaining files. We com pare a graph convolutional network (GCN), a graph attention network (GAT), and a graph-f ree multi-layer perceptron (MLP), using identical features and training procedures for t he learned models, against a Naive Bayes reference. Across 100 runs on ten open-source J ava and C/C++ systems, all three neural models outperform the reference on both F1 metri cs. GAT achieves the highest averages of system-level median macro and micro F1, at 83.5 and 92.2, respectively, whereas MLP remains competitive or superior on several systems. On Chrome, the largest system with 18,343 files, GAT and GCN exceed MLP by 22 and 19 ma cro-F1 points, respectively, and have substantially fewer persistently misclassified fil es. Connectivity and homophily are associated with where neural message passing helps, w hile multi-relational aggregation outperforms a single merged edge type on every system for both GCN and GAT. A follow-up study finds that several model rankings depend on the training seed, qualifying smaller advantages without removing the large graph benefit ob served on Chrome.
0
cs.CLcs.LG Junyi Zou, Avrova Donz · Oct 14, 2026

MMLA-RSI proposes recursive self-improvement through two Memory-Mediated Learning Architecture routes: Reasoning-Time Training (RTT) treats the numerical policy-state updater as an object of improvement, while Reasoning-Time Memory Adaptation (RTMA) treats the memory-management rule as an object of improvement. Each admitted successor takes over its parent's role in producing the next generation's rule. The paper's main contribution is a formally bounded construction with shared interfaces and a common algorithm, though the authors explicitly state that beneficial recursive chains remain to be evaluated.

An agent can improve its behavior within an unresolved problem by changing how it acts and which experience it retains. We propose MMLA-RSI, a recursive self-improvement scheme with two implementations in Memory-Mediated Learning Architecture: Reasoning-Time Training (RTT) changes the policy state $\Phi$, while Reasoning-Time Memory Adaptation (RTMA) changes authoritative memory $M$. Each route also makes its adaptation method an object of improvement. A current numerical updater trains the policy proposing its successor; a current memory manager chooses the records available to the process proposing its successor. An independently admitted successor takes over the same role in the next generation. We give bounded executable rule languages, shared task/proposal interfaces, and one algorithm for either route or their composition. The RTT instance changes scalar schedules in a clipped-momentum update. The RTMA instance chooses NULL or a complete-row write with possible eviction. A fixed interpreter, trusted committer, and independent evaluator bound what can change. State versions, read closures, and conditional properties describe causal execution, accepted transitions, and finite work. Existing numerical-component studies motivate the construction. In four-candidate program selection, retaining LARC feedback updates reduces expected error by 24.65 and 36.65 percentage points for two trained initializations. In chronological continuous-integration replay, feedback binding is positive while retention increases loss. Completed SQL studies improve substantially over an unadapted substrate but do not establish the required additional online benefit. These results characterize their declared components. The two recursive routes and their joint realization are specified here; their beneficial recursive chains and discovery efficiency remain to be evaluated.
0
cs.LG Jincheng Gao, Jiahe Yi, Shuo Sun · Oct 14, 2026

MorphingNet attacks the scalability problem in symbolic regression by turning formula search into continuous network training. Its units are pole-free rational functions and complex exponential-logarithm shapes whose parameters morph smoothly between elementary forms, so the network itself is a closed-form formula at every step. A post-training structure-recovery step rewrites the learned expression into compact elementary notation. The paper reports state-of-the-art SRBench accuracy, exact recovery of most ground-truth formulas, and GPU speedups of two orders of magnitude over genetic programming on large data.

We introduce MorphingNet, a network for symbolic regression (SR) whose units change their function continuously. Instead of searching over expressions built from a fixed operator library, MorphingNet trains the form of each function as a parameter, so SR becomes one continuous optimization problem. Its units are either pole-free rational functions or complex exponential--logarithm functions, which move between exponentials, sinusoids, damped oscillations, power laws and logarithms as their parameters change. The network is a readable formula at every step of training, and no operator library is needed to train it; a structure-recovery step afterwards rewrites the trained formula into short elementary expressions. Training is dense linear algebra, so it is GPU-friendly and scales to large data. MorphingNet achieves state-of-the-art accuracy on SRBench, recovers most generating formulas on synthetic benchmarks, and on large data is up to two orders of magnitude faster than the strongest search method. By turning the search for a formula into the training of a network, MorphingNet offers a new paradigm for SR and for scientific discovery from data.
0
cs.LG Chengyi Pan, Huangqimei Zheng, Hao Xue · Oct 14, 2026

CTIA-ODE proposes a continuous-time architecture for long-term multivariate forecasting that sidesteps rigid alignment between endogenous and exogenous variables. Its core mechanism, Continuous-Time Inter-Variable Attention (CTIA), lets a Neural ODE’s evolving hidden state query a static exogenous key-value field during integration. The framework is stabilized by a Linear Residual Augmentation trend anchor and an Outlier-Robust Hybrid Loss. The authors report state-of-the-art results across eight benchmarks with a compact memory footprint.

Although multivariate time series forecasting has broad application prospects, effectively incorporating exogenous variables remains challenging. Prevailing discrete-time architectures often impose rigid temporal alignment across heterogeneous variables, which may limit their ability to capture continuously evolving external effects. To address this limitation, we introduce CTIA-ODE, a continuous-time forecasting framework centered on Continuous-Time Inter-Variable Attention (CTIA). Specifically, CTIA is embedded within an ordinary differential equation (ODE) solver, enabling the continuously evolving endogenous trajectory to dynamically query a static exogenous context along the integration path without requiring explicit temporal alignment. To improve the stability of the continuous latent dynamics, we further introduce a dual-stabilization strategy: architecturally, a Linear Residual Augmentation (LRA) module provides a macroscopic trend anchor, while an Outlier-Robust Hybrid Loss (ORHL) mitigates the influence of heavy-tailed anomalies during optimization. Extensive experiments across eight real-world datasets demonstrate that CTIA-ODE achieves state-of-the-art forecasting performance across diverse datasets and prediction horizons, while maintaining a favorable accuracy-efficiency trade-off with a compact memory footprint and low inference latency, particularly under long-horizon forecasting settings. Anonymous code is available at https://anonymous.4open.science/r/CTIA-ODE-15A6/.
0
cs.CVcs.LG Junhoo Lee, Hyeonjin Kim, Sangbum Han et al. · Oct 14, 2026

Discriminative models generalize to unseen samples, suggesting they capture data-manifold structure beyond their task labels. This paper asks whether that structure can be exposed as a generative signal without retraining, labels, or auxiliary generators. The authors propose the Discriminative Score Function (DSF), which turns the loss-gradient geometry of a fixed discriminative model into an image-space update rule. DSF enables unconditional generation, conditional guidance, inpainting, and zero-shot quantization calibration across classifiers, detectors, and self-supervised encoders.

Discriminative models are trained with task-specific objectives such as classification, detection, or representation learning, yet their practical utility depends on generalizing beyond the training task to unseen samples from the same domain. This suggests that a discriminative model may retain an implicit trace of the data manifold beyond its task labels. We ask whether this trace can be exposed as a generative signal without task-specific guidance. We introduce the Discriminative Score Function (DSF), which extracts an image-space generative field from the loss-gradient geometry of a pretrained discriminative model. DSF provides a training-free update rule that moves images toward model-consistent visual regions without retraining or task-specific targets. We apply DSF to diverse pretrained discriminative models, including ResNet classifiers, DETR detectors, and DINOv2 ViT encoders. DSF generates unconditional images from noise and supports conditional guidance, editing, inpainting, and explanation through the same field. DSF-generated samples also provide effective zero-shot quantization calibration when real images or labels are unavailable, preserving feature geometry better than data-free calibration baselines. These results show that discriminative training leaves a generative functional signal for synthesis and calibration. Project page: https://junhoo.me/dsf.
0
stat.MLcs.LGstat.CO Thibaut Pellerin, Alain Durmus, Elinor Berger et al. · Oct 14, 2026

Standard diffusion models rely on additive Gaussian noise, yet many imaging modalities---including PET and astronomical photon counting---produce discrete Poisson observations. This paper introduces Mixed Poisson Diffusion Models (MPDMs), a generative framework in which the forward corruption is driven by multivariate counting processes rather than Brownian motion. The core payoff is a family of posterior samplers structurally matched to Poisson inverse problems, featuring exact sampling for Poisson denoising and principled adaptations of DPS, PnP-DM, and DAPS.

Diffusion models are a powerful framework for learning complex probability distributions. While widely used diffusion models for continuous data rely on Gaussian noising processes, we instead develop diffusion models based on Poisson noise. More precisely, we introduce and study a new mathematical framework, called Mixed Poisson Diffusion Models (MPDM), with noising dynamics driven by multivariate counting processes. Our methodology is motivated by Poisson inverse problems. The success of Gaussian diffusion models has led to their use as prior models in a growing number of posterior sampling methods for inverse problems. However, most of these approaches focus on Gaussian inverse problems, in which observations are generated through a deterministic forward operator corrupted by additive Gaussian noise. By contrast, Poisson inverse problems, where observations are discrete and Poisson distributed, remain comparatively underexplored in this literature. Building on our Poisson generative dynamics, we develop a natural framework for posterior sampling in Poisson inverse problems and derive several posterior sampling algorithms based on the proposed model. We evaluate our approach on realistic Poisson inverse problems and show that the resulting framework is well suited to this class of problems.
0
cs.LG Yaozhi Wen, Kun Zhao, Wei Wei et al. · Oct 14, 2026

NeuroDETR tackles asynchronous EEG event detection, where the goal is to identify both the class and temporal boundaries of neural events directly from continuous recordings without predefined windows. The method introduces a DETR-style architecture that uses multiscale discrete spatiotemporal vector quantization to build compact event memories, combined with an event-aware sparse attention module called FOCAL that concentrates refinement on likely event periods. Evaluated across eight diverse EEG datasets spanning P300, motor imagery, speech, and seizure detection, the authors report state-of-the-art event-level F1 scores compared to five baselines.

Asynchronous EEG event detection supports practical brain-computer interfaces by identifying event classes and temporal boundaries directly from continuous recordings without predefined event boundaries at inference time. The primary challenge lies in extracting event-related patterns embedded in ongoing background activity. To address this, we propose \textbf{NeuroDETR}, a discrete spatiotemporal multiscale framework that combines structured tokenization with event-aware feature refinement. Multiscale temporal codebooks capture brief responses and sustained dynamics, while Spatial-Pyramid VQ hierarchically encodes spatial structure across EEG channels. These complementary representations form a compact event memory for jointly classifying events and estimating their onsets and offsets. Before tokenization, we apply FOCAL, an event-aware sparse attention module that uses predicted event presence to guide temporal feature refinement and spatial aggregation, emphasizing likely event periods while preserving context across the recording. Experiments on eight EEG datasets that span diverse paradigms demonstrate state-of-the-art performance. These findings provide a basis for the development of BCI systems that recognize meaningful neural activity during continuous interaction and monitoring.
0
cs.CLcs.LG Linhao Zhong, Bozhen Fang, Xuehao Wang et al. · Oct 14, 2026

This paper confronts the evaluation crisis in agentic self-distillation, where a flood of recent methods claim gains over GRPO using inconsistent protocols and reused baselines. The authors conduct a unified benchmark of ten representative approaches across four tasks, three model sizes, and 132 training runs totaling over 12,000 H800 GPU-hours, measuring not only final accuracy but also training stability and wall-clock cost. Their central finding is sobering: no method consistently beats GRPO, many suffer severe hidden instability, and extra compute rarely translates into better agents.

Self-distillation in agentic reinforcement learning has produced a growing number of methods. However, existing evaluations often reuse published baseline scores under differing experimental conditions, provide few direct comparisons among contemporaneous methods, and give limited attention to training stability and computational cost. These limitations obscure the source of reported gains and the practical strengths and weaknesses of competing methods. Costly agent training further motivates a common basis for comparison. To address these gaps, we compare ten representative methods against GRPO on ALFWorld, ScienceWorld, Search-QA, and WebShop across three model sizes. Our study comprises 132 training runs totaling 12,124.4 recorded H800 GPU-hours, with repeated evaluations over eight rollout seeds for the pretrained backbones and all eleven trained methods. Beyond final task performance, we examine training progress, stability, and measured computational cost. Our evaluation yields three key findings. First, gains over GRPO remain limited in scope, with no method consistently outperforming it across settings. Second, strong final performance can mask unstable training. Third, additional training cost does not consistently translate into better performance. We establish this benchmark to support fair, comprehensive comparison and guide future research toward reliable and cost-effective improvements in agent training. Project page: https://aim-uofa.github.io/Self-Distillation-AgentRL/
0
eess.SPcs.LG Mohamed Kamel, Tamer Nadeem · Oct 14, 2026

Single-channel RF receivers cannot use spatial filtering when a desired signal overlaps with a structured co-channel interferer in both time and frequency, a regime where linear filters fail because the impairment is non-Gaussian. FlowSep-RF addresses this by learning a clean-endpoint rectified flow that maps a raw IQ mixture directly to the target waveform, enabling separation with a single neural function evaluation instead of iterative diffusion sampling. The core idea is to treat the received mixture as the source and the clean signal as the target of a near-straight flow, combining time-domain and frequency-domain features to improve reconstruction fidelity and downstream demodulation accuracy. If the gains hold under broader channel conditions, this could provide a low-latency replacement for score-based separators in practical baseband receivers.

Single-channel radio-frequency (RF) signal separation is important for interference mitigation when a signal of interest overlaps with a co-channel interferer in time and frequency. Classical matched filtering and LMMSE receivers are limited under structured non-Gaussian interference, while diffusion and score-based methods often require iterative sampling or optimization. We propose FlowSep-RF, a rectified flow matching framework that learns a clean-endpoint transport from the received mixture to the signal of interest. The model combines time-domain IQ samples with frequency-domain features and recovers the separated waveform using one endpoint velocity evaluation. We evaluate FlowSep-RF on QPSK mixtures with OFDM-BPSK, OFDM-QPSK, and RFChallenge CommSignal2 interference against classical, discriminative, and generative baselines. FlowSep-RF achieves the lowest average BER and MSE among evaluated baselines across all three settings with one-step inference.
0
cs.LG Peilong Zhou, Hongtao Zhang, Yulong Huang et al. · Oct 14, 2026

Looped transformers apply one shared block repeatedly, letting inference depth exceed training depth. This paper asks whether computing an answer at the recurrent frontier guarantees that the model will retain it or that its hidden state can be frozen and reused. Using in-context Mealy-machine execution with exact symbolic targets, the authors show that exact computation, strong retention, and state reusability are three distinct properties: training on a single randomly sampled position per example yields near-perfect retention, yet freezing states when answers first appear collapses accuracy until a short delay passes.

Looped transformers compute by applying one block repeatedly, so they can run for more loops than they were trained with. Recurrent reasoners are known to overthink: iterating past the trained depth can turn a correct answer into a wrong one. Such a late error can reflect failed computation, loss of retention, or a state that later computation cannot reuse. We separate these cases in in-context Mealy-machine execution, where a simulator supplies intermediate targets and exact counterfactuals for hidden-state interventions. Models trained to answer at loop 32 are fully correct there but keep only 22% of these answers at loop 64. Retention depends on what training asks for: supervising one randomly chosen position per example, instead of always the last, keeps 99.8%, and randomizing the number of training loops also helps; truncating gradients, as in progressive training against overthinking, does not. The computation itself is exact: position $k$ is computed at loop $k$, and swapping its loop-$k$ hidden vector for the one from another input redirects every later position to that input's exact continuation (237/237 pairs in each of ten models). Retention, however, is not reuse. Even in models that keep nearly all answers, holding each position $k$ fixed from loop $k$ drops accuracy to 4--6%, whereas holding it from loop $k+4$ is harmless. A released recurrent maze network trained to avoid overthinking behaves the same way. Learning to compute is thus not learning to retain, and avoiding overthinking does not make a state reusable when its answer first appears.
0
cs.CRcs.LG Florian Spelter, Johannes F. Loevenich, Tobias H\"urten et al. · Oct 14, 2026

Flow-based intrusion detection systems increasingly rely on deep learning, but their dependence on large training datasets makes them vulnerable to stealthy data poisoning. This paper demonstrates that latent backdoors can be implanted by poisoning as little as $0.001\\%$ of training data while maintaining near-nominal clean accuracy, and proposes a post-training diagnostic methodology that localizes trigger-relevant features via univariate perturbation search and comparison against a distilled baseline. By enabling targeted data sanitization or neuron pruning, the work exposes concrete operational risks and offers a scalable, interpretable path toward mitigating backdoor exposure in high-dimensional tabular network data.

Although Artificial Intelligence (AI)-based Intrusion Detection Systems (IDSs) are increasingly deployed in operational networks, their dependence on large-scale training data introduces a critical vulnerability to stealthy poisoning attacks. We demonstrate that latent backdoors can be successfully implanted in flow-based IDS models by poisoning as little as 0.001% of the training data, maintaining near-nominal performance on clean inputs while ensuring high attack success rates. To address this threat, we propose a diagnostic-first methodology that localizes trigger-relevant features through constrained perturbation analysis and detects poisoning-induced behavioral drift by comparison with a distilled baseline model. The proposed methodology further enables targeted mitigation via automated data sanitization or neuron pruning. Our findings expose the operational risks of near-undetectable backdoors and provide a scalable approach to analyzing, quantifying, and mitigating backdoor exposure in AI-based IDS.
0
cs.LGstat.ML Hirohane Takagi, Masashi Sugiyama · Oct 14, 2026

Negative guidance in diffusion models aims to steer generation away from undesired conditions, but existing methods rely on local repulsive terms that do not target an explicit density and whose bias cannot be reduced by more computation. This paper recasts the problem as sampling from the explicit avoidance density $p_{\text{target}}(x) \propto p(x)(1-p(c \mid x))^{\lambda}$, estimates the per-timestep posterior $p_t(c \mid x_t)$ via online Positive-Unlabeled learning, and corrects particle drift with Sequential Monte Carlo. The result is a principled sampler that is asymptotically exact in the particle limit and empirically Pareto-dominates existing methods on avoidance-quality trade-offs across Gaussian mixtures, CIFAR-10, and Stable Diffusion.

Negative guidance of diffusion models generates samples that avoid an undesired condition, specified either by reference samples or by a conditional score alongside an unconditional model. Existing methods inject repulsive terms derived from local information at each sample, so the resulting sampler does not target an explicit density, and its bias cannot be systematically reduced by increasing computation. We instead present a negative-guidance method that samples from an explicit target density, which uniformly handles both reference-sample and score-based specifications and suppresses probability mass near the undesired condition with an avoidance strength that parametrizes the target density itself. Realizing this target requires estimating, at each diffusion timestep, how likely each sample is to belong to the undesired condition. We cast this estimation as a Positive-Unlabeled learning problem that admits lightweight online training along the diffusion trajectory, using condition samples as positives and diffusion particles as unlabeled data. Because the guided particles drift from the unconditional marginal, we further correct the residual mismatch via Sequential Monte Carlo, yielding a sampler that is asymptotically exact in the particle limit and that turns any existing negative-guidance method into its proposal kernel. Empirically, our method approaches the ground-truth target on Gaussian mixtures and Pareto-dominates existing methods on trade-offs between avoidance and generation quality, distributional bias, and diversity, including in an image-generation setting with a pretrained score network.
0
cs.LG Zixiao Wang, Xingbo Du, Duzhen Zhang et al. · Oct 14, 2026

Reward model overoptimization in RLHF is usually framed as reward hacking; this paper recasts it as rent-seeking from economics, introducing measurable quantities like rent $\rho(x,y)=z_p(x,y)-z_g(x,y)$, a capture index $\mathrm{CI}=1-\Delta z_g/\Delta z_p$, and deadweight loss. The authors test predictions from regulation economics in a controlled testbed with three 1B-parameter model families, showing that policies learn to exploit whatever specific signals each proxy reward model pays for—whether fabrication or deflection—making overoptimization institutionally contingent rather than universally length-driven.

Reward model overoptimization in RLHF is usually described as reward hacking. We recast it as rent-seeking: the policy, a firm, earns reward from a regulator, the proxy reward model, that true quality does not back. The lens turns overoptimization into measurable quantities, namely rent, a capture index and deadweight loss, and into testable predictions from the economics of regulation. In a fixed, version-controlled testbed with three model families, accuracy-matched proxy reward models and two gold reward models, RL turns 41% to 66% of its proxy gain into rent while gold reward plateaus. Rent follows each regulator's prices. Neither proxy verifies facts, but one pays for confident specifics and the other for displays of diligence, so RL policies learn to fabricate details about nonexistent entities (97% to 100% of probes) or to deflect factual questions (67% to 98%). Kinship between policy and regulator adds a fixed premium of about 0.1 standard deviations but gives the policy no extra leverage under optimization. At a fixed expected fine, frequent mild audits beat rare severe ones, and random audits are as efficient per audit as targeted ones. Bringing the bunching estimator of tax economics to RL training, we also find a first sign that policies learn to stay just below a targeted audit's threshold.
0
cs.CVcs.LG Bilal A. Sidiqi, Yipei Wang, Shaheer U. Saeed et al. · Oct 14, 2026

The paper tackles misalignment between general MRI pretraining objectives and downstream clinical tasks by continually pretraining a 3D prostate MRI ViT encoder. Using LoRA on a prostate segmentation task, the authors evaluate six downstream tasks and use CKA to link representational geometry to performance. The motivation is strong—vision foundation models often need heavy task-specific fine-tuning—but the empirical gains are fractional and inconsistently positive.

Building foundation models for MRI has emerged as a promising direction for advancing medical image analysis and improving accessibility of task specific models. Existing approaches primarily focus on effective pretraining strategies and evaluating their performance across diverse downstream tasks. While these methods show performance gains, limited attention has been given to understanding task alignment between pretraining and downstream objectives. Unlike language models, where the two stages are naturally aligned, such alignment is less evident in vision based applications. Motivated by this gap, we explore continual pretraining. Specifically, continually pretraining a pretrained ViT encoder using Low-Rank Adaptation (LoRA) on a prostate segmentation task to enhance task alignment and downstream performance. We further perform layer-wise representational analysis using Centered Kernel Alignment (CKA) to assess representational shifts and their correlation with task outcomes. Our experiments show improved performance in 3 of 6 downstream tasks and comparable results in the others. Our Feature analysis reveals that, for segmentation tasks, continually pretrained encoder features exhibit greater similarity to those of the finetuned encoder than the original pretrained encoder, correlating with observed performance gains.
0
cs.LGq-bio.QM Sam Gijsen, Moritz Seiler, Kerstin Ritter · Oct 14, 2026

Conditional generative models are increasingly used to predict biological and scientific responses under unseen experimental conditions. This paper investigates whether flow-matching models accurately recover non-additive interactions between factors or instead hallucinate them when extrapolating beyond the training support. The authors propose decomposing interaction errors into recovery and hallucinated components and find that aggregate fidelity metrics often conceal failures to learn interactions, while successful extrapolation depends critically on whether condition structure continues smoothly from observed to held-out regions.

Conditional generative models are increasingly used for scientific prediction beyond the direct training support. Yet scientific questions often concern whether one intervention changes the effect of another. Such interactions can be of primary interest while explaining little of the total variation, allowing models to appear accurate overall without recovering the relevant scientific effect. It remains unclear how these models learn interactions and when these successfully extrapolate to unseen conditions. We therefore study interaction recovery in flow-matching models using a physiological time-series simulator, drug dose-response data, combinatorial cell perturbations, and molecular conformer generations, where we separate recovery from hallucinated interaction structure. In the controlled setting, individual effects are learned much earlier than interactions and models hallucinate interactions where none exist. Across settings, we find qualitatively different generalization problems, with successful prediction depending on whether condition structure continues from observed to held-out targets. When it does, interactions are recoverable. If not, models can generate plausible responses while missing the interaction. We furthermore find that hallucinations increase as targets become less identifiable, and that successful interaction recovery in output space can fail under nonlinear readouts. Finally, we show that aggregate fidelity metrics can conceal whether models learn and extrapolate interactions.