Your paper timeline
Scroll AI takes the way you would scroll a great paper aggregator: quick signal first, deeper critique when something earns your attention, and challenges when a claim feels off.
797 papers in cs.LG
Trending mixes fresh papers with community signal.
0
cs.LG Aneesh Barthakur, Mathias Niepert, Luiz F.O. Chamon · Oct 1, 2026

Guided diffusion forces a costly trade-off between enforcing requirements and preserving sample quality, usually resolved by small guidance scales and long sampling trajectories. This paper introduces LEEGS, which learns a time-dependent guidance schedule end-to-end by backpropagating the guidance loss through the sampling trajectory. A cheap gradient approximation—treating the guidance gradient as a stop-gradient—cuts training time by roughly 4$\times$, letting the method train on only 30 examples and match or beat strong baselines across image inverse problems, face-ID guidance, and PDEs with as few as 50–100 NFEs.

Diffusion models are a powerful generative paradigm used across multimedia and scientific applications. Guided diffusion methods impose requirements on the generation by adding the gradient of a differentiable loss (the guidance function) as a drift term during inference. The weight of this drift (the guidance scale) is critical for the trade-off between data quality and requirement satisfaction. To achieve both of these goals, guided diffusion must resort to small guidance scales and lengthy sampling, incurring high computational costs. This work proposes learned end-to-end guidance schedules (LEEGS) to achieve these objectives with fewer sampling steps. LEEGS trains a time-dependent schedule by minimizing the guidance function over a small set of examples using stochastic gradient descent. Backpropagating through guided sampling is computationally expensive, so LEEGS uses an approximation of the gradient that cuts training time by a factor of 4. We evaluate LEEGS on diverse guidance tasks, including (a) image inpainting, (b) noisy image inverse problems, (c) face-ID-guided generation, and (d) forward and inverse PDE problems, outperforming baselines at equal budget (50 or 100 NFEs), or matching constant guidance with only 10% of the steps.
0
quant-phcs.CRcs.IT Marcel Mordarski, Benjamin Gras, Abdelrahman Shehata et al. · Oct 1, 2026

Quantum key distribution deployments provision security margins assuming stationary channels, yet physical devices drift between calibrations. This paper frames adaptive eavesdropping under trusted channel noise and device drift as a constrained Markov decision process, where an attacker selects compact learnt quantum circuits each round while the noise follows an Ornstein–Uhlenbeck process. By bounding the adaptive value between the best fixed circuit and a dynamic-programming upper bound, the authors let operators measure how conservative their margins are and tune recalibration intervals, addressing a key economic barrier to QKD adoption.

Quantum key distribution (QKD) links are provisioned from security analyses of stationary channels, whereas the devices that determine the channel drift between recalibrations. Whether an eavesdropper who cannot alter the channel's own noise gains by following that drift has not been quantified. Adaptive eavesdropping is posed here as a constrained Markov decision process in which the attacker selects one circuit per round while the noise level follows an Ornstein--Uhlenbeck process and the abort condition is a budget over each block of rounds. The value of adaptation is bounded by the best fixed circuit and a dynamic-programming upper bound. The actions are learnt attacks. Whereas Decker et al. trained a parametrised circuit on a fixed gate template against a fixed channel, here the gate structure and rotation angles are searched jointly. This yields circuits compact enough to form a discrete action set, extending the construction to noise models lacking a known template, including the amplitude damping channel. On device-independent E91 under bilateral depolarising noise, a reinforcement-learning attacker raises her Holevo information from $0.135$ for the best fixed circuit to $0.348$ at zero detection, $98\%$ of the upper bound. On BB84 under a drifting bit-flip channel, she exceeds a conservative noise-indexed rule by $0.024$ in fidelity, reaching $99\%$ of the upper bound. Under stationary noise, the attacker's gain from basis asymmetry changes sign between an averaged and a per-basis error-rate constraint. The search, started from random gate sequences, recovers the analytical cloners and the collective-attack key rate, and meets the lower bound of the Winick--L\"utkenhaus--Coles objective from above.
0
cs.AIcs.LG Negin Ashrafi, Jia Luo, Stacey M. Frumm et al. · Oct 1, 2026

Molecular tumor boards must distinguish truly unsupported therapies from evidence-supported recommendations that still require oncologist review because of incomplete clinical information or poor performance status. This paper introduces OpenMTB-Audit, a 500-case synthetic NSCLC benchmark with four safety labels, and exposes pervasive over-refusal: every tested LLM configuration collapses the Partially Supported label, misclassifying valid-but-caveated therapies as either Unsupported or Supported. To address this, the authors propose MTB-AuditAgent, a deterministic seven-module pipeline that reduces over-refusal to 6.7% and achieves 91.2% accuracy. The work matters because label collapse in high-stakes oncology auditing can conceal unsafe endorsements or inappropriately block appropriate treatments.

Molecular tumor boards integrate genomic findings, clinical context, and therapeutic evidence to support precision oncology. As AI enters this workflow, a key safety challenge is distinguishing truly unsupported recommendations from evidence-supported options that still require oncologist review because of incomplete information, poor ECOG performance status, or other clinical caveats. We introduce OpenMTB-Audit, an open-source benchmark of 500 synthetic non-small cell lung cancer cases spanning five adversarial error categories and four safety labels: Supported, Partially Supported, Unsupported, and Insufficient Information. Across eight large language model configurations, we identify pervasive over-refusal: all LLM configurations failed to retain the Partially Supported label in 83.3-100% of true Partially Supported cases, achieving high aggregate safety scores through label collapse rather than clinically calibrated reasoning. To address this limitation, we developed MTB-AuditAgent, a deterministic seven-module framework separating evidence verification, missing-information detection, safety classification, and abstention. It reduces over-refusal to 6.7% and achieves 91.2% accuracy (95% CI: 88.6-93.6%). A two-oncologist annotation study found disagreement concentrated at the boundary between information sufficiency and treatment optimization, underscoring the need to preserve clinically meaningful distinctions.
0
cs.LGcs.AI Ashok Prasad Neupane, Saugat Adhikari, Pramish Paudel et al. · Oct 1, 2026

iADD tackles the alignment-diversity trade-off in reinforcement-learning finetuning of diffusion models such as Stable Diffusion. Its core idea is to update early denoising timesteps rather than late ones, combined with a sparse incremental curriculum and Feynman–Kac particle branching during training. Better alignment without mode collapse matters because existing reward-optimization methods frequently degrade visual quality and diversity.

Reinforcement learning based post training of diffusion models, such as Denoising Diffusion Policy Optimization (DDPO), optimizes a reverse diffusion process under a reward function. However, current approaches to reward optimizations do so at the cost of diversity and quality. In this paper, we provide better tradeoffs through careful theoretical considerations and method design. We analyze the theoretical framework and mathematically demonstrate that \emph{only-latter timestep} updates of diffusion model may be harmful for diversity contrary to the conclusions presented in a previous work. Additionally, we propose an incremental Feynman-Kac training based on strong theoretical foundations in order to achieve the best-yet alignment-diversity tradeoffs. We perform extensive experiments and compare our method against related diffusion policy optimization approaches in three different tasks and also provide strong ablations for each component, thus validating strong performance gains in both alignment and diversity.
0
cs.LG Riccardo Ali, Alessio Borgi, Mario Severino et al. · Oct 1, 2026

The paper isolates matrix-valued transport from the broader sheaf-diffusion framework by casting it through quiver representations. Its core insight is that standard multi-head attention implements diagonal transport over the head space, and that allowing edge-dependent off-diagonal routing before aggregation yields a strictly richer message-passing primitive. The authors introduce Topological Attention (Top-A), which adds cross-head communication while exactly recovering vanilla attention when the learned routing vanishes. Benchmarks on relational, heterogeneous, and algorithmic tasks show that the gains are task-dependent: largest when individual interactions require distinct transformations, but not systematically present under heterophily alone.

Sheaf Neural Networks generalize scalar-weighted message passing by replacing scalar edge weights with linear transport maps between local feature spaces. Yet the role of this matrix-valued transport is entangled with the broader sheaf-diffusion construction. We isolate the transport primitive through quiver representations and establish a direct connection with multi-head attention. Treating attention heads as coordinates of a local transport space reveals that standard multi-head attention implements diagonal edge maps: along each directed interaction, a source head can contribute only to the corresponding receiver head. Allowing off-diagonal entries instead enables edge-conditioned communication across heads before neighborhood aggregation. We show that this operation cannot, in general, be absorbed into a single shared linear map applied after aggregation. Building on this characterization, we introduce Topological Attention (Top-A), a multi-head attention that learns edge-dependent off-diagonal routes while preserving the original same-head paths and exactly recovering vanilla attention when the additional routing vanishes. We evaluate Top-A on relational reasoning, heterogeneous graph learning, and algorithmic reasoning, including out-of-distribution generalization, with heterophilic node classification as a contrast setting. The results show that cross-head transport is most useful when the task benefits from interaction-dependent transformations, while heterophily alone provides no systematic advantage. These findings identify edge-conditioned cross-head communication as a distinct computational primitive of matrix-valued transport.
0
cs.ROcs.LG Michael Y. Fatemi, Jinhao Liang, Ferdinando Fioretto · Oct 1, 2026

Multi-agent motion planning with diffusion models currently demands costly training on large feasible-trajectory datasets, making learned planners map-specific and data-hungry. This paper introduces TFDP, a training-free diffusion-style planner that replaces global neural score networks with analytical local scores derived from smoothness, obstacle-avoidance, inter-agent separation, and velocity-limit terms. By exploiting the spatial and temporal locality of trajectory refinement, the method scales to hundreds of agents and obstacles in seconds without any demonstrations or neural network training.

Path finding and multi-robot motion planning require trajectories that are smooth, goal-directed, and collision-free in environments with complex geometric constraints. Recent diffusion-based planners have shown that trajectory generation can be cast as iterative denoising which has opened the doors to learning-based approaches that can handle multi-modal trajectory distributions and refine entire trajectories. However, a key limitation is that diffusion planners require training on large collections of feasible trajectories, rendering them map-specific, and difficult to deploy when high-quality demonstrations are unavailable. This paper introduces a training-free diffusion-based motion planner that replaces learned global trajectory scores with analytical local scores derived from obstacle, smoothness, velocity, and inter-agent feasibility terms. The proposed idea relies on a key observation: the score of a trajectory can be reconstructed by considering only local interactions between neighboring waypoints and nearby constraints. This structure exploitation yields a decomposed denoising procedure that retains the optimization structure of classical trajectory methods while inheriting the iterative refinement behavior of diffusion models. Experiments on a large collection of complex environments and large multi-agent planning tasks show that the proposed analytical score produces smooth and feasible trajectories within limited computational costs, for example in generating feasible paths for 300+ agents in environments containing 100+ obstacles in under 6 seconds on a GPU, outperforming strong learning-based and optimization baselines, while avoiding the data requirements of learned diffusion planners.
0
stat.MLcs.LG Leyang Wang, Yakun Wang, Song Liu et al. · Oct 1, 2026

Comparing high-dimensional discrete distributions is notoriously difficult because the state space grows exponentially and differences can hide in complex joint dependencies. This paper introduces Zero Flux, a flow-based discrepancy that extends the continuous Zero Flow criterion to discrete state spaces. The core idea is to construct a discrete bridge between two distributions via coordinate-wise independent coupling, measure local probability fluxes at the midpoint, and show that all fluxes vanish if and only if the distributions are identical. This yields a decomposable, sample-based discrepancy with finite-sample guarantees, offering both a global comparison score and interpretable coordinate-wise contributions.

Comparing two high-dimensional discrete distributions has always been a challenging task due to the exponentially growing state space and complex changes in interactions. A recent work suggests comparing distributions through a vector field trained using flow matching between two continuous distributions. The resulting vector field at mid-point vanishes if and only if two distributions identical. However, such a flow-based criterion does not naturally apply to discrete distributions. We extend this principle to the discrete domain and introduce the \emph{Zero Flux} criterion, a discrepancy based on local probability fluxes. Under independent coupling, we show that all local probability fluxes vanish at the midpoint if and only if two distributions are the same. This discrepancy decomposes the joint distributional difference into smaller, local contributions and can be efficiently estimated from samples. We establish finite sample error bounds for our estimator. Experiments on synthetic and real categorical data demonstrate reliable recovery of sparse dependence signals and stable tracking of distribution shifts in high dimensions.
0
cs.LGcs.IR Parthiv Chatterjee, Dhiraj Golhar, Ummesalma Diwan et al. · Oct 1, 2026

Personalization encoders compress lengthy user interaction histories into a fixed preference state for downstream ranking or generation, but this compression can suppress evidence that remains available in cached per-timestep representations. This paper proposes REPAIR, a lightweight module that identifies and aggregates corrective signals from a frozen encoder's own cache across long, short, and episodic temporal views, then adds the correction back to the compressed state before the task head. By keeping the host encoder and task head frozen, REPAIR offers a practical way to improve deployed personalization systems without retraining them.

Personalization encoders compress evolving interaction histories into preference states used to rank items or condition text generation. A task head operating only on this state can miss useful evidence that remains in the frozen encoder's cached representations for individual timesteps. We study this recoverability gap and propose REPAIR, which compares cached representations with the current preference state in a compact learned coordinate space. It resolves corrective evidence over extended history, recent interactions, and localized bursts. It then selects which patterns at which timesteps contribute and adds their aggregate correction to the state before the task head. Encoder-host repair reuses representations from the existing forward computation without re-encoding the history. Across MovieLens, PENS, MIND, and Amazon Reviews 2023, training only REPAIR improves MRR and nDCG@10 for all twelve representative recommendation hosts while both encoder and task head remain frozen. Head-only finetuning of the same hosts yields smaller gains. For example, Mamba4Rec on MovieLens gains 3.96 MRR points, compared with 0.19 from head-only finetuning. Rank and temporal diagnostics support a compact, host-dependent corrective structure. In personalized generation, IMPerSumm improves the two reported weighted PerSEval variants, which assess responsiveness to user preference, by up to 25.23%. These results support post-compression state correction and distinguish the availability of preference evidence from its downstream use.
0
cs.LGmath.OC Jichao Jiang (1), Cristian McGee (1), El Houcine Bergou (2) et al. · Oct 1, 2026

TACO attacks the memory bottleneck of full-parameter LLM fine-tuning by replacing dense AdamW states with a geometry-induced sparse update. The core idea is to use a dimension-normalized $\ell_1 \to \ell_1$ operator norm, whose exact steepest-descent direction is a ternary, column-wise one-sparse matrix. By retaining only a tiny FP8 heavy-hitter history per column, TACO slashes persistent optimizer state by over $170\times$ and peak memory by nearly $3\times$, enabling 30–32B parameter models to fine-tune on a single 80 GB GPU.

Full-parameter fine-tuning of large language models (LLMs) incurs substantial optimizer state memory overhead, limiting the model sizes that fit on modern GPUs. Existing approaches either compress optimizer state, abandon first-order gradients, or change the update geometry while retaining dense state. The recently introduced Muon optimizer reduces optimizer memory through matrix-valued updates. Still, its geometry differs from AdamW and can lead to performance degradation when fine-tuning AdamW-pretrained models. To reduce optimizer memory without sacrificing accuracy or computational efficiency in LLM fine-tuning, we propose Ternary Absolute-max Column-wise One-sparse optimizer, or TACO, which follows Muon's operator-norm steepest-descent view but takes the geometric route further. TACO computes the exact steepest-descent direction under a dimension-normalized $1\to1$ operator norm by selecting the sign of the largest magnitude entry in each column of two-dimensional weight matrices. This retains first-order gradients while making optimizer state memory nearly negligible. Our practical TACO optimizer maintains only a small set of low precision gradient components per column, reducing persistent optimizer state by $174\times$ relative to AdamW8bit (from 27.7 GB to 0.16 GB) and peak training memory by $2.9\times$ (from 80.6 GB to 27.5 GB) on OPT-13B, while achieving comparable accuracy and runtime. TACO further enables full-parameter fine-tuning of 30-32B-parameter models on a single 80 GB H100 GPU across multiple model families and tasks.
0
cs.LGstat.ML Luca Geminiani, Nadja Klein · Oct 1, 2026

IQS-BO tackles the per-iteration cost of Bayesian optimisation by replacing surrogate fitting and acquisition-function maximisation with a single transformer forward pass. The core idea is to cast query selection as supervised classification: given a finite candidate set, the model predicts which candidate maximises the objective, trained with cross-entropy on synthetic priors where the true best candidate is known for free. The method can run fully in-context or on top of a fixed surrogate, and it introduces a mixture prior of GP and non-stationary functions to improve optimisation performance.

Bayesian Optimisation (BO) is a powerful framework for the optimisation of expensive black-box functions, but typically requires refitting a surrogate and maximising an acquisition function at every evaluation step. In-context approaches based on Prior-data Fitted Networks (PFNs) amortise part of this cost by pre-training transformers on functions drawn from synthetic priors. PFNs4BO amortises the surrogate but still relies on a numerically maximised acquisition function, while FIBO performs BO fully in-context by sampling optimiser locations from a learned density, which fixes the decision rule and admits no surrogate. Learned acquisition functions score a finite candidate set with a trained network, but, lacking a label for the query, learn the score by reinforcement learning on previously solved tasks. We propose IQS-BO, a PFN that learns the query decision by supervised learning on synthetic priors. In a single forward pass, IQS-BO predicts the probability that each candidate maximises the objective over the set, and we show that the minimiser of its objective is the posterior probability of this event. The model can be pre-trained without a surrogate for fully in-context BO, or take the predictions of a fixed probabilistic surrogate as additional input, amortising only the decision step. Our method proposes queries at a fraction of the cost of acquisition-based methods, while either matching or outperforming standard BO with Gaussian processes (GPs) and available in-context methods on synthetic and real-world benchmarks. Finally, we propose a mixture prior for pre-training PFNs which combines samples from GPs with functions exhibiting warped inputs, isolated narrow optima, or plateaus that are poorly modeled by stationary kernels common in GP surrogates. We show that pre-training on this prior can lead to improved optimisation performance.
0
cs.ROcs.AIcs.HC Amr Mousa, Rifny Rachman, Neil Karavis et al. · Oct 1, 2026

Quadrupedal locomotion requires balancing command tracking, stability, and energy efficiency, yet conventional RL bakes these trade-offs into a static scalar reward that cannot change after training. PROMO proposes a preference-conditioned multi-objective framework that takes the relative priority vector $w_t \in \Delta^{K-1}$ with $K=3$ as a runtime input to a single policy, while keeping embodiment-specific locomotion priors fixed. On a Unitree Go2, the same policy transferred zero-shot modulates behavior through preference changes alone, cutting specific energy by up to $30.4\%$, position error by $38.7\%$, and peak body-attitude deviation by $59.0\%$.

Quadrupedal locomotion requires balancing conflicting objectives such as command tracking, stability, and energy efficiency, yet conventional reinforcement learning (RL) hardcodes these priorities into a fixed scalar reward at training time. We present PROMO (Preference-Conditioned Multi-Objective Reinforcement Learning), a semantic multi-objective approach that makes this trade-off an explicit runtime input to a single locomotion policy. PROMO conditions the policy on deployment facing preferences while keeping embodiment-specific locomotion priors fixed, thereby separating operator intent from reward shaping terms required for viable gait generation. Compared with fixed-objective controllers, multi-objective baselines, and independently trained specialists, PROMO achieves objective specialization and robustness from a single deployable policy. Across 100 sampled preferences in simulation, 67 behaviors are non-dominated under exact Pareto dominance, with a mean preference-objective correlation of 0.843, demonstrating broad Pareto coverage and predictable preference response. The same policy transfers zero-shot to a Unitree Go2, where preference changes alone reduce specific energy by up to 30.4%, position error by 38.7%, and peak body-attitude deviation by 59.0% relative to the balanced preference. These results establish preference-conditioned multi-objective RL as a practical runtime interface for adaptive legged locomotion, extending its role beyond offline Pareto-set construction. Open-source code and videos are available at https://amrmousa.com/promo/.
0
cs.NEcs.LG Aviral Gandhi, Jinglue Xu, Jialong Li et al. · Oct 1, 2026

LESS introduces a lightweight neural architecture search pipeline that marries a brief fair supernet warm-up with candidate-conditioned hard-path updates inside a CMA-ES loop. The core claim is that just six balanced SGD steps per proposal are enough to sharply reduce selection regret, enabling NAS-Bench-201 searches in roughly seven minutes on a single RTX 4090 without retuning across datasets. If this speed–accuracy trade-off holds more broadly, it could lower the barrier to reproducible NAS experiments.

Low-cost NAS must both explore high-performing architectures and identify them reliably, yet reducing evaluation cost often weakens the fidelity of candidate comparisons. Training-free methods reduce evaluation cost by replacing learned task feedback with proxy signals measured at initialization. We introduce LESS (Lightweight Evolutionary Supernet Search), a data-driven method that combines a brief fair hard-path warm-up with discrete search under a single CMA-ES distribution. Each proposal is evaluated as its decoded hard genotype after six candidate-conditioned supernet updates. On NAS-Bench-201, LESS achieves \(93.189\pm0.467\%\) CIFAR-10 test accuracy in 409.1 seconds, coming within 0.04 percentage points of FairNAS using approximately \(1/24\) of its source-reported search time. Matched controls show that calibration improves selected validation accuracy by \(0.577\) percentage points while changing best-visited accuracy by only \(0.054\) points, indicating that its primary effect is to reduce selection regret. The frozen configuration transfers without tuning to CIFAR-100 and ImageNet16-120 with \(69.615\pm1.139\%\) and \(43.720\pm1.697\%\) accuracy. Applied without tuning to the larger DARTS space, LESS achieves \(96.95\pm0.14\%\) on CIFAR-10 and \(82.43\pm0.80\%\) on CIFAR-100, with each search completing in approximately 43.5 minutes on a single GPU. Together, these results show that short, balanced, data-dependent updates enable competitive neural architecture search across datasets and search spaces within minutes.
0
cs.LGquant-ph Bisma Majid, Shabir Ahmed Sofi, Mir Mohammad Yousuf · Oct 1, 2026

This paper studies how to make Quantum Reinforcement Learning (QRL) for combinatorial optimization more robust on near-term quantum devices. Its core idea is APGEM, a LinUCB contextual-bandit controller that dynamically selects among error-mitigation techniques—Zero-Noise Extrapolation, Probabilistic Error Cancellation, Clifford Data Regression, and Readout Error Mitigation—based on a cost-aware utility that balances fidelity gain against shot cost and estimator variance. The work matters because static error mitigation cannot adapt to changing noise regimes or training phases, and adaptive orchestration could improve the reliability of hybrid quantum-classical loops.

Quantum Reinforcement Learning (QRL) integrates reinforcement learning with parameterized quantum circuits and is a promising approach to combinatorial optimization. On Noisy Intermediate-Scale Quantum (NISQ) devices, however, decoherence, gate imperfections, and measurement errors reduce policy quality and make learning less reliable. Existing error mitigation techniques are generally applied as fixed corrections that do not adapt to changing noise conditions or to the evolving state of training. This work presents Adaptive Policy-Guided Error Mitigation (APGEM) as a context-aware orchestration layer of the hybrid quantum-classical training loop that dynamically selects the most suitable mitigation strategy during QRL training. APGEM evaluates Zero-Noise Extrapolation (ZNE), Probabilistic Error Cancellation (PEC), Clifford Data Regression (CDR), and Readout Error Mitigation (REM) using policy-level indicators, including quantum-state fidelity, policy entropy, cumulative reward, and approximation ratio, and integrates the selected strategy directly into the reinforcement learning loop. The framework is evaluated on the Capacitated Vehicle Routing Problem (CVRP), a representative NP-hard problem in urban logistics, under a range of NISQ noise models and noise levels. APGEM consistently outperforms conventional static mitigation methods, reaches approximately 94% of the utility of an oracle strategy, maintains higher quantum-state fidelity as noise increases, and produces more stable learning behaviour throughout training. Ablation studies show that the framework learns context-aware mitigation policies that adapt to different noise environments and circuit execution conditions. These findings demonstrate that integrating adaptive error mitigation into the learning process substantially improves the robustness and reliability of QRL on NISQ hardware.
0
cs.LG SeongHyeon Kim, Chaeyun Jang, Seungyoo Lee et al. · Oct 1, 2026

Unified language models must handle heterogeneous capabilities such as mathematics, coding, instruction following, and controllable reasoning, yet standard single-trajectory training is notoriously sensitive to ordering and data ratios. Naive weight-space merging of single-objective experts offers a modular alternative but often collapses distinct behaviors or degrades cross-domain performance. This paper proposes Mixture-Trained Merging (MTM), which trains each branch on an objective-biased data mixture rather than a pure objective so that the branches stay in mutually compatible regions of parameter space and can be merged at deployment time.

Unified language models are increasingly expected to combine heterogeneous capabilities, such as mathematics, code, instruction following, and controllable thinking behavior, within a single set of parameters. A common solution is sequential post-training on multiple objectives, but this entangles all objectives along one optimization trajectory and makes the final model highly sensitive to training order, data ratios, schedules, and stopping criteria. Weight-space merging offers a modular alternative, but naive merging of single-objective experts often fails: domain capabilities degrade sharply, or think/non-think modes collapse into one dominant behavior. We attribute both failures to incompatible weight-space geometry: experts trained on single objectives drift to distant regions of parameter space, placing their interpolations outside any shared low-loss basin. We propose Mixture-Trained Merging (MTM), which trains each branch on an objective-biased data mixture rather than a single objective, exposing it to cross-objective interactions and making branches compatible at merge time. MTM uses merged-model evaluations as a low-cost signal for selecting branch mixtures, avoiding expensive data-mixture ablations. The procedure is iterative: each round promotes the base model using globally selected merge coefficients and refines each branch mixture using domain-preferred coefficients under constraints that preserve other objectives. To scale beyond simplex grid search, MTM uses qNEHVI-based multi-objective Bayesian optimization. Across code, mathematics, instruction following, and think/non-think control, MTM outperforms naive merging and preserves behavioral separation where single-objective merging collapses, suggesting that effective unified models require branches trained to be mergeable.
0
cs.LGcs.AI Assaf Ben-Kish, Akarsh Kumar, James Glass et al. · Oct 1, 2026

Catastrophic forgetting remains a critical barrier in multi-phase training of large language models. This paper recasts forgetting as a geometric phenomenon at the level of each weight matrix: standard gradient updates act on all inputs, not just those from the new task, corrupting prior mappings. The authors propose Local Support Learning (LSL), which pairs task-specific adapters with per-matrix Gaussian Mixture Model gates that restrict updates to activations near the current training distribution. Empirically, LSL retains pretrained math, coding, and instruction-following capabilities in models up to 7B parameters while learning new domains efficiently.

We explore catastrophic forgetting in the context of large pre-trained models. By considering forgetting as a geometric problem in the input space of each weight matrix, we uncover a natural retention objective under which updates produced by gradient-based optimizers are suboptimal. Following this observation, we propose Local Support Learning (LSL), a general-purpose framework that augments gradient-based training for retention of prior capabilities without access to prior data. During a new learning phase, LSL pairs two components with distinct roles: a standard weight adapter, trained as usual to minimize the loss, and a gating function that enables the adapter only on input activations from its own training distribution, making the update local to that distribution. The key challenge is that this gate must route data from all learning phases while training only on data from the current one. We address this with a gate based on a Gaussian Mixture Model (GMM), whose likelihood decays rapidly away from its training data, giving it a natural tendency to stay closed on data from prior phases. We show that this post-training approach can resolve forgetting in LLMs of up to 7 billion parameters, retaining both pretrained and finetuned capabilities across multiple training phases, while being efficient in memory and compute, robust to hyperparameter choice, and showing scaling potential.
0
cs.LGcs.AIcs.RO Sebastian Sanokowski, Alireza Sarmadi, Majid Khadiv · Oct 1, 2026

Policy-gradient methods suffer from high variance, while pathwise actor updates require accurate action derivatives that standard value-function training does not guarantee. FERPO addresses this by deriving an entropy- and KL-regularized target action distribution from critic values alone, then fitting the policy with forward KL via self-normalized importance sampling---never differentiating the critic with respect to actions. The method targets massively parallel on-policy robot learning, where reliable policy improvement is critical for both sample and wall-clock efficiency.

Several state-of-the-art methods for online reinforcement learning in continuous control improve policies using action gradients of a learned critic. However, critics are typically trained to predict returns, and accurate value predictions do not necessarily yield accurate action derivatives, potentially leading to unreliable policy updates. We propose Forward Entropy-Regularized Policy Optimization (FERPO), an on-policy maximum entropy reinforcement learning algorithm that performs policy improvement using critic values without differentiating the critic with respect to actions. FERPO derives an optimal target action distribution from a policy-improvement objective regularized by entropy and Kullback-Leibler (KL) divergence. We then fit the actor to this target by minimizing a forward-KL objective, estimated using self-normalized importance sampling (SNIS) with actions drawn from the rollout policy. By limiting the target distribution's deviation from the rollout policy, the KL regularization helps keep these importance weights well behaved. In contrast to reverse-KL objectives, which can favor a subset of the target distribution's modes, the forward-KL objective encourages coverage of multiple high-value modes and thereby promotes exploration. Experiments and ablations on MuJoCo Playground and ManiSkill show competitive performance and sample-efficiency gains. Computational benchmarks also demonstrate faster actor updates than Relative Entropy Pathwise Policy Optimization (REPPO).
0
cs.CVcs.AIcs.CL Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc et al. · Oct 1, 2026

Where-OPD proposes a new on-policy self-distillation paradigm for multimodal large language models. Instead of giving the teacher zoomed-in image crops, it provides textual spatial hints that identify question-relevant visual elements and their coordinates, using procedurally generated synthetic counting scenes where object locations are known by design. The student learns to match the teacher’s behavior from the raw image and question alone, and the authors claim that training on just 3,000 such synthetic scenes transfers to broad real-world perception gains across 15 benchmarks.

On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a frozen or EMA version of themselves that receives privileged information. Its application to multimodal large language models (MLLMs), however, remains largely unexplored. Recent approaches use privileged visual information, such as image crops corresponding to a question, to improve fine-grained perception, but their gains are confined to tasks that benefit from such visual zooming and require either human-annotated grounding data or external teacher models. We introduce a different form of on-policy self-distillation for MLLMs that provides the teacher with textual, spatially grounded guidance identifying the visual elements relevant to a query. We use procedurally generated scenes with automatically available object identities and spatial coordinates, enabling scalable and annotation-free post-training. The teacher uses this spatial guidance to locate and integrate evidence from multiple relevant image regions, while the student learns to reproduce the resulting behavior from the image and question alone. Our approach consistently improves performance on counting, document and chart understanding benchmarks across multiple models. Importantly, although post-training uses only synthetic scenes, the resulting improvements transfer to real-world perception benchmarks, yielding a 3.23-point gain in average performance across CVBench, V*, ZoomBench, BLINK, HR-Bench, and MME-RealWorld. These results show that spatially grounded privileged information can induce broader perceptual capabilities through on-policy self-distillation, enabling substantial synthetic-to-real transfer beyond the task and data distribution used for post-training. Project page: https://github.com/sirkosophia/Where-OPD
0
cs.LG Akshay Balsubramani · Oct 1, 2026

The paper solves the generalized Schrödinger bridge on finite graphs—transporting probability mass between distributions while charging costs for visited states—by showing that state costs fold into the reference process via a Feynman–Kac tilt. This reduces the problem to solving a plain bridge against the tilted generator $A = Q - \operatorname{diag}(f)$ with sparse matrix-exponential iterations, eliminating the need for learned controls or tuned temporal-difference penalties. The method scales to millions of nodes and extends to congestion costs through a provably convergent damped best-response outer loop.

The generalized Schr\"odinger bridge on a graph moves mass between two distributions while charging a cost for the states visited. It has been approached by learning the rates of a controlled continuous-time Markov chain, with a temporal-difference penalty that restores the cost. A state cost folds into the reference process as a Feynman-Kac tilt. The cost-augmented bridge is then a plain bridge against the tilted reference, and the penalty is unnecessary. The bridge is computed exactly by alternating two endpoint rescalings, each one sparse matrix-exponential application; nothing is discretized in time or learned. The alternation converges at a rate set by the endpoint coupling alone. For a quadratic congestion cost on time-averaged occupancies, damped best response around the exact bridge is gradient descent on a strongly convex function, and its residual bounds its error. On a protein-folding model, a free-energy cost lowers the expected barrier of the folding paths. On the learned approach's road network, roll-outs of the exact bridge match the target within sampling error, and on networks with millions of intersections its memory grows linearly.
0
cs.CLcs.AIcs.LG Hui Ren, Zihan Li, Chang Liu et al. · Oct 1, 2026

Hierarchical Continuous Diffusion Language Models tackle a core bottleneck in non-autoregressive generation: discrete diffusion decodes parallel tokens as independent marginals, while continuous diffusion lacks valid token anchors until the final step. The authors propose HC-DLM, a hierarchy where a continuous latent trajectory is the sole persistent state and tokens are read out and fed back at every step to scaffold the next latent update. A single variational bound yields the training objective, and experiments span structured reasoning, mathematical planning, and language modeling.

Discrete diffusion language models offer a compelling alternative to autoregressive generation for tasks demanding bidirectional reasoning and global constraint satisfaction. Yet they share a structural bottleneck: when decoding in parallel, each token is sampled independently from its marginal, severing the statistical dependencies among the tokens decoded together. Continuous diffusion language models avoid this by denoising a shared continuous state, but their denoiser sees only that state, so nothing ties it to a valid token configuration until it is finally decoded. To address this, we propose Hierarchical Continuous Diffusion Language Models (HC-DLM), which couple discrete token generation with a continuous latent trajectory in a single, principled denoising process, whose training objective is derived from a variational bound on the token likelihood. In contrast to recent methods that attach continuous context to a self-contained discrete chain, HC-DLM makes the latent the only persistent generative state: tokens are read out from it at every step and feed back as a scaffold for the next latent update. On structured reasoning (Sudoku), mathematical planning (Countdown) and language modeling (LM1B), HC-DLM improves over discrete and continuous diffusion baselines at matched model size, in puzzle accuracy on Sudoku and Countdown and in generative perplexity on LM1B. Project page: https://hc-dlm.github.io/.
0
cs.LGcs.AI Jinhwan Sul, Panagiotis Theodoropoulos, Vincent Pacelli et al. · Oct 1, 2026

Schrödinger bridge (SB) transport assumes a fixed initial distribution, which breaks at test time when inputs shift. This paper introduces Distributionally Robust Schrödinger Bridge (DRSB), which learns a single controller by minimizing the worst-case sum of control energy $\int_0^1 \frac{1}{2}\|u_t^\theta(X_t)\|^2 dt$ and a terminal KL penalty $\rho \mathrm{KL}(p_1^{u,\mu}\|\nu)$ over an ambiguity set $\mathfrak{M}_\epsilon(\hat{\mu})$ of initial distributions. The authors derive a variational reformulation, connect it to stochastic optimal control and distributionally robust optimization, and develop Wasserstein and Sinkhorn variants with alternating block updates. Experiments on 2D transport and FFHQ image-to-image translation show improved robustness to input perturbations, with a tradeoff in nominal performance.

Schr\"odinger bridge (SB) learns stochastic transport between prescribed initial and target distributions. When the initial distribution shifts at test time, the learned dynamics can fail to recover the target distribution. We introduce the Distributionally Robust Schr\"odinger Bridge (DRSB), which learns a single controller that accounts for uncertainty in the initial distribution. The DRSB objective consists of control energy and a KL penalty between the resulting terminal distribution and the target distribution. DRSB seeks a single controller that minimizes the worst-case value of this objective as the initial distribution varies within an ambiguity set around the nominal distribution. We derive an exact variational formulation of this objective and connect its fixed-terminal-cost subproblem to stochastic optimal control and distributionally robust optimization. This formulation motivates an alternating algorithm that updates the adversarial initial distribution, estimates the terminal log-density ratio, and trains the controller. We develop Wasserstein and Sinkhorn variants using stochastic control optimality conditions to approximate the gradients required for adversarial updates. Experiments on two-dimensional transport tasks and image-to-image translation show improved robustness to input perturbations relative to standard SB, with a tradeoff in nominal performance. On Gaussian mixture transport, Sinkhorn DRSB also achieves lower mean sliced Wasserstein distance than fixed-level noise augmentation at both tested unseen noise levels.