Your paper timeline
Scroll AI takes the way you would scroll a great paper aggregator: quick signal first, deeper critique when something earns your attention, and challenges when a claim feels off.
797 papers in cs.LG
Trending mixes fresh papers with community signal.
0
cs.AIcs.IRcs.LG Aleksei Medvedev, Alejandro Ariza-Casabona, Steven Derby et al. · Oct 1, 2026

Generative recommenders represent items as short discrete Semantic IDs (SIDs) that are decoded autoregressively, yet most prior work derives these codes by quantising isolated content embeddings. This paper argues that SID construction is fundamentally a recursive clustering problem over a collaborative item graph, and proposes GrIS, a framework that separates graph construction from hierarchical partition to balance semantic and behavioural signals. By exposing these two axes, the authors show that content-only tokenisers are merely empty-graph instances of a broader design space, and they demonstrate new instantiations that improve SID quality for downstream recommendation.

Existing work on Semantic IDs (SIDs) for generative recommendation treats SID construction as a representation learning problem: encode items into a quantised latent space and read off codes. We argue this view is incidental. SID construction is, at heart, a recursive clustering problem, and once stated this way the natural object to cluster is a graph whose nodes carry semantic content and whose edges carry collaborative signal; SID assignment becomes a hierarchical graph partition. This reframing yields a unified framework, Graph-Informed Semantic IDs (GrIS), that subsumes prior approaches rather than displacing them. RQ-VAE and RQ-KMeans are recovered as the special case where the graph is empty, exposing content-only quantisation as one corner of a larger design space along two so-far-collapsed axes: graph construction and recursive partition algorithm. We explore two contrasting instantiations: RecDMoN, which performs hierarchical assignment via differentiable graph pooling, and RQ-GAE, which extends RQ-VAE with graph-aware item representations and a graph reconstruction objective. On multiple real-world datasets, GrIS consistently improves over CF-aware SOTA, with gains of up to +52\% Hit@10. Because graph construction and partition are explicit, separately configurable components, improvements on either axis can be combined and evaluated systematically.
0
cs.CVcs.LG Yuan Huang, Zirui Song, Xiuying Chen · Oct 1, 2026

The paper tackles a critical problem in automated image-editing evaluation: multimodal large language models (MLLMs) acting as judges may be swayed by irrelevant presentation cues rather than the edit itself. Because visual interventions can alter the very quality being assessed, the authors introduce EditJudgeBias, a counterfactual benchmark of 1,196 real samples where 13 cues are injected across four evaluation stages and rigorously verified to preserve underlying quality. They audit five judges along three dimensions—invariance to cues, agreement with humans, and pairwise stability—and show that every judge shifts beyond its own noise floor, with fabricated social proof inflating scores and display-order swaps reversing up to 60.9% of verdicts.

Multimodal large language models (MLLMs) are increasingly used as automated judges for instruction-based image editing and as reward signals for model training. However, systematically auditing whether these judges are influenced by cues irrelevant to editing quality is challenging because visual interventions may themselves alter the quality being evaluated. A judgment shift can therefore be attributed to bias only when the intervention is verified to preserve the underlying editing quality. To address this challenge, we introduce EditJudgeBias, a counterfactual benchmark with verified quality preservation, comprising 1,196 real editing samples and 13 cues injected across four evaluation sites. We verify quality preservation for the requested edit using calibrated multimodal validators, controls, and human inspection. We then audit five MLLM judges along three complementary dimensions: invariance to quality-preserving cues, agreement with human judgments, and stability of pairwise preferences. Importantly, observed shifts are evaluated against each judge's own zero-dose and re-query noise floors rather than against zero. Experiments show that quality-preserving cues move every judge beyond its own noise. Fabricated majority opinions increase ratings, irrelevant visual elements cause larger shifts than whole-image manipulations, and swapping candidate order reverses up to 60.9% of pairwise decisions. Edit-region cues also tend to reduce human agreement. The three measures characterize judges differently, showing that robustness cannot be captured by a single metric.
0
cs.LG Tugrul Cabir Hakyemez, Ener Uras Gokhan · Oct 1, 2026

Predictive maintenance models are typically tuned to maximize $R^2$, a symmetric metric that ignores the fact that late remaining useful life (RUL) predictions are far more costly than early ones. This study treats the optimization objective itself as a variable, comparing single-objective and multi-objective hyperparameter optimization across MLP, LSTM, XGBoost, TCN, and Transformer on NASA C-MAPSS and BackBlaze benchmarks. The core result is that NSGA-II with Entropy–CRITIC selection cuts directional imbalance by roughly a third on simulated data without sacrificing accuracy, though this advantage collapses into near-random behavior on real-world hard-drive data.

In predictive maintenance, early and late RUL prediction errors carry asymmetric consequences, yet hyperparameter optimization typically targets a single accuracy metric that treats both directions equally. This study treats the optimization objective itself as a design variable. Five architectures (MLP, LSTM, XGBoost, TCN, and Transformer) are evaluated under three regimes: single-objective maximization of $R^2$, single-objective minimization of the NASA scoring function, and a multi-objective formulation that jointly optimizes both criteria. The multi-objective search employs NSGA-II with Entropy-CRITIC weighting for Pareto selection. Seventy-five model-dataset-strategy combinations are assessed on the NASA C-MAPSS turbofan and BackBlaze hard-disk drive benchmarks. On C-MAPSS, all strategies achieve comparable accuracy ($R^2 \approx 0.89$), yet multi-objective optimization reduces directional imbalance by approximately 33%, improving calibration of early versus late predictions. Model rankings prove configuration-dependent, with simpler architectures frequently outperforming deeper temporal models. On BackBlaze, the objectives shift from complementary to conflicting, producing divergent Entropy-CRITIC weights and a substantial generalization gap (best $R^2 \approx 0.34$). These results demonstrate that the optimization objective materially shapes prognostic behavior and that multi-objective search provides a practical mechanism for calibrating prediction timeliness in RUL modeling.
0
cs.DCcs.LG Christophe C\'erin, Mamadou Sow, Fr\'ed\'eric Andr\`es · Oct 1, 2026

This paper tackles privacy, energy, and infrastructure constraints in smart buildings by proposing a decentralized cloud-fog-edge architecture that treats the building itself as a data center. Its core idea couples the KOptim container orchestrator with FIWARE for dynamic SLA management, while running lightweight online ML algorithms such as GHA-PCA and K-means directly on Arduino-class microcontrollers. The humanitarian motivation—bringing autonomous AI to underserved communities—is compelling, although the current evaluation remains largely preliminary.

In this article, we present our vision and recent advancements toward creating a decentralized system capable of learning from real-time data within buildings to support sustainable and privacy-preserving smart environments. Our approach promotes the concept of the building itself as the data center, aligning with the principles of edge computing to safeguard confidentiality and reduce reliance on external cloud infrastructure. This is particularly valuable in humanitarian contexts, where data sovereignty, energy efficiency, and infrastructure constraints are critical. We detail a lightweight, "Kubernetes-like" orchestration framework for deploying AI services within such environments and demonstrate our progress in implementing AI algorithms on low-power, cost-effective microcontrollers such as those in the Arduino ecosystem. By enabling in-situ learning directly on sensors or microcontrollers, our work aims to bring intelligent services to resource-limited settings, fostering autonomy, resilience, and sustainable development in vulnerable or underserved communities. The contributions in this article are related, firstly, to our project "Online Machine Learning Algorithms for Embedded Systems" and the evaluation of two new online algorithms. Secondly, we envision a cloud-fog-edge architecture based on the KOptim and FIWARE components, and we propose a methodology for coupling them. Experimental results of the online algorithms are also presented, showcasing real-world traces.
0
cs.LG Abdullah Al Noman, Fahmid Al Rifat, Tahrima Hashem et al. · Oct 1, 2026

This workshop paper examines how fairness assessments for ICU mortality models on MIMIC-IV hinge on both the chosen performance metric and the demographic resolution of evaluation. It introduces a lightweight classifier-head adaptation, IBHFT, that jointly balances ethnicity–gender–insurance representation without conditioning on mortality outcomes, using this case study to show that marginal summaries and single-metric reports can paint a misleading picture of clinical fairness. The core argument matters because deploying models based on aggregate AUROC or one-axis parity alone can obscure dangerous error-rate disparities at clinically consequential intersections.

Fairness conclusions in clinical prediction can depend strongly on both the metrics reported and the demographic resolution at which performance is evaluated. We revisit these evaluation choices for ICU mortality prediction on MIMIC-IV, comparing predictive-utility and subgroup-error metrics across several fairness interventions. As a complementary case study, we introduce a lightweight adaptation strategy that jointly balances ethnicity--gender--insurance representation without conditioning on mortality outcomes, allowing demographic representation balancing to be examined separately from outcome-conditioned or direct error-rate interventions. We evaluate its behavior at both marginal and corresponding three-way intersectional subgroup levels, while accounting for the statistical support of finer-grained estimates. The results show that interventions can receive substantially different assessments across accuracy/AUROC, sensitivity, and false-positive rate, and that marginal demographic summaries can conceal heterogeneous error profiles within their constituent intersections, including among larger subgroups. These findings highlight the importance of evaluating fairness interventions at both complementary metric and subgroup resolutions, while accounting for the intervention target and the reliability of subgroup estimates.
0
cs.LGq-bio.BM Tong Chen, Maximilian Holsman, Lin Zhao et al. · Oct 1, 2026

pCoMole addresses the problem of editing existing biomolecular sequences to improve multiple properties while obeying hard feasibility constraints such as PAM specificity or emission range. Its core idea is to steer a pre-trained Edit Flow via a Doob-$h$ transform, using an augmented Tchebycheff utility over terminal states that is approximated through short Monte Carlo rollouts. This matters because existing shrinkage methods like RayGun and SCISOR optimize length alone, whereas biomolecular engineering inherently requires trading off function, developability, and manufacturability. The work is notable for combining a clean theoretical justification with preliminary wet-lab validation on shortened eGFP variants.

Biomolecular therapeutics often start from known sequences and require targeted editing to improve multiple properties while satisfying hard biochemical and manufacturability constraints. However, existing generative methods do not jointly support multi-objective optimization, hard feasibility, and sequence editing in discrete, variable-length biological spaces. In this work, we introduce Pareto-Constrained Molecule Editing (pCoMole), a framework built on discrete flow matching that steers a pre-trained Edit Flow toward user-specified preferences while enforcing terminal feasibility. pCoMole defines a feasibility-gated terminal distribution using an augmented Tchebycheff utility and realizes the resulting preference tilt through a Doob-h transform of the underlying edit process. To make this construction practical, we approximate the required harmonic function using short Monte Carlo rollouts over candidate edits, yielding an efficient guided editor with provable preference consistency. We validate pCoMole by shrinking GFP while retaining fluorescence-related properties, shortening diverse Cas9 orthologs while preserving PAM specificity, and compressing peptide binders into short peptidomimetics that optimize seven drug-related properties under hard constraints. In wet lab testing, two 229-residue pCoMole-designed eGFP variants retained clear green fluorescence in BL21 cells after 10 deletions, with either one or two substitutions. Together, pCoMole enables constraint-aware, Pareto-aligned editing of biomolecular sequences in discrete, variable-length spaces.
0
cs.LGcs.AIstat.ML Kabir Murjani, Nisarg Patel · Oct 1, 2026

This paper asks whether a fixed offline corpus generated by a behaviour policy can ever distinguish two candidate Regular Decision Processes (RDPs). It proves that when two candidates are $\pi$-observationally equivalent, their posterior odds remain frozen at the prior for any sample size (Theorem 6), gives an exact measure-zero characterization via the product-automaton blame language (Theorem 12), and derives PEC, a linear-time algorithm that certifies equivalence and names the missing experiment. The work matters because existing PAC offline guarantees for RDPs assume a distinguishability condition that was previously impossible to verify.

Non-Markovian environments are often modeled as Regular Decision Processes (RDPs), where dynamics depend on the interaction history through a finite automaton. Existing offline guarantees for RDPs rely on a distinguishability assumption on the behaviour policy but provide no means of verifying it. When the assumption is violated, distinct models may explain the data equally well. We study when data collected under a fixed behaviour policy can distinguish two candidate RDPs. We prove that the posterior odds between observationally equivalent candidates remain equal to the prior odds at every sample size, even when the policy visits every automaton state, and verify both results formally in Lean 4. We then characterize this equivalence exactly and derive PEC, an algorithm that decides it in time linear in the size of the product automaton. The distinguishability assumption of prior work fails on three of our four test environments, and the experiment identified by PEC restores it in each case.
0
stat.MLcond-mat.dis-nncs.LG Jorge Medina Moreira, Lorenzo Bardone, Lenka Zdeborov\'a · Oct 1, 2026

This paper develops a high-dimensional theory for why masked autoencoders learn better representations than standard autoencoders. In a spiked model with heterogeneous noise, the authors prove that a linear masked autoencoder recovers the latent feature at linear sample complexity $n=O(d)$ where PCA fails. They further quantify how dynamic mask resampling lowers sample complexity compared to static masking, and validate the qualitative predictions in CNNs, vision transformers, and a BERT pilot.

Why can masked prediction learn useful representations that unmasked reconstruction misses? We study this question in a high-dimensional model of a masked autoencoder (MAE) trained on data with shared latent structure and heterogeneous noise. We prove that masked linear reconstruction can recover the latent feature at linear sample complexity in regimes where unmasked linear reconstruction, equivalent to PCA, fails. The analysis also quantifies the statistical advantage of mask resampling, an established ingredient of masked pretraining. By introducing a fixed collection of $K$ masks per sample, we characterize its effect on feature recovery and downstream performance, identifying regimes where greater mask diversity lowers sample complexity. Guided by this prediction, we find that random cropping and flipping in standard image-training pipelines can obscure the advantage of mask resampling by renewing the prediction task even when the patch mask is fixed. Removing these transformations reveals a downstream advantage for dynamic over static masking in CNN autoencoders and vision transformers. A complementary BERT pilot finds benefits from greater mask diversity on downstream language tasks. Our results separate the benefit of the masked prediction objective from that of mask diversity, and show how a tractable theory can guide experiments that uncover advantages hidden by standard training practices.
0
math.OCcs.LGstat.ML Jiayi Song, Zi Xu · Oct 1, 2026

This paper proves worst-case lower bounds for stochastic first-order algorithms in nonconvex--concave minimax optimization, explicitly permitting variance-reduction via same-sample oracle queries. Its main result shows that the stochastic barrier remains $\Omega(L^3 D_Y^2 \Delta \sigma^2 \varepsilon^{-6})$ even with mean-square smoothness. Complementary nonconvex--strongly-concave bounds are also provided. Together the results establish that variance reduction does not improve the fundamental accuracy exponent in the minimax setting.

We establish complexity lower bounds for stochastic first-order algorithms in nonconvex--concave minimax optimization, allowing algorithms to use variance reduction. Our main contribution is a lower bound for a zero-respecting algorithm class that permits variance reduction, extending beyond the algorithmic restrictions imposed by some existing lower bounds. We consider objectives with an $L$-Lipschitz continuous joint gradient, a compact convex dual domain of Euclidean radius at most $D_Y$, and a primal value function, defined by maximizing the objective over the dual variable, with initial suboptimality at most $\Delta$. The target accuracy $\varepsilon$ is measured by the gradient norm of the Moreau envelope of the constrained primal value function with parameter $1/(2L)$. Under an unbiased stochastic first-order oracle with variance at most $\sigma^2$ and mean-square smoothness, we prove the lower bound $\Omega\!\left(L^2D_Y\Delta\varepsilon^{-3}+L^3D_Y^2\Delta\sigma^2\varepsilon^{-6}\right)$. This result quantifies the dependence on accuracy, dual-domain radius, and oracle noise even when variance reduction is allowed. We also establish complementary lower bounds for nonconvex--strongly-concave minimax optimization. With dual strong-concavity parameter $\mu>0$ and condition number $\kappa:=L/\mu$, we obtain $\Omega\!\left(L\Delta\sqrt{\kappa}\,\varepsilon^{-2}+L\Delta\kappa\sigma^2\varepsilon^{-4}\right)$ under the bounded-variance oracle model. Under the additional mean-square smoothness condition with constant $\bar L$, we obtain $\Omega\!\left(L\Delta\sqrt{\kappa}\,\varepsilon^{-2}+\Delta\bar L\sigma\kappa^{3/2}\varepsilon^{-3}\right)$. Together, these results identify complexity barriers across the concave and strongly concave regimes, with the main nonconvex--concave bound remaining valid for algorithms that use variance reduction.
0
math.OCcs.LG Wei Jiang, Rui Yan, Sifan Yang et al. · Oct 1, 2026

This paper studies stochastic multi-level compositional optimization, where the objective is a nested composition of $T$ smooth non-convex functions and only stochastic per-layer function and gradient values are available. The core idea is to use momentum-based estimators to track inner function values level-by-level, and then apply the chain rule to build a momentum gradient estimator for the overall objective. It matters because it establishes that the optimal $\mathcal{O}(\epsilon^{-4})$ sample complexity is achievable under standard smoothness assumptions, closing a gap with single-level lower bounds without relying on the stronger average-smoothness conditions common in prior multi-level literature.

This paper investigates stochastic multi-level optimization where the objective is a nested composition of several smooth non-convex functions. We assume that only stochastic estimates of the gradient and function values for each level are accessible. Consequently, obtaining an accurate estimate of the overall gradient is challenging due to the nested structure. To address this, we employ a momentum-based estimator with mini-batches to track the function values of each level, which are subsequently used to construct momentum gradient estimators. We establish an optimal sample complexity of $\mathcal{O}(\epsilon^{-4})$ for finding an $\epsilon$-stationary point, avoiding the stronger average smoothness assumption commonly relied upon in prior literature. Furthermore, by employing a normalization technique, we attain the same rate without requiring problem-dependent constants to set hyperparameters. To achieve the optimal rate without mini-batches, we further develop a batch-free method that incorporates a first-order approximation and a clipping technique for function value estimation. Finally, we validate the effectiveness of our proposed methods through experiments on risk-averse portfolio optimization and hierarchical tilted empirical risk minimization.
0
cs.MAcs.AIcs.LG Andrea Iannoli, Ivan Zyrianoff, Angelo Trotta et al. · Oct 1, 2026

Unmanned aerial vehicle swarms operating with independent edge-deployed language models face a dual challenge: reasoning context degrades as interaction histories grow, and indiscriminate information sharing wastes limited bandwidth. This paper proposes a distributed architecture in which each UAV maintains structured atomic notes in bounded core, local, and peer-specific memory queues, coupled with a deterministic Interest-Aware Gossip Engine that forwards messages based on semantic novelty and recency. The work matters because it targets the practical gap between centralized LLM coordination, which introduces bottlenecks and single points of failure, and fully decentralized control, which risks context degradation and communication collapse.

Unmanned aerial vehicle (UAV) swarms increasingly rely on language-model agents to provide adaptive mission-level reasoning in uncertain environments. Fully distributed control, in which each UAV hosts an independent Small Language Model (SLM), removes reliance on a centralized coordinator but introduces an information-management problem: long-running interaction histories can degrade the reasoning context, while indiscriminate information dissemination increases communication and inference overhead. We address these challenges with a distributed UAV-agent architecture that enables continuous local SLM control through an event-driven reason-act-observe lifecycle. Runtime knowledge is represented as structured atomic notes and organized into core, local, and peer-specific memory. A deterministic interest-aware gossip engine selectively disseminates these notes according to recipient-specific semantic novelty and recency. We evaluate the architecture using ten UAVs in a simulated search-and-rescue mission. Our approach completes all experimental runs, whereas unrestricted flooding messages completes only 70-85\%, and delegating forwarding decisions to the SLM prevents mission completion in every run. Compared with unrestricted flooding, our approach approximately halves inference-token consumption, reduces transmitted data, and achieves lower survivor-count error.
0
cs.LG Wentao Yue, Tianyou Lai, Hongji Li et al. · Oct 1, 2026

Federated learning on edge devices must simultaneously handle varying resource budgets and domain-shifted local data. FedSAP frames this coupled challenge as a structured pruning problem where channels are assigned to Global, Private, or Dropped states. By inferring pseudo-domains from shallow gradient similarity and allocating layer-wise budgets adaptively, the method improves global mean accuracy under aggressive heterogeneous compression.

Federated learning (FL) on heterogeneous edge devices must jointly accommodate unequal resource budgets and domain-shifted local data. Existing resource-adaptive methods decide how much of a model each client trains but not where retained capacity should reside or how it should be shared, whereas federated domain-generalization methods usually assume a shared full architecture. Uniform compression can therefore discard high-utility channels, and a single aggregation path can mix transferable features with domain-sensitive updates. We propose FedSAP, a domain-aware heterogeneous FL framework that casts structured pruning as budget-constrained tri-state channel allocation. FedSAP converts each keep ratio into non-uniform layer budgets, assigns stable channels to a Global pool, useful domain-sensitive channels to pseudo-domain-specific Private pools, and low-utility channels to a Dropped state. This partition lets broadly useful features benefit from cross-client pooling while isolating domain-sensitive updates from incompatible clients. Domain-Guided Assignment infers pseudo-domains from shallow-gradient similarity, while Type-Matched Aggregation restricts each channel to its intended sharing scope. Across three random seeds, FedSAP reaches 76.00% and 72.67% mean global accuracy on Digits and Office-Caltech, exceeding the strongest baseline by 1.70 and 4.92 percentage points while supporting client pruning ratios of up to 80% across heterogeneous clients.
0
cs.LG Sahel Torkamani, Henry Gouk, Rik Sarkar · Oct 1, 2026

Explaining why neural networks generalize and how training unfolds remains a central challenge in deep learning. This paper introduces Magnitude Potential (MP), a point-set coverage measure from metric space theory, and defines the MP ratio $\mathcal{R}^t_{X,X_c}(z)$ as the within-class potential divided by the full-dataset potential at the logit layer. The authors argue this single geometric score predicts per-sample memorization, detects decision-boundary phase transitions, and foreshadows grokking without requiring model retraining or architectural assumptions.

Explaining generalization and training dynamics in neural networks remains a challenge, and various approaches have been developed to study different aspects of these phenomena. In this paper, we introduce the idea of {\em magnitude potential} -- a quantity based on the theory of metric magnitude -- that reflects how well an arbitrary point is represented by a given set. We find that this basic quantity can be applied to examine various features in neural generalization. The ratio between the magnitude potential with respect to a class and with respect to the entire data, computed at the logit layer, is informative of the representation of the point. In experiments, these ratios for individual training points are found to be correlated with the Feldman memorization scores. Magnitude potential ratios aggregated across points detect structural changes in the decision boundaries and provide a geometric indicator of grokking in modular arithmetic. Although the magnitude potential ratio and neural collapse are both closely associated with intra-class and inter-class geometric structure, the magnitude potential ratio remains informative even when neural collapse is explicitly suppressed.
0
cs.LGstat.ML Vladimir R. Kostic, Karim Lounici, H\'el\`ene Halconruy et al. · Oct 1, 2026

LITL addresses the problem of recovering target Langevin dynamics when only biased or static source samples are available, complemented by black-box evaluative feedback. The core idea is to learn the dominant spectral subspace of the target infinitesimal generator and its projected drift via importance-weighted Dirichlet representation learning, explicitly avoiding target trajectories, gradients, or retraining. This matters for molecular dynamics, auditing generative models, and post-hoc steering of learned representations under distribution shift.

Many scientific and machine learning systems, from molecular dynamics to diffusion models and beyond, are governed by stochastic dynamics with low-dimensional structure, evolving on slow timescales. However, target trajectories, used to identify and interpret such dynamics, are often inaccessible: only biased or static samples that explore the underlying manifold are available. We introduce Langevin-Informed Transfer Learning (LITL), a framework for recovering target Langevin dynamics from biased source samples using only black-box feedback. LITL learns the leading spectral structure of the target infinitesimal generator and the projected drift through Dirichlet representation learning, enabling kinetic reconstruction in spectral form and slow-manifold gradient field estimation. We further introduce a spherical variant well suited to steering normalized latent representations commonly used in learning systems toward desired objectives. We establish finite-sample guarantees for eigenvalue, eigenfunction, and projected drift estimation in Sobolev norms, thereby ensuring generalization of these quantities and their first-order derivatives. Empirically, LITL recovers physical transition timescales from biased molecular simulations, builds kinetic structure from static samples of generative models, reconstructs spherical symmetries of physical systems, and enables post-hoc latent steering of trained neural networks under black-box feedback. Together, these results position spectral operator learning as a practical framework for recovering stochastic dynamics under distribution shift and unlock applications across machine learning and the physical sciences.
0
cs.LGcs.AIcs.SD Constance Douwes, Paul Magron, Romain Serizel · Oct 1, 2026

Deep learning audio research hides a massive energy footprint behind the final training run. This paper develops a methodology to estimate the full development-phase cost of audio DL projects by mining job logs from the Grid5000 shared computing platform. By tracing thousands of individual jobs across four representative projects, the authors show that prototyping, debugging, and hyper-parameter search can consume $3 imes$ to $256 imes$ more energy than training the reported best models, calling for systematic lifecycle reporting.

The environmental impact of deep learning has attracted increasing attention over the past decade. Existing studies mainly focus on the energy and carbon emissions of model training and inference, while the whole development phase is often overlooked. Yet, architecture prototyping and intensive experiments are conducted during this stage, which is highly energy-demanding. In this article, we propose a methodology to estimate these costs, based on activity logs from the Grid5000 shared computing platform used by the LORIA laboratory. As a case-study, we focus on audio projects developed in the Multispeech research team. We evaluate the overall energy cost of four projects, and we compare them to those of training the reported models. Our results show that the energy required for the development phase is 3 to 256 times greater than that required to train the best-performing model alone. These results advocate for a more systematic reporting of energy consumption across the entire life cycle of deep learning-based audio projects.
0
cs.CVcs.AIcs.LG Yuzhou Wang, Emile Anand, Ijay Narang · Oct 1, 2026

Hob-VL introduces a diagnostic benchmark for visually grounded Boolean reasoning, asking vision-language models to evaluate ten-literal Boolean combinations over images and to identify objects satisfying Boolean descriptions. The core idea is to pair each expression with a mechanically verified De Morgan equivalent and to test both symbolic and natural-language presentations, isolating compositional reasoning from surface memorization. It matters because real-world visual queries—warehouse checks, agricultural monitoring, medical imaging—depend on reliably combining multiple visual facts under logical constraints.

Reliable visual reasoning requires composing multiple visual observations and returning consistent answers to logically equivalent questions. We introduce Hob-VL, a benchmark for visually grounded Boolean reasoning. Hob-VL comprises two tasks: (1) evaluating whether a Boolean rule holds in an image, and (2) identifying the (unique) object satisfying a Boolean description. Hob-VL contains 6,000 human-verified balanced Yes/No questions, each defined by a Boolean combination of ten visual statements, across 1,000 generated scenes and 46 diverse labeled photographs, along with 1,000 object-identification questions over the same photographs. Our question families are deliberately constructed to challenge reasoning through misleading local cues and nested logical operations, and include symbolic and structured natural-language presentations. Across eight model configurations with thinking disabled or minimized, Boolean accuracy ranges from 48.52% to 50.57%, while the identification accuracy reaches at most 43.0%. A thinking-enabled GLM configuration achieves uneven gains while retaining substantial errors and inconsistencies. Hob-VL exposes these failures through executable reference answers and matched evaluations.
0
cs.LGcs.AI Guy Amit · Oct 1, 2026

Decision models that score a variable-sized set of candidate actions from a serialized sequence are vulnerable to permutation sensitivity because standard causal attention lets later candidates attend to earlier ones and uses absolute positions. The paper proposes candidate-independent block-causal attention (CI), which blocks cross-candidate information flow and resets per-candidate positions so that each candidate is scored based only on the shared prefix and its own tokens. This matters for System 1 components inside generative systems, where runtime-dependent candidate order should not become a latent feature; the authors show that CI cuts flip rates by an order of magnitude across three backbones while preserving competitive accuracy.

Decision models often score a variable-sized set of candidate actions encoded in a single sequence. This setting is increasingly relevant for System 1 components inside generative systems, where candidates may be proposed or ordered differently across runs. Standard causal cross-encoding is expressive, but it can make a candidate's score depend on serialization order rather than on the underlying decision problem. We introduce candidate-independent block-causal attention, which preserves causal computation within the shared context and each candidate while blocking cross-candidate information flow and resetting candidate positions. We compare this architecture with standard causal attention and complementary invariant baselines across Gemma 3 1B, Qwen3 1.7B, and Qwen3 4B backbones. Candidate-independent attention consistently reduces permutation sensitivity while retaining competitive decision quality; ablations indicate that candidate isolation is the primary source of the effect, with position resetting completing the intended symmetry. A larger Qwen3-4B study further examines the behavior of the proposed architecture with substantially more training data. Code is available at the \href{https://github.com/guyAmit/ci-decision-models}{\textcolor{blue}{project repository}}, and the \href{https://huggingface.co/Guy-Amit/qwen3-4b-ci-decision-4096-poc}{\textcolor{blue}{Qwen3-4B model artifact}} is available on Hugging Face.
0
cs.LG Yaniv Oren, Viliam Vadocz, Wiktor Zabka et al. · Oct 1, 2026

Iterative policy improvement is central to practical RL, but approximate value estimates make greedification unreliable. This paper defines optimal policy improvement restricted to a state subset as solving an induced MDP, and derives $\mathcal{I}_{\text{opt}}$, a novel greedification operator that formulates approximate evaluation as probabilistic decision-making under uncertainty. It weights advantages by action-dependent uncertainty, generalizes popular regularized operators, and improves aggregate performance across model-based and model-free, discrete and continuous, and online and offline settings.

Practical Reinforcement Learning (RL) algorithms learn to solve Markov Decision Processes (MDPs) through iterative policy improvement in the presence of approximate evaluation. We study policy improvement from first principles, defining optimal policy improvement as producing the best policy attainable in a single update under specified constraints. We show that optimal improvement restricted to a set of states is equivalent to solving an induced MDP, characterizing planning with an explicit or implicit model as a path towards optimal policy improvement. Because practical methods commonly solve such induced problems through iterative improvement in the form of greedification, we take steps towards optimal greedification under the central practical constraint of approximate evaluation. We formulate greedification under this constraint as probabilistic decision-making under uncertainty and derive a novel operator that is optimal with respect to the resulting objective. Empirically, the operator and its practical gradient-based approximations improve aggregate performance across GumbelAlphaZero, SAC, ReBRAC and Generalized Policy Iteration, in experiments spanning discrete and continuous actions, model-based and model-free, online and offline RL.
0
cs.LGcs.AI Alessandro Micheli, Andrea Zerio, Samir Bhatt · Oct 1, 2026

This paper extends one-step generative modeling to finite discrete spaces by replacing Euclidean drift with a discrete Wasserstein gradient flow. Using Maas geometry on a reversible Markov kernel, it defines a relative-KL flow that is realized at the particle level by Markov jumps and amortized into a neural generator. The result is a theoretically grounded analogue of continuous drifting that preserves one-step inference. The work is validated only on a small controlled torus, leaving real-world scalability as open future work.

We introduce a new framework for one-step generative modelling on finite state spaces. To extend drifting beyond continuous domains, we use discrete Wasserstein geometry to define a target-relative KL gradient flow over the transitions of a reversible Markov kernel. We realize this probability flow at the particle level through Markov jumps and amortize the resulting transport updates into a latent-conditioned generator, so that the iterative dynamics are required only during training while inference remains one-step. In a controlled setting where the underlying distributions and transport dynamics can be computed exactly, we verify KL dissipation, consistency between the particle dynamics and the probability flow, and the predicted numerical scaling. We further show that a finite-capacity neural generator can track these exact transport targets while retaining one-step generation. These results validate the basic construction and provide a foundation for scaling Discrete Drifting to structured discrete data.
0
cs.CLcs.LGcs.SD Yuxiang Wang, Kunyu Feng, Yuancheng Wang et al. · Oct 1, 2026

AURAL tackles the tension between reasoning quality and low latency in speech language models by replacing explicit chain-of-thought with adaptive latent reasoning. Instead of generating slow text intermediates, the model predicts chunks of continuous latent states drawn from a Gaussian mixture, cutting time to the first token by more than 11x while largely matching explicit CoT accuracy across two backbones. The work also constructs AuralReason-683K, a bilingual corpus of 683K utterances with concise reasoning traces used to initialize latent training.

Model intelligence and fast response jointly shape the quality of interaction with speech language models, yet remain difficult to achieve together. Explicit chain-of-thought (CoT) improves reasoning and audio understanding, but generating intermediate reasoning tokens delays responses. Describing fine-grained acoustic cues further lengthens CoT and increases latency. Latent reasoning can reduce this overhead, yet existing methods often trail CoT and remain limited by single-path supervision and reasoning budgets that do not adapt to problem difficulty. We introduce AURAL, which models a distribution over multiple plausible reasoning continuations in latent space and jointly predicts chunks of future states to reduce sequential forward passes and reasoning latency. To provide initial supervision for latent reasoning, we construct AuralReason-683K: 683K bilingual speech utterances (about 1,000 hours) with concise CoT for emotion recognition, empathetic dialogue, and general reasoning. AURAL-RL then explores beyond these traces, rewarding concise reasoning that yields high-quality answers and adapting reasoning effort to each problem. Across two backbones, AURAL-RL achieves performance comparable to CoT-RL, with larger gains over the respective supervised checkpoints on most metrics. Analysis further shows that harder questions elicit more latent reasoning steps. On Qwen2.5-Omni, it reduces time to the first answer token by 11.8x, from 1.22 to 0.10 s, versus 0.05 s for direct answering.