Your paper timeline
Scroll AI takes the way you would scroll a great paper aggregator: quick signal first, deeper critique when something earns your attention, and challenges when a claim feels off.
1579 papers
Trending mixes fresh papers with community signal.
0
cs.LG Jan Tauberschmidt, Jephte Abijuru, Samuel Okon et al. · Oct 1, 2026

Many physical systems are driven and observed only on lower-dimensional submanifolds embedded in a larger ambient domain, forcing classical solvers to waste computation on full volumetric states and preventing black-box surrogates from reusing a fixed medium representation. This paper introduces the Green’s Observation Operator (GObO), which learns a time-dependent Green’s kernel restricted to the source and observation submanifolds so that, after a single conditioning pass on the ambient medium, evaluating an entirely new source requires only a cheap lower-dimensional integral with no additional network inference. The idea is supported by exponential spectral approximation guarantees, an exact finite streaming state, and strong empirical speedups and source-generalization gains on three-dimensional heat and advection–diffusion problems.

Many physical systems are driven and observed only on lower-dimensional submanifolds of a larger spatial domain, while their dynamics are governed by the ambient medium occupying that domain. Examples include laser-heated parts imaged by an infrared camera, and ground-level emissions measured on a sensor plane. Full-domain solvers, however, compute the entire volume for every new source although only the observation submanifold is needed, and black-box surrogates do not exploit that the ambient medium remains fixed. We introduce the \emph{Green's Observation Operator (GObO)}, which maps the ambient medium once to the Green's kernel of a linear PDE restricted to the source and observation submanifolds. New sources then cost one lower-dimensional integral and no network evaluation. Exponential rates in the kernel yield an exact finite streaming state with horizon-independent memory; we prove its stability and an approximation rate for the restricted heat kernel. On three-dimensional heat conduction and advection--diffusion with collocated and distinct source and observation geometries, GObO trained on static sources predicts responses to moving sources zero-shot with 4--8$\times$ lower error than black-box surrogates, at 1.4\,ms per query after a single conditioning pass. The same kernel transfers across resolutions and admits corrections for mild nonlinearities, including radiative losses and temperature-dependent conductivity, without retraining, at the cost of lower in-distribution accuracy.
0
cs.LGcs.AImath.OC Jakob Paul Zimmermann, Moritz Grillo, Andrei Balakin et al. · Oct 1, 2026

Understanding when neural-network loss landscapes contain spurious local minima is a foundational problem in deep learning. This paper studies shallow bias-free ReLU networks in a Gaussian teacher-student setting and proves that a learned linear skip connection eliminates all spurious local minima for planar teacher features with positive output weights once the student is at least as wide as the teacher. In contrast, without the skip, spurious minima can persist at every width—a striking separation between architectural bias and overparameterization.

Understanding loss landscapes is central to explaining neural-network training, yet their structure remains only partially understood even in simple models. We study the Gaussian population loss of shallow, bias-free ReLU networks in the teacher--student setting. This provides a simple model for studying essential aspects such as feature learning and overparameterization. For teacher networks with positive output weights and planar features, we show that including a learned linear skip removes all spurious local minima with non-negative student output weights once the student network is at least as wide as the teacher network. In contrast, without the skip, we construct a fixed teacher network with positive output weights and only three hidden neurons in input dimension two whose spurious local minima persist at every student width at least three. Thus, a learned linear skip can remove spurious minima that persist under arbitrary overparameterization. Furthermore, we show that a positive output weight student network always learns the subspace spanned by the teacher features: student features at local minima with non-negative student output weights lie in the span of the teacher features. For ReLU networks in two dimensions, even heavily overparameterized student networks have effective width controlled by the teacher width: every critical point with positive student output weights has at most twice as many distinct student feature directions as teacher neurons. Finally, we transfer the benignity result to empirical minima over parameter balls of any prescribed radius, with the required sampling accuracy depending on that radius.
0
cs.LGeess.SP Haochen Chai, Xinbi Luo, Zining Liu et al. · Oct 1, 2026

Wrist electrodermal activity differs in amplitude from person to person, so affect-recognition models normalize the signal before classification. A hidden protocol choice is whether the normalization statistics come from training subjects only or from the held-out user's own complete recording, which retroactively gives the model a calibration that a newly worn device lacks. The paper introduces SAFE-EDA, a compact convolutional network pretrained on expert artifact annotations, and shows that its measured benefit over training from scratch ranges from 0.020 to 0.227 depending on this normalization choice, with the larger gains occurring in the realistic first-use setting. The authors also survey fifty published WESAD studies and find only ten percent report the source of their normalization statistics, making published scores difficult to compare.

Wrist electrodermal activity (EDA) differs in amplitude from one person to the next, so affect-recognition models normalize their input before classification. Studies that test such models on held-out subjects seldom report where the normalization statistics come from, yet statistics computed from the held-out subject's own recording give the model information that a device does not have when it is first worn. We asked how this choice alters the measured benefit of pretraining. A compact convolutional network, SAFE-EDA, was pretrained on expert artifact annotations from 43 subjects and compared with the same network trained from scratch on the Wearable Stress and Affect Detection (WESAD) dataset (15 subjects, leave-one-subject-out), with two normalization sources crossed with four window hops. When the statistics came only from training subjects, pretraining raised macro-F1 by 0.078 to 0.227; when they came from the held-out user's full recording, the gain fell to between 0.020 and 0.050 and was no longer significant. Artifact supervision was far more useful than self-supervised pretraining on the same recordings (0.078 versus 0.008). Across 13 configurations in two datasets, the pretrained network was better in 12, but on the second dataset (26 subjects) per-user normalization increased the gain instead of reducing it, so the interaction depends on the data. Only five of 50 published WESAD studies state which data were used for normalization. Reporting this choice is necessary to separate first-use performance from performance after calibration.
0
cs.LG Ryunyi Lee, Kangjun Noh, Somin Kim et al. · Oct 1, 2026

Range-GRPO addresses semi-supervised LLM post-training when labeled data is scarce by treating LLM-as-a-Judge scores as conformally calibrated reward intervals instead of scalars. Its core idea is to replace standard GRPO pointwise comparisons with pairwise comparisons of interval midpoints, weighting each pair by $\omega_{ij}=\lambda/(h_i+h_j+\lambda)$ so that uncertain rewards can actually flip the direction of the learning signal. When all intervals collapse to points ($h_i=0$), the method exactly recovers Dr.GRPO, and experiments suggest it improves both in-distribution and out-of-distribution reasoning with a smaller compute budget.

As the use of large language models (LLMs) expands, post-training has become increasingly important for adapting them to downstream tasks. However, obtaining reliable supervision remains costly, especially in domains without reference answers or executable verifiers. LLM-as-a-Judge provides scalable pseudo-rewards for unlabeled responses, but a single point score does not explicitly represent reward uncertainty. This motivates representing pseudo-rewards as conformally calibrated reward ranges. We propose Range-GRPO, a semi-supervised post-training framework that combines limited labeled data with unlabeled prompts. In Group Relative Policy Optimization (GRPO), learning signals depend on relative reward comparisons within each rollout group. The proposed objective compares reward ranges pairwise rather than reducing them to point rewards, allowing interval uncertainty to affect both the magnitude and direction of these signals. Our theoretical analysis characterizes this distinction and shows that the proposed objective recovers the Dr.GRPO advantage when all reward ranges collapse to points. Empirically, Range-GRPO achieves the highest in-distribution and out-of-distribution average performance among the evaluated semi-supervised methods while requiring fewer training resources.
0
cs.LG Yihao Ding, Daniel Yitian Su, Yiran Zhang et al. · Oct 1, 2026

The paper tackles autoregressive modelling of mineral-exploration drillholes, where lithology is revealed sequentially from shallow to deep. It introduces DrillBench, a 49,671-hole benchmark across Western Australia with a graded transfer spectrum from local prediction to cross-province generalisation, evaluating next-layer prediction (macro-F1) and free-running generation (joint $\mathrm{KL}(\text{true}\|\text{generated})$). The core finding is a stark transfer boundary: spatial and geochemical conditioning boosts local accuracy but collapses under shift, while sequence-only autoregressive models degrade more gracefully. To address this, the authors propose a backbone-agnostic recipe combining large-scale pretraining on historical archives with spatial retrieval of neighbouring lithology, showing that retrieval aids local cover prediction and pretraining aids bedrock and cross-province transfer.

Autoregressive modelling has achieved remarkable success in language and sequence tasks by learning to predict future states from previous observation. Mineral-exploration drillholes provide a natural but largely unexplored setting for this paradigm: as drilling proceeds, lithology is revealed sequentially from shallow to deep, making prediction of deeper strata inherently autoregressive. Existing drillhole modelling, however, is dominated by spatial interpolation and reconstruction, or largely rely on masked modelling, leaving strictly autoregressive prediction largely underexplored. We introduce DrillBench, a benchmark of 49,671 Western Australian drillholes for next-layer prediction and autoregressive stratigraphic generation across a graded transfer spectrum, from local prediction through spatial shift to cross geological province transfer. Benchmarking classical, geostatistical, and neural models reveals a clear \emph{transfer boundary}: spatial and geochemical conditioning provides large local gains but deteriorates sharply under stronger shift, whereas lithology-sequence autoregressive models transfer more robustly. Guided by this finding, we develop a backbone-agnostic recipe combining large-scale pretraining on historical drillholes with spatial retrieval of neighbouring lithology. Retrieval is most effective in weathered cover, when local spatial continuity remains informative, whereas pretraining contributes more strongly in bedrock and under broader geological shift. Together, they retain strong local performance while improving generalisation under spatial and cross-province shift, most markedly on the most distant splits. The benchmark and code are available at https://github.com/yihaoding/drillbench.
0
cs.LGcs.AI Feiyu Gavin Zhu, Qi Xu, Zhifei Deng et al. · Oct 1, 2026

Existing methods for structured imitation learning rely on extensive human input or static LLM knowledge that can conflict with expert demonstrations. This paper proposes a closed-loop framework that treats policy development like software debugging: an LLM writes diagnostic analysis code to test semantic latents from tabular rollouts, identifies structure-demonstration misalignment, and iteratively edits the symbolic policy structure before re-fitting continuous parameters through imitation learning. The work matters because it automates refinement of interpretable policies and shows on two control tasks that it can outperform zero-shot LLM structures while using far less subsequent RL compute.

Structured policies improve efficiency, robustness, and interpretability in imitation learning by introducing task-specific inductive bias, but existing structure generation methods rely either on extensive human input or on static domain knowledge encoded in LLMs, which may be inconsistent with the expert demonstrations. We propose a closed-loop framework that iteratively refines structured policies using LLM-guided analysis of policy rollouts. By logging rollouts as semantically meaningful tabular data and prompting the LLM to generate diagnostic analysis code, our method identifies suboptimalities in the policy structure and iteratively corrects them without requiring human instruction. Experiments on car racing and door opening tasks show that our approach improves imitation learning performance by up to 15% over zero-shot LLM-generated structures and requires 75% less compute to achieve the same reinforcement learning performance. These results demonstrate that tabular rollout analysis provides an effective feedback signal to align LLM-generated policy structures with expert demonstrations, and we can utilize it to generate good policy structures automatically.
0
math.OCcs.AIcs.LG Jinhwan Sul, Alex Oshin, Evangelos A. Theodorou · Oct 1, 2026

Primal-dual hybrid gradient methods can solve large linear programs on GPUs, but their speed depends on delicately coordinated step sizes, restarts, and acceleration heuristics. GALLOP replaces these hand-tuned rules with a reinforcement learning policy that jointly picks continuous parameters and discrete restart decisions inside a generalized PDHG update. The approach is notable because the learned policy is dimension-agnostic and reportedly generalizes from small training instances to LPs hundreds of times larger without retraining.

Primal-dual hybrid gradient (PDHG) methods solve large-scale linear programs (LPs) using GPU-friendly matrix-vector products and projections, but their practical performance depends on coordinating algorithm parameters, acceleration, and restarts. We introduce GALLOP, which uses reinforcement learning to jointly learn continuous algorithm parameters and discrete restart decisions without differentiating through the solver. Its generalized accelerated PDHG update combines separate primal and dual extrapolation, history corrections, and restart anchoring with independently adjustable coefficients. We train a dimension-agnostic feedback policy using a groupwise proximal policy optimization objective that clips likelihood ratios separately for different control groups and excludes inactive acceleration controls on restart transitions. We evaluate GALLOP on six LP families and a public item-placement benchmark. On the main evaluation settings across the six families, GALLOP reduces iteration counts by factors of $1.9$-$5.6$ and achieves up to a $16.0\times$ speedup in algorithm wall-clock time over MPAX. With one policy trained per family, the learned policies generalize without retraining to within-family LPs $3\times$-$400\times$ larger than the largest training instances, including Transport LPs with $10.24$ million variables.
0
cs.LGmath.OC Rim Hajal, Mathieu Besan\c{c}on, J\'er\^ome Malick · Oct 1, 2026

Low-budget active learning seeks a tiny labeled subset—a coreset—that still trains an accurate model, which is crucial in medical domains where labeling is expensive. The authors propose using the Sinkhorn divergence, an entropic optimal transport distance, as a coreset quality criterion, and develop a Frank–Wolfe warm-started swap heuristic (FW-Swap) to optimize it. Their framework yields dimension-free generalization guarantees and shows competitive or superior accuracy on vision and EEG benchmarks with very small budgets.

We consider low-budget active learning, which consists of selecting a limited number of points, the coreset, such that a model can be trained to high accuracy on the selection only. This problem is particularly relevant in contexts where labeling requires costly expert intervention, as in medical applications. We leverage features extracted from a pretrained self-supervised model to represent the data, and perform coreset selection directly in this feature space. In this paper, we use entropic optimal transport, specifically the Sinkhorn divergence, as the coreset selection criterion, which first allows us to get dimension-free sample complexity results, and second admits computationally efficient gradient evaluations. This opens the way to using gradient-based algorithms to rapidly compute solution candidates, further improved by a swap-based local search, with guarantees on the solution quality. Experiments on image benchmarks and medical datasets show that our method outperforms state-of-the-art heuristics in low-budget settings.
0
cs.LG Changliang Zhou, Yuanyao Chen, Rongsheng Chen et al. · Oct 1, 2026

MiLoop addresses a key inefficiency in constructive neural combinatorial optimization: dynamic-embedding methods typically rebuild subproblem representations from scratch at each step with deep attention stacks, and often depend on external solution labels or aggressive training-time pruning. The paper proposes Memory-in-the-Loop (MiLoop), an autoregressive framework that propagates a global node-wise memory matrix across construction steps, fusing current inputs with historical context before shallow attention layers and updating memory selectively via adaptive gates. By leveraging the multi-step rollouts already required for RL, MiLoop enables a three-layer policy to learn effective dynamic embeddings without external labels. Strong results on four COPs from 100 to 10 million nodes demonstrate both high solution quality and cross-scale generalization.

Constructive neural combinatorial optimization (NCO) has emerged as a promising paradigm that learns to construct solutions to combinatorial optimization problems (COPs) step by step, which reduces reliance on handcrafted rules and enables fast inference. While many methods with dynamic embeddings generalize well, they typically rebuild subproblem representations from scratch at each step using deep attention stacks. Many high-performing methods in this category rely on solution labels or pseudo-labels for efficient training, or on aggressive search space pruning during reinforcement learning (RL). To address these limitations, we propose Memory-in-the-Loop (MiLoop), a purely RL-based constructive framework that leverages the multi-step computation already required by a rollout for selective memory propagation. Each rollout provides solution-quality feedback for learning while propagating historical representations, thereby enabling a shallow policy to learn effective dynamic embeddings without external solution labels or training-time search-space pruning. Specifically, MiLoop fuses current embeddings with historical memory before the attention layers and applies adaptive gated updates afterward. The updated representations support both current decisions and stepwise reuse. Extensive experiments across four COPs demonstrate that MiLoop consistently produces high-quality solutions on instances ranging from 100 to 10 million nodes, highlighting its strong generalization ability.
0
cs.LG Adir Dayan, Yam Eitan, Haggai Maron · Oct 1, 2026

Cross-architecture weight-space transformations map trained networks into different architectures (e.g., compression, upscaling), yet prior equivariant weight-space models only handle fixed architectures. CrossGMN addresses this by treating the task as refining a target initialization using a source network, making equivariance well-defined: invariant to source permutations and equivariant to target permutations. The authors prove universality for continuous cross-architecture operators and show strong empirical speedups for knowledge distillation, especially on implicit neural representations.

Weight-space networks operate directly on parameters of other neural networks, enabling tasks such as predicting model properties, editing trained models, and generating weights. Weight-space symmetries such as neuron permutations make equivariance a key design principle. However, existing equivariant weight-space architectures have primarily been studied for transformations that preserve the network architecture. In contrast, many practical transformations, including model compression and upscaling, map a trained source network into a target network with a different architecture. In this setting, the source and target permutation symmetries act on different parameter spaces, making equivariance less straightforward to formulate. Our key idea for addressing this mismatch is to reformulate cross-architecture operators with two inputs: a trained source network and an initialization of the target network. This lets us define equivariant cross-architecture operators that refine the initialization of the target network using information from the source network, while being invariant to source-network permutations and equivariant to target-network permutations. Based on this formulation, we introduce CrossGMN, a graph metanetwork that jointly processes both networks through symmetry-preserving cross-network message passing. We prove CrossGMN is universal for continuous cross-architecture operators on compact sets under a general-position assumption. We evaluate CrossGMN for model compression, predicting a smaller network's parameters to accelerate subsequent knowledge distillation. Across 2-D and 3-D INRs and image classification with MLPs, CNNs, and Vision Transformers, CrossGMN speeds up distillation by up to 8.89x, transfers across datasets without retraining (3.78x), and a single model can accelerate compression from heterogeneous source architectures into a common target architecture.
0
cs.AIcs.IRcs.LG Aleksei Medvedev, Alejandro Ariza-Casabona, Steven Derby et al. · Oct 1, 2026

Generative recommenders represent items as short discrete Semantic IDs (SIDs) that are decoded autoregressively, yet most prior work derives these codes by quantising isolated content embeddings. This paper argues that SID construction is fundamentally a recursive clustering problem over a collaborative item graph, and proposes GrIS, a framework that separates graph construction from hierarchical partition to balance semantic and behavioural signals. By exposing these two axes, the authors show that content-only tokenisers are merely empty-graph instances of a broader design space, and they demonstrate new instantiations that improve SID quality for downstream recommendation.

Existing work on Semantic IDs (SIDs) for generative recommendation treats SID construction as a representation learning problem: encode items into a quantised latent space and read off codes. We argue this view is incidental. SID construction is, at heart, a recursive clustering problem, and once stated this way the natural object to cluster is a graph whose nodes carry semantic content and whose edges carry collaborative signal; SID assignment becomes a hierarchical graph partition. This reframing yields a unified framework, Graph-Informed Semantic IDs (GrIS), that subsumes prior approaches rather than displacing them. RQ-VAE and RQ-KMeans are recovered as the special case where the graph is empty, exposing content-only quantisation as one corner of a larger design space along two so-far-collapsed axes: graph construction and recursive partition algorithm. We explore two contrasting instantiations: RecDMoN, which performs hierarchical assignment via differentiable graph pooling, and RQ-GAE, which extends RQ-VAE with graph-aware item representations and a graph reconstruction objective. On multiple real-world datasets, GrIS consistently improves over CF-aware SOTA, with gains of up to +52\% Hit@10. Because graph construction and partition are explicit, separately configurable components, improvements on either axis can be combined and evaluated systematically.
0
cs.CVcs.LG Yuan Huang, Zirui Song, Xiuying Chen · Oct 1, 2026

The paper tackles a critical problem in automated image-editing evaluation: multimodal large language models (MLLMs) acting as judges may be swayed by irrelevant presentation cues rather than the edit itself. Because visual interventions can alter the very quality being assessed, the authors introduce EditJudgeBias, a counterfactual benchmark of 1,196 real samples where 13 cues are injected across four evaluation stages and rigorously verified to preserve underlying quality. They audit five judges along three dimensions—invariance to cues, agreement with humans, and pairwise stability—and show that every judge shifts beyond its own noise floor, with fabricated social proof inflating scores and display-order swaps reversing up to 60.9% of verdicts.

Multimodal large language models (MLLMs) are increasingly used as automated judges for instruction-based image editing and as reward signals for model training. However, systematically auditing whether these judges are influenced by cues irrelevant to editing quality is challenging because visual interventions may themselves alter the quality being evaluated. A judgment shift can therefore be attributed to bias only when the intervention is verified to preserve the underlying editing quality. To address this challenge, we introduce EditJudgeBias, a counterfactual benchmark with verified quality preservation, comprising 1,196 real editing samples and 13 cues injected across four evaluation sites. We verify quality preservation for the requested edit using calibrated multimodal validators, controls, and human inspection. We then audit five MLLM judges along three complementary dimensions: invariance to quality-preserving cues, agreement with human judgments, and stability of pairwise preferences. Importantly, observed shifts are evaluated against each judge's own zero-dose and re-query noise floors rather than against zero. Experiments show that quality-preserving cues move every judge beyond its own noise. Fabricated majority opinions increase ratings, irrelevant visual elements cause larger shifts than whole-image manipulations, and swapping candidate order reverses up to 60.9% of pairwise decisions. Edit-region cues also tend to reduce human agreement. The three measures characterize judges differently, showing that robustness cannot be captured by a single metric.
0
cs.LG Tugrul Cabir Hakyemez, Ener Uras Gokhan · Oct 1, 2026

Predictive maintenance models are typically tuned to maximize $R^2$, a symmetric metric that ignores the fact that late remaining useful life (RUL) predictions are far more costly than early ones. This study treats the optimization objective itself as a variable, comparing single-objective and multi-objective hyperparameter optimization across MLP, LSTM, XGBoost, TCN, and Transformer on NASA C-MAPSS and BackBlaze benchmarks. The core result is that NSGA-II with Entropy–CRITIC selection cuts directional imbalance by roughly a third on simulated data without sacrificing accuracy, though this advantage collapses into near-random behavior on real-world hard-drive data.

In predictive maintenance, early and late RUL prediction errors carry asymmetric consequences, yet hyperparameter optimization typically targets a single accuracy metric that treats both directions equally. This study treats the optimization objective itself as a design variable. Five architectures (MLP, LSTM, XGBoost, TCN, and Transformer) are evaluated under three regimes: single-objective maximization of $R^2$, single-objective minimization of the NASA scoring function, and a multi-objective formulation that jointly optimizes both criteria. The multi-objective search employs NSGA-II with Entropy-CRITIC weighting for Pareto selection. Seventy-five model-dataset-strategy combinations are assessed on the NASA C-MAPSS turbofan and BackBlaze hard-disk drive benchmarks. On C-MAPSS, all strategies achieve comparable accuracy ($R^2 \approx 0.89$), yet multi-objective optimization reduces directional imbalance by approximately 33%, improving calibration of early versus late predictions. Model rankings prove configuration-dependent, with simpler architectures frequently outperforming deeper temporal models. On BackBlaze, the objectives shift from complementary to conflicting, producing divergent Entropy-CRITIC weights and a substantial generalization gap (best $R^2 \approx 0.34$). These results demonstrate that the optimization objective materially shapes prognostic behavior and that multi-objective search provides a practical mechanism for calibrating prediction timeliness in RUL modeling.
0
cs.DCcs.LG Christophe C\'erin, Mamadou Sow, Fr\'ed\'eric Andr\`es · Oct 1, 2026

This paper tackles privacy, energy, and infrastructure constraints in smart buildings by proposing a decentralized cloud-fog-edge architecture that treats the building itself as a data center. Its core idea couples the KOptim container orchestrator with FIWARE for dynamic SLA management, while running lightweight online ML algorithms such as GHA-PCA and K-means directly on Arduino-class microcontrollers. The humanitarian motivation—bringing autonomous AI to underserved communities—is compelling, although the current evaluation remains largely preliminary.

In this article, we present our vision and recent advancements toward creating a decentralized system capable of learning from real-time data within buildings to support sustainable and privacy-preserving smart environments. Our approach promotes the concept of the building itself as the data center, aligning with the principles of edge computing to safeguard confidentiality and reduce reliance on external cloud infrastructure. This is particularly valuable in humanitarian contexts, where data sovereignty, energy efficiency, and infrastructure constraints are critical. We detail a lightweight, "Kubernetes-like" orchestration framework for deploying AI services within such environments and demonstrate our progress in implementing AI algorithms on low-power, cost-effective microcontrollers such as those in the Arduino ecosystem. By enabling in-situ learning directly on sensors or microcontrollers, our work aims to bring intelligent services to resource-limited settings, fostering autonomy, resilience, and sustainable development in vulnerable or underserved communities. The contributions in this article are related, firstly, to our project "Online Machine Learning Algorithms for Embedded Systems" and the evaluation of two new online algorithms. Secondly, we envision a cloud-fog-edge architecture based on the KOptim and FIWARE components, and we propose a methodology for coupling them. Experimental results of the online algorithms are also presented, showcasing real-world traces.
0
cs.LG Abdullah Al Noman, Fahmid Al Rifat, Tahrima Hashem et al. · Oct 1, 2026

This workshop paper examines how fairness assessments for ICU mortality models on MIMIC-IV hinge on both the chosen performance metric and the demographic resolution of evaluation. It introduces a lightweight classifier-head adaptation, IBHFT, that jointly balances ethnicity–gender–insurance representation without conditioning on mortality outcomes, using this case study to show that marginal summaries and single-metric reports can paint a misleading picture of clinical fairness. The core argument matters because deploying models based on aggregate AUROC or one-axis parity alone can obscure dangerous error-rate disparities at clinically consequential intersections.

Fairness conclusions in clinical prediction can depend strongly on both the metrics reported and the demographic resolution at which performance is evaluated. We revisit these evaluation choices for ICU mortality prediction on MIMIC-IV, comparing predictive-utility and subgroup-error metrics across several fairness interventions. As a complementary case study, we introduce a lightweight adaptation strategy that jointly balances ethnicity--gender--insurance representation without conditioning on mortality outcomes, allowing demographic representation balancing to be examined separately from outcome-conditioned or direct error-rate interventions. We evaluate its behavior at both marginal and corresponding three-way intersectional subgroup levels, while accounting for the statistical support of finer-grained estimates. The results show that interventions can receive substantially different assessments across accuracy/AUROC, sensitivity, and false-positive rate, and that marginal demographic summaries can conceal heterogeneous error profiles within their constituent intersections, including among larger subgroups. These findings highlight the importance of evaluating fairness interventions at both complementary metric and subgroup resolutions, while accounting for the intervention target and the reliability of subgroup estimates.
0
cs.LGq-bio.BM Tong Chen, Maximilian Holsman, Lin Zhao et al. · Oct 1, 2026

pCoMole addresses the problem of editing existing biomolecular sequences to improve multiple properties while obeying hard feasibility constraints such as PAM specificity or emission range. Its core idea is to steer a pre-trained Edit Flow via a Doob-$h$ transform, using an augmented Tchebycheff utility over terminal states that is approximated through short Monte Carlo rollouts. This matters because existing shrinkage methods like RayGun and SCISOR optimize length alone, whereas biomolecular engineering inherently requires trading off function, developability, and manufacturability. The work is notable for combining a clean theoretical justification with preliminary wet-lab validation on shortened eGFP variants.

Biomolecular therapeutics often start from known sequences and require targeted editing to improve multiple properties while satisfying hard biochemical and manufacturability constraints. However, existing generative methods do not jointly support multi-objective optimization, hard feasibility, and sequence editing in discrete, variable-length biological spaces. In this work, we introduce Pareto-Constrained Molecule Editing (pCoMole), a framework built on discrete flow matching that steers a pre-trained Edit Flow toward user-specified preferences while enforcing terminal feasibility. pCoMole defines a feasibility-gated terminal distribution using an augmented Tchebycheff utility and realizes the resulting preference tilt through a Doob-h transform of the underlying edit process. To make this construction practical, we approximate the required harmonic function using short Monte Carlo rollouts over candidate edits, yielding an efficient guided editor with provable preference consistency. We validate pCoMole by shrinking GFP while retaining fluorescence-related properties, shortening diverse Cas9 orthologs while preserving PAM specificity, and compressing peptide binders into short peptidomimetics that optimize seven drug-related properties under hard constraints. In wet lab testing, two 229-residue pCoMole-designed eGFP variants retained clear green fluorescence in BL21 cells after 10 deletions, with either one or two substitutions. Together, pCoMole enables constraint-aware, Pareto-aligned editing of biomolecular sequences in discrete, variable-length spaces.
0
cs.LGcs.AIstat.ML Kabir Murjani, Nisarg Patel · Oct 1, 2026

This paper asks whether a fixed offline corpus generated by a behaviour policy can ever distinguish two candidate Regular Decision Processes (RDPs). It proves that when two candidates are $\pi$-observationally equivalent, their posterior odds remain frozen at the prior for any sample size (Theorem 6), gives an exact measure-zero characterization via the product-automaton blame language (Theorem 12), and derives PEC, a linear-time algorithm that certifies equivalence and names the missing experiment. The work matters because existing PAC offline guarantees for RDPs assume a distinguishability condition that was previously impossible to verify.

Non-Markovian environments are often modeled as Regular Decision Processes (RDPs), where dynamics depend on the interaction history through a finite automaton. Existing offline guarantees for RDPs rely on a distinguishability assumption on the behaviour policy but provide no means of verifying it. When the assumption is violated, distinct models may explain the data equally well. We study when data collected under a fixed behaviour policy can distinguish two candidate RDPs. We prove that the posterior odds between observationally equivalent candidates remain equal to the prior odds at every sample size, even when the policy visits every automaton state, and verify both results formally in Lean 4. We then characterize this equivalence exactly and derive PEC, an algorithm that decides it in time linear in the size of the product automaton. The distinguishability assumption of prior work fails on three of our four test environments, and the experiment identified by PEC restores it in each case.
0
stat.MLcond-mat.dis-nncs.LG Jorge Medina Moreira, Lorenzo Bardone, Lenka Zdeborov\'a · Oct 1, 2026

This paper develops a high-dimensional theory for why masked autoencoders learn better representations than standard autoencoders. In a spiked model with heterogeneous noise, the authors prove that a linear masked autoencoder recovers the latent feature at linear sample complexity $n=O(d)$ where PCA fails. They further quantify how dynamic mask resampling lowers sample complexity compared to static masking, and validate the qualitative predictions in CNNs, vision transformers, and a BERT pilot.

Why can masked prediction learn useful representations that unmasked reconstruction misses? We study this question in a high-dimensional model of a masked autoencoder (MAE) trained on data with shared latent structure and heterogeneous noise. We prove that masked linear reconstruction can recover the latent feature at linear sample complexity in regimes where unmasked linear reconstruction, equivalent to PCA, fails. The analysis also quantifies the statistical advantage of mask resampling, an established ingredient of masked pretraining. By introducing a fixed collection of $K$ masks per sample, we characterize its effect on feature recovery and downstream performance, identifying regimes where greater mask diversity lowers sample complexity. Guided by this prediction, we find that random cropping and flipping in standard image-training pipelines can obscure the advantage of mask resampling by renewing the prediction task even when the patch mask is fixed. Removing these transformations reveals a downstream advantage for dynamic over static masking in CNN autoencoders and vision transformers. A complementary BERT pilot finds benefits from greater mask diversity on downstream language tasks. Our results separate the benefit of the masked prediction objective from that of mask diversity, and show how a tractable theory can guide experiments that uncover advantages hidden by standard training practices.
0
math.OCcs.LGstat.ML Jiayi Song, Zi Xu · Oct 1, 2026

This paper proves worst-case lower bounds for stochastic first-order algorithms in nonconvex--concave minimax optimization, explicitly permitting variance-reduction via same-sample oracle queries. Its main result shows that the stochastic barrier remains $\Omega(L^3 D_Y^2 \Delta \sigma^2 \varepsilon^{-6})$ even with mean-square smoothness. Complementary nonconvex--strongly-concave bounds are also provided. Together the results establish that variance reduction does not improve the fundamental accuracy exponent in the minimax setting.

We establish complexity lower bounds for stochastic first-order algorithms in nonconvex--concave minimax optimization, allowing algorithms to use variance reduction. Our main contribution is a lower bound for a zero-respecting algorithm class that permits variance reduction, extending beyond the algorithmic restrictions imposed by some existing lower bounds. We consider objectives with an $L$-Lipschitz continuous joint gradient, a compact convex dual domain of Euclidean radius at most $D_Y$, and a primal value function, defined by maximizing the objective over the dual variable, with initial suboptimality at most $\Delta$. The target accuracy $\varepsilon$ is measured by the gradient norm of the Moreau envelope of the constrained primal value function with parameter $1/(2L)$. Under an unbiased stochastic first-order oracle with variance at most $\sigma^2$ and mean-square smoothness, we prove the lower bound $\Omega\!\left(L^2D_Y\Delta\varepsilon^{-3}+L^3D_Y^2\Delta\sigma^2\varepsilon^{-6}\right)$. This result quantifies the dependence on accuracy, dual-domain radius, and oracle noise even when variance reduction is allowed. We also establish complementary lower bounds for nonconvex--strongly-concave minimax optimization. With dual strong-concavity parameter $\mu>0$ and condition number $\kappa:=L/\mu$, we obtain $\Omega\!\left(L\Delta\sqrt{\kappa}\,\varepsilon^{-2}+L\Delta\kappa\sigma^2\varepsilon^{-4}\right)$ under the bounded-variance oracle model. Under the additional mean-square smoothness condition with constant $\bar L$, we obtain $\Omega\!\left(L\Delta\sqrt{\kappa}\,\varepsilon^{-2}+\Delta\bar L\sigma\kappa^{3/2}\varepsilon^{-3}\right)$. Together, these results identify complexity barriers across the concave and strongly concave regimes, with the main nonconvex--concave bound remaining valid for algorithms that use variance reduction.
0
math.OCcs.LG Wei Jiang, Rui Yan, Sifan Yang et al. · Oct 1, 2026

This paper studies stochastic multi-level compositional optimization, where the objective is a nested composition of $T$ smooth non-convex functions and only stochastic per-layer function and gradient values are available. The core idea is to use momentum-based estimators to track inner function values level-by-level, and then apply the chain rule to build a momentum gradient estimator for the overall objective. It matters because it establishes that the optimal $\mathcal{O}(\epsilon^{-4})$ sample complexity is achievable under standard smoothness assumptions, closing a gap with single-level lower bounds without relying on the stronger average-smoothness conditions common in prior multi-level literature.

This paper investigates stochastic multi-level optimization where the objective is a nested composition of several smooth non-convex functions. We assume that only stochastic estimates of the gradient and function values for each level are accessible. Consequently, obtaining an accurate estimate of the overall gradient is challenging due to the nested structure. To address this, we employ a momentum-based estimator with mini-batches to track the function values of each level, which are subsequently used to construct momentum gradient estimators. We establish an optimal sample complexity of $\mathcal{O}(\epsilon^{-4})$ for finding an $\epsilon$-stationary point, avoiding the stronger average smoothness assumption commonly relied upon in prior literature. Furthermore, by employing a normalization technique, we attain the same rate without requiring problem-dependent constants to set hyperparameters. To achieve the optimal rate without mini-batches, we further develop a batch-free method that incorporates a first-order approximation and a clipping technique for function value estimation. Finally, we validate the effectiveness of our proposed methods through experiments on risk-averse portfolio optimization and hierarchical tilted empirical risk minimization.