Your paper timeline
Scroll AI takes the way you would scroll a great paper aggregator: quick signal first, deeper critique when something earns your attention, and challenges when a claim feels off.
797 papers in cs.LG
Trending mixes fresh papers with community signal.
0
cs.LG Xiufeng Yang, Konstantinos Vrettos, Zeineb Ben Chaaben et al. · Oct 22, 2026

Many clinical decisions are rapid multiple-choice tasks that require uncertainty estimates for safe deployment. This paper evaluates four System 1 models, zero-shot non-generative classifiers that return answer probabilities in a single forward pass, on 86 clinical tasks across six capability tiers spanning text and imaging. The authors find that these models excel at medical knowledge questions and report extraction, operating in under half a second and for a few joules per decision, but they fall sharply on clinical judgement and volumetric imaging tasks. The results map a clear capability gradient and make a pragmatic case for where fast, cheap decision models can fit into clinical workflows.

Many clinical decisions are a choice among a few fixed options, taken quickly and in large numbers. Such decisions also need an uncertainty estimate to know when to defer to a clinician. Generative language models answer slowly in free text and state their confidence poorly. In contrast, System 1 models answer zero-shot with a probability for each answer. However, their performance across clinical tasks is unknown. Four System 1 models, two proprietary and two with open weights, were evaluated zero-shot on 86 public tasks from 14 clinical domains in 20 languages, spanning examination questions and medical imaging. Tasks were assigned to six capability tiers before analysis. The primary outcome was the area under the receiver-operating-characteristic curve (AUROC). Secondary outcomes included calibration, language effects and comparisons with published results and generative models. Across 179,111 cases, the mean AUROC of the best model per task was 0.94 for medical knowledge, 0.93 for text understanding, 0.83 for clinical judgement from text, 0.89 for image perception, 0.84 for 2D image findings and 0.69 for volumetric findings. Imaging tasks scored 0.09 (95% CI, 0.06 to 0.11) lower than text tasks (P < 0.001). The best model matched or exceeded the best published result on 13 of 26 text tasks and on 8 of 35 imaging tasks. Triage nurses outperformed the best model (accuracy, 0.85 vs. 0.56). One decision took 0.34 to 0.44 seconds and 24 to 81 joules on open-weight models run locally. Accuracy on identical questions fell by up to 39 percentage points from English to Yoruba. Generative models of the same family were 11 to 76 times slower, with 3 to 5 percentage points higher balanced accuracy. System 1 models decide quickly, cheaply and with usable confidence when the answer is stated in a text or recalled from medical knowledge, but not when it must be judged or read from an image.
0
cs.LGstat.ML Wenye Ren, Anna Gy\H{o}rffy-Kerekes, Tancredi Dorsi et al. · Oct 22, 2026

Global AUC can hide regions where a classifier discriminates poorly, which is dangerous in high-stakes clinical risk stratification. This paper introduces LOCATOR, a method that computes a localized AUC ($\mathrm{LAUC}$) for each participant by weighting all event–non-event pairs by Gower-distance similarity, then trains an XGBoost regressor on $\logit(\mathrm{LAUC})$ from baseline covariates to flag high- and low-competence regions without subsetting the data. The authors derive asymptotic properties of the $\mathrm{LAUC}$ estimator and apply the framework to the guideline-endorsed PREVENT model in NHANES, with external validation in UK Biobank.

Risk prediction models are frequently evaluated with AUC, which means that during evaluation we are blind to performance heterogeneity, which could lead to bad decision-making in clinics. In this work, we aim to change that by identifying patient cohorts for whom the risk prediction model might not be an accurate predictor. To separate these cohorts, we use a version of the weighted AUC, which we call Localized AUC or LAUC. Using this metric, we build a method that we coin LOCATOR. This method is able to identify cohorts of patients for whom the risk prediction model is inaccurate. We test this method both in real-world data and in simulations. We also build a statistical background for the use of Localized AUC for our purposes.
0
stat.MLcs.LGmath.PR Yuansi Chen, Francesco Pedrotti, Peter A. Whalley · Oct 22, 2026

Classical analyses of the Unadjusted Langevin Algorithm (ULA) in Wasserstein-2 distance predict a dimension-dependent mixing time and bias, yet empirically ULA often estimates low-dimensional statistical functionals accurately without scaling the step size with dimension. This paper proposes using the subspace robust Wasserstein (SRW) distance to analyze ULA on ridge-separable Bayesian posteriors, proving that iteration complexity can be dimension-free in both the overparameterized regime ($N \lesssim d$) and the large-sample regime ($N \gtrsim d^2$), while showing such guarantees are impossible near the proportional regime $N \asymp d$. The results explain a bell-shaped bias profile and provide a rotationally invariant alternative to prior $\ell^\infty$-based dimension-free analyses that fail under isotropy.

The Unadjusted Langevin Algorithm (ULA) is widely used for high-dimensional Bayesian inference. Classical analyses in Wasserstein-2 distance predict an asymptotic bias and a mixing time that grow with dimension, but this metric is stronger than necessary for many statistical tasks. We instead analyze the bias in subspace robust Wasserstein distance, which controls the mean and arbitrary low-dimensional marginals and is therefore well suited to observables typically estimated in practice. For ridge-separable potentials formed with i.i.d. data, we prove high-probability bounds showing a bell-shaped mixing profile as the number of data points $N$ varies relative to the dimension $d$. In the overparameterized regime, the effective dimension is the rank $N$ of the data-dependent perturbation, leading to complexity depending on $N$ rather than $d$. In the large-sample regime, Gaussian-perturbation estimates yield dimension-free complexity once $N$ is sufficiently large. In contrast, near the proportional regime $\asymp d$, we prove that dimension-free guarantees are impossible in general.
0
cs.LG Gr\'egoire Ouerdane, Sinan Ibrahim, Stefan Streif et al. · Oct 22, 2026

This paper attacks a foundational weakness in RL evaluation: standard benchmarks rank algorithms by relative return without knowing the optimal policy, so they cannot measure true distance to optimality. The authors propose converse optimality, a construction that generates control tasks together with certified optimal value functions and policies, enabling absolute metrics like optimality gap and regret. The framework separates continuous-time sensitivity studies from native discrete-time oracle-referenced comparisons, and extends the certificate logic to non-smooth values, non-affine action interfaces, and partially observed settings. By turning benchmarks into known-answer instruments, the work promises to expose failure modes and regime dependencies that relative rankings completely hide.

Benchmarking in reinforcement learning is usually relative: algorithms are ranked by returns on environments with unknown optima. Distance to optimality remains unobservable, while conclusions may depend on reward design, stochasticity, hyperparameters, environment selection, and simulator details. We develop a ground-truth-first framework based on converse optimality, constructing each task with a certified optimal value function and policy withheld from learners and used only for evaluation. The framework unifies two modes: sensitivity benchmarking samples certified environments from continuous-time generators to assess robustness to task variation, while oracle-referenced comparison uses native discrete-time generators to evaluate policies via optimality gap, regret, and policy error. Continuous-time models support physically interpretable sensitivity studies but introduce dependence on the simulator step ($\Delta t$) and numerical scheme. Native discrete-time models align the environment, oracle, and learner on the same transition law, eliminating external integration choices as a confounder. We reproduce quantitative results covering optimizer sensitivity and oracle-referenced comparisons on serial planar arms, dynamically extended nonholonomic vehicles, and quadrotor-inspired tasks. The methodology extends beyond smooth affine Gaussian systems: virtual-input pullback handles non-affine raw-action interfaces and finite-covariance non-Gaussian disturbances. A non-smooth converse optimality formulation enables certified benchmarks with kinked value functions and switching surfaces. For partially observed evaluation, an information-state formulation reduces benchmarking to converse optimality on a sufficient statistic and provides an information-penalty reference. Code, fixtures, and diagnostics are available on the public project page.
0
cs.LG Maharani A. P. Irawan, Kuntai Du, Jiayi Yao et al. · Oct 22, 2026

Gandalf tackles SSD wear-out in LLM serving systems that offload ever-growing KV caches beyond GPU and host memory. The core idea is that hybrid attention architectures (full plus linear/sparse) generate KV states with predictably different reuse lifetimes, so preferring SSD placement for long-lived full-attention KV while keeping short-lived linear states in DRAM can slash write volume without sacrificing cache hits. The paper claims average 7.3x lifetime extensions on trace-driven simulation, which matters because production agentic workloads can exhaust enterprise SSDs in months.

KV caching reduces the serving cost of large language models (LLMs), but the growing KV cache footprint is increasingly exceeds GPU and host-memory capacity. Therefore, people use SSD to expand the KV cache storage capacity with high I/O bandwidth. However, in production LLM workload, such as multi-round conversation and LLM agents, the total amount of KV cache writes significantly exceeds SSD capacity, shortening SSD lifetime from years to months. To improve SSD lifetime, a natural solution is to reduce the KV cache writes by gating the KV cache writes via historical access patterns. However, that significantly lowers the KV cache hit rate (and increase the LLM serving cost) since KV caches from different attention types might have identical historical access pattern but different future access patterns. Our insight is that, to reduce KV cache writes without hurting KV cache hit rate, one need to not only consider the access pattern, but also application-level information such as the attention type of KV cache. Leveraging this observation, we propose Gandalf , the first SSD KV cache offloading system with SSD life time awareness. Gandalf prioritize the admission of full-attention KV cache into SSD over the cache generated by other types of attention, which significantly increases SSD lifetime without sacrificing cache hit rate. Our evaluation shows that Gandalf extends the SSD lifetime by 7.3X on average, while having negligible impact on cache hit rate across multiple workload traces and wide range of hardware capacity. Gandalf design can be extended to other flash-based storage, such as High Bandwidth Flash (HBF).
0
cs.LG Ioannis Kouroudis, Zhaosu Gu, Poonam et al. · Oct 22, 2026

Machine learning for atomistic materials discovery is often bottlenecked by the cost of generating density functional theory (DFT) training data for new chemical systems. PYTHIA attacks this by freezing a pretrained MACE foundation model, adapting its atomic features through a lightweight Transformer, and feeding the result into a Gaussian Process that both predicts adsorption energies and quantifies uncertainty for Bayesian Optimization. The framework is designed to make complex architectures usable on small DFT datasets, enabling simultaneous optimization of adsorbate placement and substrate defects in understudied materials.

Machine learning has permeated material optimization, providing high value. Nevertheless, its re- liance on a high number of samples limits its applicability to relatively explored material families. In this paper we extend the applicability of machine learning to novel and understudied materials through the PYTHIA (Probabilistic Yet Transferable Hierarchical Interaction Architecture) framework. Pythia utilizes two of the most powerful machine learning concepts, foundational models and transfer learning. We employ a complex foundational model trained on a large dataset of chemically similar molecules, on a target that is physically close to our prediction target. Then, we freeze the weights and perform subspace alignment with the use of a transformer layer and add uncertainty quantification with gaussian processes. These two layers can be trained on a limited dataset yet allow us to leverage the gained knowl- edge and the studied materials and complex architectures. The resulting model is used as a surrogate for a Bayesian Optimization scheme. To showcase the robustness of the framework, we applied it to the investigation of adsorption and defect optimization of two distinct systems. Not only were we able to correctly and efficiently identify the optimal adsorption sites of even complex a
0
cs.CVcs.LG Ignacy Kolton, Wenxuan Li, Zongwei Zhou et al. · Oct 22, 2026

DLoss tackles objective misalignment in radiology report generation: standard cross-entropy treats diagnostic and narrative tokens identically, so models can sound fluent while omitting critical findings or inserting false ones. The core idea is to pool decoder probabilities across sequence positions with different operators—noisy-OR for supported diagnoses and tempered log-sum-exp for unsupported ones—and add this as an auxiliary loss without changing architecture or inference. Evaluated across six 3D MedVLMs on ~50,000 abdominal CT scans, DLoss improves clinical metrics on most benchmarks, though gains vary widely by backbone.

Automated radiology report generation requires models to do more than produce fluent clinical narratives: they must explicitly state supported diagnoses while avoiding unsupported ones. However, existing medical vision-language models (MedVLMs) are typically trained with token-level maximum likelihood objectives, which prioritize text agreement over diagnostically critical terms. As a result, models can miss key diagnoses and introduce wrong ones. We identify this as an objective misalignment between conventional language modeling and the asymmetric diagnostic requirements of radiology reporting. To encode these requirements, we introduce Diagnostic Loss (DLoss), a differentiable, architecture-agnostic objective that operates directly on decoder token probabilities. DLoss separately aggregates probability mass over vocabularies associated with diagnoses supported by the reference report and with unsupported alternatives. Using asymmetric pooling across token positions in the report, it encourages supported diagnoses to receive sufficient probability in at least one position while suppressing spurious probability spikes for unsupported diagnoses across all positions during training. The loss can be applied to existing 3D MedVLMs without modifying their vision backbones, language decoders, or inference procedures, and introduces no additional inference-time computation. We train and evaluate DLoss on 50,000 abdominal CT scans across several 3D MedVLM architectures on both in-distribution and out-of-distribution report-generation benchmarks. Across these settings, DLoss improves diagnostic faithfulness while maintaining competitive performance on conventional language-generation metrics, showing the value of asymmetric sequence-level supervision for radiology report generation. The code for DLoss is available at https://github.com/gmum/DLoss.
0
cs.LG Ana Leticia Garcez Vicente, Saber Salehkaleybar · Oct 22, 2026

Continuous-time linear SDEs provide a natural model for interacting dynamical systems, but recovering causal drift coefficients from equilibrium data is difficult when reciprocal interactions (directed 2-cycles) or latent variables are present. This paper establishes new graphical sufficient conditions for generic identifiability of the drift matrix from stationary covariance in non-simple graphs, and develops an instrumental-variable framework to identify target direct effects under latent confounding. Both contributions extend the rank-based algebraic approach of Dettling et al. (2023) to settings previously unexplored for stationary linear SDEs.

Continuous-time stochastic differential equations (SDEs) provide a natural framework for modeling interacting dynamical systems. A central causal question is whether causal effects among observed variables can be identified from observational data drawn from the stationary distribution, given the underlying causal graph. For linear SDEs, existing identifiability results are largely restricted to fully observed systems whose causal graphs contain no directed 2-cycles. We first establish new graphical conditions for the generic identifiability of causal effects in systems that may contain directed 2-cycles, thereby allowing for reciprocal interactions. Next, we address the previously unexplored setting of latent variables by developing an instrumental-variable framework for identifying a target direct causal effect under partial observability.
0
cs.LGstat.ML Tim Nico Bauerschmidt, Isabel Valera, Jilles Vreeken · Oct 22, 2026

Next-event prediction in sequential data is usually handled by opaque neural networks, while existing symbolic rule methods lack a principled way to form a unified predictive model. This paper introduces Rules, a neuro-symbolic network that learns Temporal Conjunctive Prediction Rules via differentiable temperature annealing, and Runes, which adds a residual transformer to correct rule-based logit predictions. It matters because it offers an explicit accuracy-interpretability trade-off, producing compact, inspectable symbolic rules for safety-sensitive domains while remaining competitive with black-box transformers.

In real-world applications, being able to explain why a prediction is made is often as important - if not more so - than achieving high accuracy. However, existing methods for sequential event prediction typically prioritize accuracy at the expense of interpretability. Neuro-symbolic approaches offer a promising direction to improve interpretability by learning symbolic rules, but extending them to next-event prediction is challenging due to the presence of temporal dependencies. We introduce RULES, a neuro-symbolic model that learns sequential rules directly from data, and combine it with a residual network to form RUNES (Rule Network for Event Sequences). RULES accurately recovers ground truth rules from synthetic data and discovers compact, interpretable rule sets on real data. RUNES combines these rules additively with a residual network, achieving competitive accuracy while providing transparent access to active rules and how residuals adjust predictions.
0
stat.MLcs.LG Caixing Wang, Yue Wang · Oct 22, 2026

Random projections (RP) offer a powerful way to reduce the $O(n^3)$ cost of kernel spectral algorithms, but existing theory has largely been confined to well-specified settings or narrow mis-specified regimes under restrictive boundedness assumptions. This paper establishes high-probability minimax optimal learning rates in Sobolev norms for RP-based spectral algorithms under general source conditions and a Bernstein noise condition, without assuming uniform boundedness of the regression function $f_\rho$. By exploiting the RKHS embedding property and a refined operator similarity analysis, the authors enlarge the optimal source range from $s \geq 1 - 1/\beta$ to $s > \alpha_0 - 1/\beta$, showing that projection dimensions proportional to the effective dimension preserve statistical efficiency while yielding substantial computational savings.

Random projections (RP) offer an effective method for reducing computational and storage overhead while preserving the geometric structure of data. However, existing studies have primarily focused on the optimal generalization performance of specific kernel-regularized algorithms with RP in the well-specified setting and under restrictive conditions. In this paper, we provide a comprehensive and improved analysis of the generalization performance of RP-based spectral algorithms under general conditions \change{without sacrificing statistical efficiency}. By leveraging the embedding property of the reproducing kernel Hilbert space (RKHS) and a refined analysis of the operator similarity, we establish optimal learning rates in Sobolev norms that match the minimax lower bounds up to logarithmic factors. For two common implementations of RP, randomized sketches and Nystr\&#34;{o}m sub-sampling (uniform or leverage-based), we show that the projection dimension required for optimality is proportional to the average or maximal effective dimension, yielding a significant reduction in computational cost while maintaining the statistical efficiency. Our results do not rely on the uniform boundedness assumption on the regression function and are valid across a wide range of source conditions, i.e., $s> \alpha_0-1/\beta$, where $s,\beta$, and $\alpha_0$ denote the smoothness index, capacity index, and the embedding index, respectively. In the benign case when $\alpha_0=1/\beta$, the optimality holds for all $s\in (0,2\tau]$ with $\tau$ and $\gamma$ denoting the qualification index and norm index. Experimental results demonstrate both the theoretical validity and the practical effectiveness of the proposed method.
0
cs.LGcs.CRcs.DC Miha O\v{z}bot, Seiichi Ozawa, Igor \v{S}krjanc · Oct 22, 2026

Federated clustering under non-IID data must minimize communication rounds while preventing leakage of local data properties. This paper proposes Evolving Federated Density Peak Clustering (eFDPC), where each client incrementally learns diagonal-covariance Gaussian micro-clusters from private mini-batches, selects local density peaks, and transmits masked peaks through a shuffler-based anonymization layer. A server-side second density-peak step reconstructs unlinkable global centers from additive shares, aiming for accurate one-shot clustering without centralizing raw samples.

In this paper, we present a novel one-shot federated clustering approach based on evolving fuzzy clustering and shuffler-based anonymization. In federated clustering, each round reveals additional model information to the server and can increase the opportunity to infer properties of local datasets, even when raw samples are never shared. Consequently, fewer client-server exchanges and unlinkability are important, so transmitted information cannot be attributed to any single client. Data-stream clustering algorithms, which learn in a single pass over the data, align well with the one-shot federated learning paradigm, where a single communication round can be sufficient. The proposed Evolving Federated Density Peak Clustering (eFDPC) method combines evolving fuzzy data-stream clustering and density peak clustering in a federated setting. Each client independently evolves micro-clusters from private data mini-batches through cluster addition and merging. To limit communication, density peak clustering is applied to the learned micro-clusters and only the selected peaks are transmitted, rather than the full local models. A shuffler-based layer is introduced between the clients and the server to anonymize contributions prior to reconstruction and global peak selection. We consider a peak-only variant, eFDPC-P, as well as eFDPC-G with server-side label propagation. Experiments on synthetic and real-world benchmark datasets show that eFDPC outperforms recent federated clustering baselines in one-shot settings and scales to higher-dimensional, non-IID data.
0
cs.LG Xinyu Zhao, Qingyun Sun, Ziwei Zhang et al. · Oct 22, 2026

Training-free generalist graph anomaly detection must identify heterogeneous anomalies on unseen graphs without target-domain labels. Jev-GAD addresses this by extracting structural, attribute, and neighborhood evidence, converting it into graph-relative percentiles $e_j(v)=\mathcal{R}_G(\delta_j(v))$, and packaging selected evidence into semantically labeled decision states for a frozen pretrained language model. The model returns anomaly scores via non-generative option-logit readout $s_{\mathrm{dec}}(v)=\sigma(\ell_v^a-\ell_v^n)$, which are then fused with four complementary statistical scorers through equal-weight rank normalization. While the multi-view evidence design is principled, the paper hinges on a costly, manually engineered LLM interface and reports gains that sit atop an unstable baseline landscape.

Graph anomaly detection (GAD) aims to identify irregular entities in graph-structured data, yet heterogeneous anomaly patterns and distributional differences across graphs challenge generalizable detection. Existing training-free approaches avoid task-specific optimization but primarily rely on predefined numerical scoring rules, leaving the semantic interpretation of heterogeneous anomaly evidence largely unexplored. In this paper, we propose Jev-GAD, a training-free generalist graph anomaly detection framework that bridges heterogeneous numerical evidence and pretrained decision mechanisms. To address heterogeneous anomaly patterns, we construct complementary structural, attribute, and neighborhood evidence and transform heterogeneous statistics into graph-relative anomaly percentiles. To move beyond predefined numerical scoring rules, we leverage a single frozen pretrained language model for evidence-conditioned anomaly assessment. To bridge the semantic gap between numerical evidence and pretrained models, we design a semantic Graph-to-State (G2S) transformation that organizes graph context and semantically labeled node evidence into structured decision states for non-generative option-logit readout. The resulting decision scores are integrated with four complementary statistical scores through equal-weight rank-based fusion, without GAD-specific gradient-based optimization. Extensive experiments on seven graph anomaly detection benchmarks spanning citation, social, and co-review networks demonstrate the effectiveness of Jev-GAD, achieving average AUROC and AUPRC scores of 89.18\% and 52.40\%, respectively. Further analyses validate the contributions of pretrained decision readout and semantic evidence grounding.
0
cs.IRcs.LG Zhongkai Shangguan, Yanbin Kang, Yihan Cao et al. · Oct 22, 2026

PAIR tackles a fundamental mismatch in professional network retrieval: relevance is complementary rather than symmetric. LinkedIn proposes a triple-tower retrieval framework that learns from exposure-conditioned preference signals via a multi-negative contrastive objective, combines static profile embeddings with lightweight dynamic history updates in embedding space, and supports multiple serving budgets through Matryoshka training. The work matters because demonstrated retrieval gains translated into real business impact at billion-user scale.

General-purpose embeddings trained on profile similarity often misrank candidates in professional networks, where relevance is inherently complementary rather than symmetric. For example, job seekers usually engage with recruiters rather than peers with similar credentials. Current user recommendation systems are often built on profile similarity or social signals such as shared connections, which are effective in friend-oriented social networks where users tend to connect with similar people or within existing social circles. However, this does not hold in professional platforms like LinkedIn, where interactions are often goal-driven and cross-role interactions, requiring matching based on complementary intents rather than similarity or social proximity. In this paper, we propose Preference-Aligned Intent Retrieval, a retrieval framework that is designed to model user intent preference instead of profile similarity. Our approach is inspired by the preference-learning perspective of Direct Preference Optimization, but instantiates preference learning as an exposure-conditioned multi-negative contrastive objective over embeddings. This formulation simplifies training and scales efficiently to large candidate sets without requiring an explicit reference model. Furthermore, we incorporate user interaction history by aggregating click histories with positional encoding to capture temporal order, which enables constant-cost updates of user representations over a fixed history window, allowing new interactions to be incorporated without re-encoding history. As a result, personalization can be updated efficiently using lightweight CPU operations without repeated LLM inference during serving. We evaluate PAIR on LinkedIn's Premium retrieval task. A 0.6B parameter model achieves a MAR@100 of 0.45.
0
cs.LG John J. Vastola, Kanaka Rajan · Oct 22, 2026

This paper introduces a continuous-time linear-quadratic model of open-loop target pursuit to study how agents learn interception strategies. It derives exact closed-form solutions for optimal control trajectories, REINFORCE policy learning dynamics, and temporal-difference value learning dynamics. The central finding is that the eigenvalue spectrum of the learning dynamics matrix generically displays a gap, meaning agents learn to reach the target before optimizing their movement path. The results offer a rigorous theoretical benchmark for pursuit behavior and its learning dynamics.

Agents tasked with intercepting a moving target must learn how to reach it, ideally as efficiently as possible. As observers of animal pursuit behavior have noticed, different strategies are possible: on one extreme, the agent reactively moves toward the target's current location; on the other, the agent predicts the target's future location and moves directly there. Motivated by the desire to understand how such strategies might be learned, we introduce a continuous-time linear-quadratic model of open-loop target pursuit whose optimal control strategies interpolate between these possibilities. Usefully, a small number of interpretable parameters control which type of strategy is optimal, and it is possible to derive closed-form solutions for optimal strategies, policy learning dynamics, and value learning dynamics. Exploiting our model's linear structure, we find that the time scales of learning precisely correspond to the eigenvalues of certain matrices, and that relevant eigenvalue spectra generically have a gap. We show that this gap indicates that agents tend to learn how to reach their target before optimizing their movement. Our results provide a detailed mathematical characterization of pursuit behavior and its learning dynamics, which can both serve as a benchmark for empirical work, and as a foundation for more elaborate theoretical treatments of pursuit.
0
cs.LGcs.CVcs.RO Hossein Resani, Javen Qinfeng Shi · Oct 22, 2026

Inside-JEPA tackles the opacity of latent world models by embedding relational channels for ordered entity pairs directly into the JEPA latent state. The core idea is that an inverse-dynamics loss—recovering actions from changes in these channels—can ground them to geometric relations without any relation labels during training. This matters because it could let world-model states be read and given goals in relational terms, the natural vocabulary for planning and manipulation.

Joint-embedding predictive architectures (JEPAs) learn world models in latent space, enabling efficient planning but leaving their states opaque. We introduce Inside-JEPA, which places relational channels for ordered entity pairs inside the JEPA latent state and shows that an inverse-dynamics loss can ground these channels without relation labels when its head reads them, either alone or as part of the full state. On Something-Something V2, latent prediction alone performs no better than a random relation head, while inverse-dynamics supervision that recovers actions from channel changes improves relational AUROC by .065-.103 across ten settings with annotated boxes, by .081 on RH20T robot episodes with held-out tasks, and by .050-.061 across five relations using off-the-shelf detection and tracking. An inverse head on entity latents decodes actions more accurately but barely improves relational grounding; shuffled labels, two label-free objectives, and action labels on only 5% of the pairs also fail to ground the channels. Given an action, predicted relational changes approach the quality of observed future relations (AUROC .599 vs .621). For one-step relational goals, a supervised readout of the 16 channels ranks the correct action among the top five of 65 in 43% of cases, compared with 24% for prediction-only latents and 17% for a frequency prior.
0
cs.CLcs.LG Tardelli Ronan Coelho Stekel · Oct 22, 2026

Language models are increasingly used for discrete decisions like tool selection or intent classification, yet prompting LLMs for these closed-set choices is slow and sensitive to option order. This paper introduces Crivo, a family of compact encoders scaling from 17 million to 1 billion parameters, and runs a controlled study to determine whether scoring each option in a separate encoder pass outperforms reading all options jointly in a single sequence. The work matters because it tests the limits of parameter efficiency and architectural choices for high-speed decision nodes that must generalize across changing label sets.

Many language-model pipelines end in a closed-set decision, such as selecting a tool. Prompting a large language model for it is slow and order-sensitive. We ask how small an encoder for such decisions can be and whether it should read the options jointly or separately. In a three-seed study at 32 million parameters, per-option scoring beats joint scoring on a held-out sample of a 37-benchmark decision suite by 5.8 points of chance-corrected skill at matched optimizer steps and by 2.8 at approximately matched training compute; two external benchmarks agree, though not significantly on their unseen tasks. Crossing reader with initialisation shows that the advantage requires a reranker initialisation, whose query-candidate pair input matches per-option reading; from the raw encoder the two readers are indistinguishable. Per-option scoring needs 5 to 13 times more encoder passes: at most twice the median latency, but more than four times the mean. Of six released joint encoders (17 million to 1 billion parameters), the 68- and 150-million-parameter models exceed a 435-million-parameter DeBERTa-v3 zero-shot classifier by 4.9 and 5.9 accuracy points on 209 overlap-audited System1-Arena tasks and are at least on par with it on the 145 tasks unseen in training. The smallest answers a median request in 14 milliseconds on 16 processor threads, against 728 for the classifier. Reordering options changes 12 to 23% of the joint models' answers (per-option scoring: none). Up to 68 million parameters, they lead their size range on a public leaderboard's private, mainly in-distribution tests.
0
cs.CRcs.LG Tran Minh Thanh, Vo Hoai Viet · Oct 22, 2026

This paper proposes an adaptive framework for cryptographic primality testing that integrates arbitrary-precision arithmetic, hardware-aware sieving, and a machine-learning-guided Miller-Rabin witness selector anchored on quantum entropy. The core idea is to reduce expensive modular exponentiations by using lightweight structural features of a candidate integer to prioritize bases likely to expose compositeness early. It targets the key-generation bottleneck for RSA and ECC, yet the contribution is more architectural than empirically demonstrated, resting heavily on a withdrawn survey and a toy Python benchmark.

The security of modern public-key cryptographic systems, such as RSA, Diffie-Hellman, and ECC, depends entirely on the computational intractability of the integer factorization and discrete logarithm problems. For these asymmetric algorithms to operate, a core requirement is that the system must be capable of generating, testing, and simultaneously proving the primality of extraordinarily large numbers. To meet those requirements, this study proposes a comprehensive framework, bridging clas- sical number theory with modern machine learning models and optimization algorithms. Specifically, the paper delineates an adaptive, arbitrary-precision architecture dedicated exclusively to primality testing. Concurrently, the research deeply analyzes the theoretical foundations of testing methods (both deterministic and probabilistic), hardware-level optimization techniques, and the application of quantum-secured entropy sources, while providing rigorous mathematical proofs to guarantee security even when the prime distribution is biased.
0
cs.LG Haolin Liu, Chen-Yu Wei, Julian Zimmert · Oct 22, 2026

This paper initiates the statistical theory of reverse-KL policy distillation under misspecification, asking whether the teacher's full action distribution improves efficiency over action-level feedback and whether online student trajectories are necessary versus offline teacher data. The authors establish minimax-optimal rates for full-feedback settings, develop clipped importance-weighting and trajectory-truncation algorithms, and prove a striking separation: offline action feedback can match full-feedback rates, while online action feedback requires exponentially more data. The results give a first theoretical footing for empirical design choices in modern LLM distillation.

Reverse Kullback-Leibler (KL) distillation has shown empirical promise for training language models, but its statistical guarantees and limits remain poorly understood. In particular, the effects of two design choices remain unclear: Does access to the teacher's full action distribution provide an advantage over observing only the probability it assigns to the sampled action? Must training trajectories come from the student (online distillation), or can teacher-generated trajectories (offline distillation) suffice? Our work takes a first step toward understanding the theory underlying these questions. We study the sample complexity of online reverse-KL distillation under two feedback models: the teacher reveals either its full action distribution or only the probability of the student-generated action. We establish minimax bounds for the former and show that substantially more data may be required for the latter. In contrast, offline reverse-KL distillation can attain the same minimax rate whether the teacher reveals its full distribution or only its probability of the teacher-generated action. The key distinction is that teacher-generated actions themselves reveal the teacher's preferences, while queries at student-selected actions can leave those preferences unobserved. This highlights an additional statistical challenge for online distillation compared to offline distillation. These statistical guarantees do not follow simply from optimizing standard distillation objectives: our lower bounds show that common online objectives can fail when candidates are evaluated only on past student-generated trajectories. We develop trajectory-loss and importance-ratio truncation techniques together with policy-space learning to achieve optimal online distillation guarantees.
0
cs.LGcs.IR Jialin Lu, Xiaoyu Liu, Hengrui Luo et al. · Oct 22, 2026

As Mathlib grows past 400,000 theorems, effective semantic search is critical for both human formalization and automated proving, yet real-world queries are short and ambiguous while existing engines train on lengthy, context-rich informalizations. Lean Finder V2 closes this gap with an adversarial generator-retriever loop that synthesizes over 600,000 faithful, challenging short queries, then uses them for both retriever fine-tuning and GEPA-optimized input rewriting. The result is a new state of the art on human-curated and real-world benchmarks, plus measurable gains in downstream theorem-proving tasks.

We introduce Lean Finder V2, a semantic search framework for Mathlib that disambiguates user queries through complementary model-level adversarial training and input-level query rewriting. Human mathematicians and coding agents rarely search Mathlib by providing detailed context. In practice, they query short properties, notation, aliases, or hypotheses, leaving much of the intended mathematics implicit. In contrast, existing Mathlib search engines unrealistically assume context-rich queries. The scarcity of real-world short queries paired with ground-truth Mathlib statements makes this ambiguity even more intractable. We argue that this is essentially a data problem. Through adversarial training, we scale up the generation of labeled short and ambiguous queries, and learn to disambiguate user queries at both model-level fine-tuning and input-level query rewriting. On the human-curated MathlibQR benchmark, our Lean Finder V2 sets a new state of the art, with at least 11% relative improvement over the previous best Mathlib semantic search engines. On top of our state-of-the-art retriever, optimized query rewriting further improves MathlibQR retrieval. On unlabeled queries from human and coding-agent interactions, Lean Finder V2 is ranked as the strongest system. Integrated with Claude Code, Lean Finder V2 improves proof success on downstream theorem-proving task. Lean Finder V2 is available at: https://leanfinder.github.io/
0
cs.LGcs.AI Ali Najar, Hosein Hasani, Amir Kooshan Fattah et al. · Oct 22, 2026

Meta-RL agents struggle to adapt quickly to unseen tasks when task evidence is sparse or spread across long interaction histories, because recurrent models compress history too aggressively and transformers scale quadratically. METTA proposes a two-scale neural memory architecture that uses Test-Time Training (TTT) layers for fine-grained within-episode adaptation and compact episode summaries for cross-episode reasoning, preserving linear sequence complexity. The core claim is that this design enables stronger task-family-level generalization and more structured exploration on long-horizon sparse-reward tasks than both recurrence and full-attention baselines, while keeping wall-clock training cost low.

Meta-reinforcement learning aims to train agents that adapt quickly to unseen tasks from limited interaction, but this requires representing task evidence over long interaction histories. Classical recurrent agents compress experience into hidden states, while transformer-based agents retrieve from past trajectories at quadratic memory and computation cost. In this work, we propose Memory-Enhanced Test-Time Adaptation (METTA) for meta-reinforcement learning. It decomposes the fast adaptation phase into two temporal scales: a causal neural memory performs fine-grained within-episode adaptation with linear sequence complexity, while compact episode-level summaries support long-range cross-episode task inference without attending over the full interaction history. Moreover, equipping agents with neural memory provides sufficient expressive capacity and test-time training capability, enabling effective adaptation to unseen held-out tasks and task families at meta-test time. On unseen held-out Meta-World ML10/45 test environments, METTA achieves a success rate over $50\%$ while maintaining a low training budget and short wall-clock time. On long-context sparse-reward PointMaze tasks, it outperforms transformer-based baselines by exhibiting more structured exploration behavior. These results show that METTA provides an efficient path toward more adaptive and scalable long-context meta-RL.