Nothing here yet
Speculative decoding drafters trained on short sequences collapse on long-context inputs because their position encodings extrapolate poorly. FOLD fixes this without retraining by keeping exact positions in a local window and compressing all distant context into a shared-position far memory. A concentration factor $\alpha$ in a decoupled softmax prevents attention dilution as the memory grows. Across six target–drafter pairs and contexts up to 256K tokens, the method consistently raises speedup, in the best case more than doubling it.
LLM workflows rely on fast binary decisions like moderation and reranking, but autoregressive generation is too slow for tight latency budgets. This paper studies System One models—lightweight scorers that emit bounded decision scores—and proposes Learn Then Test (LTT) to certify thresholds that maximize the fraction of inputs receiving the utility-maximizing action while controlling the False Omission Rate (FOR) with a finite-sample guarantee. The authors show across six datasets that naive thresholding at the target risk level exceeds the budget in roughly half of settings, whereas LTT violates it in only 7.8% of cases, making a strong case that a probability-like interface is insufficient for safe deployment under asymmetric risk.
Low-rank adapters can waste their parameter budget because not every learned direction improves downstream performance. ReGRA attacks this by splitting adaptation into two stages: first train a standard LoRA adapter, then estimate task gradients, project out the components already captured by the first stage, and use the residual directions to initialize a second adapter with an orthogonality regularizer. By merging both updates into the backbone, it keeps inference cost identical while increasing average accuracy within the same total rank budget.
Power grid steady-state analysis demands fast, generalizable surrogates for power flow, contingency screening, and voltage stability margin estimation. This paper investigates whether a pretrained tabular foundation model (TabPFN) can serve as a unified neural solver through in-context learning, without task-specific fine-tuning. The authors evaluate the approach on 118- and 2000-bus systems, arguing that it offers a flexible alternative to training separate graph or MLP baselines for each task and topology.
The paper studies proper hypothesis selection: given $n$ candidate distributions and sample access to an unknown target $P$, the goal is to return one candidate with total variation distance at most $3\min_i d_{TV}(P,H_i)+\varepsilon$. The core idea is to maintain weights over the candidates, define a weighted median of their densities as an unnormalized intermediate measure, and perform multiplicative weight updates using balanced tests derived from this median. The result achieves the optimal factor-3 approximation and sample complexity $O(\varepsilon^{-2}\log(n/\delta))$ while improving the running time to $O(n\varepsilon^{-2}\log(n/\delta))$, answering an open question of Aamand et al. (COLT 2026) by reducing the failure-probability dependence from $1/\delta$ to $\log(1/\delta)$.
This paper studies online convex optimization against comparators that switch $k$ times among only $m$ distinct points, generalizing the classic experts tracking problem to continuous domains. While discretizing the domain and applying the Bousquet–Warmuth long-term-memory algorithm yields information-theoretic regret $\widetilde O(md+k)$, this is computationally inefficient. The authors prove that, under the hypergraphic planted-clique conjecture, no polynomial-time algorithm can achieve $\widetilde O(k^a d^b)$ regret whenever the exponents violate $\theta a + b < \theta + 1/2$, establishing a fundamental computational barrier for exp-concave losses even when $m=2$.
Gaussian diffusion models poorly cover extreme regions for heavy-tailed data because their light-tailed corruption kernels yield exponentially sparse tail samples. This paper introduces Annealed Lévy Diffusion (ALD), which specifies time-dependent $\alpha(t)$-stable perturbation marginals via a Gaussian scale-mixture representation, enabling a two-phase schedule that transitions from heavy-tailed exploration to Gaussian refinement. The construction retains conditional-Gaussian score matching with only one added scalar input, making it compatible with standard diffusion training while targeting distributions with polynomial tails.
C-Slerp addresses long-horizon, sparse-reward goal-conditioned reinforcement learning without prior data, rewards, or demonstrations by leveraging hyperspherical contrastive representations for adaptive waypoint planning. Instead of training a separate high-level policy, it infers intermediate subgoals in closed form via spherical linear interpolation on a learned unit hypersphere and dynamically adjusts the number of waypoints to match each task's difficulty. Experiments on robotic manipulation and navigation show the method solves tasks like eight-cube tower stacking from scratch where strong baselines entirely fail, suggesting that adaptive task decomposition—not merely added hierarchy—is key to scaling goal-reaching RL.
Contextual partial monitoring is a sequential learning problem where the learner observes only partial feedback instead of incurred losses. The paper proposes E2D.PMSide, a computationally efficient instantiation of the Estimation-to-Decisions (E2D) meta-algorithm that reduces contextual partial monitoring to online regression. Its core contribution is replacing E2D's intractable per-round minimax exploration rule with tractable surrogates—epsilon-greedy for globally observable games and inverse gap weighting for locally observable ones—thereby achieving optimal $O(T^{2/3})$ and $O(\sqrt{T})$ regret rates. The work matters because it provides the first oracle-efficient algorithms for this setting with general function approximation and demonstrates practical benefits on LLM routing and label-efficient prediction.
One-step generative models typically demand complex distillation or fragile end-to-end distributional training. This paper introduces Priming Models, a two-stage framework that first pretrains with a simple reconstruction objective—surprisingly, even fixed-noise denoising suffices—and then refines the network with feature-space distributional losses. The core claim is that this reconstruction stage fully removes the training difficulty of distributional optimization, enabling state-of-the-art quality at roughly one-tenth the compute of prior one-step methods.
Neural operators learn mappings between function spaces, yet training deep architectures remains difficult. This paper proposes viewing each operator layer as the minimizer of a Bregman-regularized functional optimization problem in Banach spaces. This unifies classical neural operators under a proximal framework and yields a new Bregman variant that adds an inverse activation term, showing improved depth scaling and accuracy on standard PDE benchmarks.
COUNT tackles the GPU-memory bottleneck of scaling language-model embeddings by replacing learned n-gram tables with precomputed corpus counts stored on cheap host RAM. It constructs input embeddings via a weighted sum of the model's own unembedding vectors, requiring no extra learned parameters and enabling training-free domain adaptation at inference. On 60M--1B-parameter backbones, the authors demonstrate consistent perplexity and downstream gains without increasing GPU memory pressure.
Current evaluations of Knowledge Extraction Enhancement (KEE) techniques collapse recovery and disruption into a single accuracy score, concealing critical trade-offs. MAPLE introduces a unified diagnostic framework built on a pretraining-verified benchmark and three complementary metrics—surface-level $\text{Gain}/\text{Loss}$, probability-level $\text{PRS}$, and beyond-sampling $\text{SGR}/\text{DGR}$—to dissect intervention effects at multiple levels. Across six techniques and three model families, MAPLE shows that prompting recovers deep failures but disrupts many successes, while decoding and steering often reshuffle token probabilities without consistent benefit. The work offers a principled, actionable standard for comparing KEE methods beyond headline accuracy.
Feldman (2021) gave an influential lower bound linking the cost of not fitting training labels to excess population risk via frequency-dependent coefficients $\tau_{n,\ell}$. This paper demonstrates that Feldman's central conditioning identity fails because the counts of other points convey information about the sampling distribution, and it provides an exact three-point counterexample together with an exponentially large multiplicative gap. The authors recover a valid bound within the same model by replacing the count-only coefficient with the exact full-sample posterior mean, which depends on both multiplicity and the complete frequency profile.
Class-incremental graph learning suffers from catastrophic forgetting because models are repeatedly fine-tuned on new classes, causing severe recency bias where earlier sessions are misclassified as recent ones. MAEM addresses this by freezing a dedicated Mixture-of-Experts expert per session and routing test nodes via the reconstruction error of session-specific masked autoencoders, eliminating the need for task identities or a shared trainable router. To prevent unbounded model growth, it merges least-used experts via Git Re-Basin alignment followed by topology-aware pseudo-data distillation, achieving strong replay-free performance across four benchmarks.
Fine-tuning pretrained diffusion and flow matching models for reward maximization tends to collapse modes or shift mass between distant regions. This paper introduces Weighted Adjoint Matching, which modifies the stochastic optimal control formulation of Adjoint Matching by placing a time-dependent weight $w(t)$ on the quadratic control cost---expensive early to preserve coarse semantics, cheap late to refine details and increase reward. The authors prove equivalence to maximizing reward under pathwise KL regularization, and demonstrate improved reward-diversity tradeoffs on Gaussian mixtures and MNIST.
Training good latent tokenizers for generative models remains poorly understood, especially how to scale the encoder itself. This paper proposes Relative Prediction Representations (RePR), which supervise patch features to factorize a pointwise mutual information matrix estimated by a conditional diffusion model. The result is a theoretically grounded, negative-sample-free objective that resists feature suppression and claims to be the first to demonstrate compute-optimal tokenizer encoder scaling, matching or beating pixel-space diffusion at equal FLOPs.
KANLab is a browser-based educational tool that adapts the TensorFlow Playground interface to Kolmogorov-Arnold Networks (KANs). It strips away modern performance-oriented additions like residual connections and learnable scaling so that displayed B-spline curves map directly to their trainable control points, letting users drag points and watch gradient descent correct them in real time. By making KAN internals tangible in a compact 2D setting, it aims to bridge the gap between novices and experts.
Standard reinforcement learning on verifiable rewards incentivizes language models to guess rather than admit uncertainty, because a wrong answer carries no penalty while abstention foregoes any chance of reward. This paper frames graceful failure as a three-outcome RL problem (Success \succ Fallback \succ Error), identifies why naive policy-gradient methods collapse to degenerate always-fallback policies, and proposes GracefulRL, a log-barrier-shaped objective that preserves gradient signal near saturation. The idea matters because coding and agentic mistakes can propagate into real harm, yet current RL pipelines train models to maximize success rate without regard for how they behave when success is unlikely.
Mixed-precision quantization for sparse MoE models needs more than expert rankings: NatScale introduces contrast calibration, a principled way to control how strongly sensitivity scores compete for storage. It applies a model-specific power transformation $w_i = g_i^{\beta_\tau}$ so that normalized squared weights achieve a target Jeffreys divergence $\mathcal{J}(q(\beta_\tau), u) = \tau$, preserving rank order while setting a common information-theoretic scale. Combined with joint gate/up/down error measurement and spectral structural priors, it allocates bit-widths under a global byte budget via LP relaxation and rounding.