This paper exposes a critical flaw in how compressed memory is evaluated for large language models: standard ablation tests like zeroing memory vectors do not prove that stored document content improves predictions. The authors distinguish three empirically separable questions—whether a model depends on memory after training, whether training with memory outperforms training without it, and whether gains actually require document content—and argue that evaluation claims must be matched to specific controlled comparisons. Across nine memory systems and multiple QA benchmarks, they show these comparisons frequently lead to different conclusions, calling for stricter evidentiary standards in context-compression research.

LLM agents frequently fail in multi-turn settings by violating policies, using tools incorrectly, or acting prematurely. This paper proposes Critique-Augmented Agentic Fine-Tuning (CAFT), which rolls out a student agent, has an expert label each wrong turn with both a textual critique and a corrected action, and then fine-tunes the student to generate that critique and correction while keeping the original mistake in context. The central finding is that these natural-language explanations provide learning signal beyond action-only supervision, improving task success across four models and two benchmarks.

Multimodal agents need persistent memory to act coherently over long horizons, yet forming and managing memory across heterogeneous, high-dimensional observations—video, audio, 3D geometry, and GUIs—has lacked a unifying analytical lens. This survey introduces a two-axis lifecycle framework that classifies how experience is represented (source, textual, structured, latent, or parametric) and how memory states and policies evolve ($T1$ Static, $T2$ Dynamic, or $T3$ Self-Evolving). By analyzing 297 architectures, the authors identify key trade-offs between fidelity and efficiency and expose the striking scarcity of systems that evolve their own memory policies at deployment time.

Best-of-N sampling improves LLM output quality by generating multiple candidates and selecting the highest reward, yet its scaling behavior under imperfect reward models is poorly understood. This paper recasts BoN as extreme-value competition between the upper tails of correct and incorrect reward distributions, yielding phase transitions that explain both monotonic gains and reward hacking. Using extreme value theory, the authors derive finite-sample scaling laws showing polynomial rather than exponential convergence, and propose a lightweight tail-tension criterion for label-free early stopping. The work matters because it replaces vague intuitions about reward hacking with precise, testable distributional conditions and a practical inference-time safeguard.

Activation steering is used to steer language models toward named behaviors, but standard baselines cannot tell whether a vector controls a specific concept or a neighboring one. This paper introduces the competing-target test, which compares a steering vector with the matched vector for a competing target and bounds their difference with an equivalence test. In three open-weight models, the authors show that vectors for Kantian deontology and act utilitarianism are largely interchangeable---they move both framework scores alike---whereas explicit prompting separates them, and the same construction does separate optimistic and pessimistic tones.

LLM-jp-4 is a family of Japanese sovereign language models—an 8B dense Transformer and a 32B-A3B mixture-of-experts—trained from scratch on roughly 12 trillion tokens with an explicit emphasis on transparency, legal reproducibility, and practical deployment. The authors impose strict open-source selection criteria on data, external models, and software, and they document every stage from tokenizer design through pre-training, mid-training, and post-training. The work aims to provide a publicly verifiable reference for sovereign AI development in Japan.

ActICL attacks the problem of suboptimal multimodal in-context learning by training an MLLM agent to iteratively acquire, inspect, and curate demonstrations through tool-augmented reasoning. Its core idea replaces static similarity retrieval with an agentic loop of search, read, curate, and solve actions, learned via a two-stage post-training pipeline of rejection-sampling SFT and joint reinforcement learning. This matters because it shows that even a modestly sized trained agent can construct contexts that substantially improve its own accuracy and also boost larger frontier answering models.

When text and images conflict in vision-language models, prior work has characterized behavior as modality-specific biases such as "blind faith in text." This paper proposes a predictive alternative: a two-parameter Bayesian cue combination rule in which each modality contributes conditionally independent evidence weighted by a constant integration bias, plus a prior. The authors show that this simple linear model accurately predicts how VLM preferences shift across dozens of input degradation conditions and even transfers to entirely new corruption types without refitting, offering a unified account of why text sometimes dominates and vision sometimes wins.

VLX-Go targets embodied visual tracking in distractor-rich shared spaces by casting local motion as vision-language-conditioned short-horizon waypoint prediction. It combines frozen DINOv3 and SigLIP features, multiscale temporal context, and a transformer backbone to predict $K=8$ robot-centric waypoints. A staged training recipe—vision-language alignment, multimodal reasoning mid-training, and tracking specialization—drives a large improvement over the OmTrackVLA baseline, while preliminary DAgger fine-tuning provides modest further gains. This matters because existing methods suffer severe success-rate drops when moving from single-target to distractor tracking.

Ablation studies in language models are confounded by self-repair, where downstream components appear to compensate for a removed part. This paper argues that such compensation is not dynamically recruited but is instead a standing, pre-existing opposition released by the intervention. The authors introduce a continuous dose axis for counterfactual interventions and show that downstream contributions follow an affine law predicted in part by static weights. If the finding generalizes, it would reshape how interpretability researchers infer component importance from ablations.

Most LLM agents treat repeated use of the same external source as a sequence of independent retrieval events. This paper reframes the deeper problem as source learning: progressively building a persistent, reusable source model that captures how a source’s knowledge is structured, interpreted, and applied. The proposed SourceLearn system combines self-directed study with task-guided refinement, and the authors report large gains across document, code, API, and interactive benchmarks.

The paper tackles how to teach LLMs new skills after pretraining without erasing old ones. While RL resists catastrophic forgetting by staying on-policy, it struggles when the model cannot generate successful trajectories on its own. The authors flip the problem: instead of changing the training loss, they use an MCMC projection sampler to reshape off-policy expert traces so they look more like samples from the base model while preserving correctness. This lets ordinary supervised finetuning match or beat RL and self-distillation on science, math, and medical tasks.

This paper adapts Non-Linear Multi-Dimensional Concept Discovery (NLMCD) from vision to LLMs to model concepts as low-dimensional manifolds rather than linear directions. It introduces a fuzzy generalized Rand index called Concept-Based Alignment (CBA) that compares independently discovered concept spaces across layers, languages, and training stages without requiring shared tokenizers or architectures. The authors claim CBA is more sensitive than linear baselines (CKA, PCA), uses it to reveal cross-layer block structures, and applies it to study multilingual sharing, cross-model alignment, and post-training drift in Tulu-3.

Can response safety be scored by cosine similarity to a safe-response embedding centroid? The paper shows that such a raw positive prototype identifies a location, not a separating direction, and formalizes why positive-only observations cannot determine the sign of the safe-unsafe gap. On three prompt-controlled safety corpora and four frozen encoders, the unreferenced prototype is weak or inverted while an explicit safe-minus-unsafe reference recovers useful ranking; the result is carefully scoped to the centroid rule, not all one-class methods.

Emergency Department triage assigns acuity scores that can be skewed by non-clinical patient descriptors. This paper asks whether open-source LLMs change their pediatric Emergency Severity Index predictions when only demographic, socioeconomic, or contextual variables are injected into otherwise identical clinical vignettes. The authors audit ten models and find that a small fine-tuned Qwen2.5-7B is the most counterfactually stable, challenging the assumption that larger or medically pretrained models are necessarily fairer. The study matters because locally deployable open-source LLMs are increasingly considered for privacy-preserving clinical decision support, yet their fairness profiles across model families were poorly understood.

Where-OPD proposes a new on-policy self-distillation paradigm for multimodal large language models. Instead of giving the teacher zoomed-in image crops, it provides textual spatial hints that identify question-relevant visual elements and their coordinates, using procedurally generated synthetic counting scenes where object locations are known by design. The student learns to match the teacher’s behavior from the raw image and question alone, and the authors claim that training on just 3,000 such synthetic scenes transfers to broad real-world perception gains across 15 benchmarks.

Hierarchical Continuous Diffusion Language Models tackle a core bottleneck in non-autoregressive generation: discrete diffusion decodes parallel tokens as independent marginals, while continuous diffusion lacks valid token anchors until the final step. The authors propose HC-DLM, a hierarchy where a continuous latent trajectory is the sole persistent state and tokens are read out and fed back at every step to scaffold the next latent update. A single variational bound yields the training objective, and experiments span structured reasoning, mathematical planning, and language modeling.

CARM targets off-policy drift in LLM reinforcement learning by preventing signed token log-ratios from canceling in sequence-level response masks. The core idea is to average absolute log-ratios rather than signed ones, which stops bidirectional policy drift from being hidden by the geometric mean. It also proves a joint bound on the fraction and severity of out-of-band token ratios for any accepted response. The method is evaluated on Qwen3.5 models and shows consistent gains over geometric-mean and token-level baselines.

The paper asks whether stochastic rounding (SR) can improve low-precision transformer inference over round-to-nearest (RN), and shows that the benefit is site-dependent: SR outperforms RN in long MLP reductions because its $O(\sqrt{n}u)$ probabilistic error envelope beats the $O(nu)$ deterministic drift of RN, while RN is superior at the language-model head because SR's non-uniform variance is heavily penalized by the Fisher-weighted cross-entropy. The authors support this with a fast variable-precision emulation tool (PRISM), a vector Bienaymé–Chebyshev bound, a second-order loss decomposition, and experiments on DistilGPT-2 that motivate mixed-precision rounding recipes.

Training-data selection for LLMs is usually heuristic or sample-specific, which prevents generalization to unseen data. This paper tackles the harder problem of learning a shared meta-network that scores any example, and shows that plugging such a network into the standard bilevel objective (ScaleBiO) causes unstable optimization and poor transfer. The authors trace the failure to two properties of the ScaleBiO loss—competitive weight suppression and easy-feature dominance—and propose TESS, a method that replaces the bilevel objective with a simple pointwise MSE regression on pseudo-labels derived from loss gaps. TESS achieves strong transfer across datasets, corpus scales, and model sizes while cutting peak memory by over 20%.