Test-time scaling generates many candidate reasoning traces, but completing all of them is prohibitively expensive. This paper proposes PKTP, a pruning framework that selects a subset of partial rollouts to maximize the probability that at least one of them ultimately succeeds—i.e., $\text{pass}@k$. It models joint correctness with a latent Gaussian threshold model, derives a closed-form submodular surrogate, and greedily selects the kept set with a $(1-1/e)$ approximation guarantee. The work matters because it directly exposes and optimizes the quality–diversity trade-off that existing pruning methods handle only indirectly.

Language models are increasingly used for discrete decisions like tool selection or intent classification, yet prompting LLMs for these closed-set choices is slow and sensitive to option order. This paper introduces Crivo, a family of compact encoders scaling from 17 million to 1 billion parameters, and runs a controlled study to determine whether scoring each option in a separate encoder pass outperforms reading all options jointly in a single sequence. The work matters because it tests the limits of parameter efficiency and architectural choices for high-speed decision nodes that must generalize across changing label sets.

N-gram embedding tables give language models cheap capacity gains but can consume hundreds of billions of parameters. This paper asks whether those embeddings can be stored at low precision without losing their benefit. The authors systematically compare quantization schemes—uniform, normal-quantile, and Lloyd–Max codebooks with per-embedding scales—and find that aggressive bit reduction often preserves most of the tables’ gain, freeing memory to grow the tables themselves.

Depth growth and looping reuse computation across layers to train or run Transformers more efficiently. This paper shows that both methods create an inductive bias toward learning from context over memorizing fixed associations in weights, accelerating in-context learning and propagating induction heads through depth. The evidence spans models up to 3B parameters and connects architecture design to the mechanistic origins of few-shot performance.

Retrieval scores for clinical RAG are often computed against a single annotated chunk per query, conflating retriever quality with the completeness of the ground-truth annotations. This paper introduces RAG-MAMA, a breast-lesion question-answering system, and proposes mapping canonical evidence spans to all containing chunks so that overlapping segments with equivalent facts are not scored as competing candidates. The authors report nearly perfect retrieval metrics ($\text{Recall}@3 = 0.94$) on an 18-query development set and cautiously frame the work as a small methodological baseline, though the narrow scale and lack of independent validation severely limit its conclusions.

This paper develops a decision-theoretic framework for valuing memory in language agents, treating memory not as raw information but as a resource whose worth depends on future decisions, access costs, and revision capabilities. The core idea is to track value from original history through retained state to realized output, using avoided task loss as a common unit and introducing risk-cost frontiers for recovery and revision under uncertain demand. This matters because current memory architectures optimize proxy metrics like relevance or compression ratio without guaranteeing that retained distinctions actually improve downstream decisions.

Language model rankings on question-answering benchmarks are known to shift with prompt wording, raising doubts about single-prompt evaluation. This paper investigates whether augmenting models with deterministic tools stabilizes pairwise comparisons, proposing a matched protocol that isolates answer readout, reasoning, tool invocation, and report access to attribute changes in wording sensitivity. Applying the protocol to six open models across five domains and five prompt wordings, the authors find that tools improve accuracy where a program directly answers the question yet frequently widen ranking instability because call frequency itself becomes wording-dependent. The work matters because it shows that deterministic tool output is not enough to guarantee stable evaluation, and it recommends reporting call frequencies and pairwise stability alongside headline accuracy.

Multilingual language models often fail to recognize that two names written in different scripts refer to the same person. This paper introduces a controlled benchmark of 601 audited identity pairs with attested Devanagari and Latin forms, plus a 374-item Malayalam replication set, to isolate cross-script entity matching failures. The authors find that Qwen2.5-7B's same-person accuracy collapses from 35.1% in Latin-Latin to 8.5% in Devanagari-Devanagari under Marathi prompts, then use probes, calibration, and activation patching to diagnose whether the model retains latent label information and where it can be causally intervened upon.

CoWriter tackles passive human supervision in LLM writing tools by adapting cognitive-science co-authoring methodologies into a proactive agentic workflow. It deploys three specialized agents—Leader, Monitor, and Reviewer—to guide users through brainstorming, outlining, drafting, and revision while sustaining engagement and narrative control. A within-subjects study with 14 graduate-student participants shows authors feel substantially greater ownership and produce texts rated higher in human-touch qualities.

Reward models score responses to steer alignment and filtering, but pairwise benchmarks compare only answers to the same prompt, leaving prompt-level score shifts completely invisible. This paper shows that eighteen released scorers exhibit far larger between-prompt variance than human ratings, proposes a three-part human-referenced audit to isolate the source, and warns that blindly subtracting prompt means can actually degrade global response selection.

Automated survey generation has relied on flat document retrieval, which misses the evolutionary structure of research fields. SurveyGraph tackles this by constructing a Methods-Entity Knowledge Graph (MEKG) that encodes typed method-level relations—such as extends, improves, and applies—across papers. The graph then serves as a unified backbone for core-paper discovery, graph-aware retrieval, hybrid outline generation, and evolutionary narrative synthesis, aiming to make survey generation more auditable and structure-aware.

StateTrack-Bench studies whether LLMs can keep obeying a persistent rule across multi-turn conversations. The authors define State Integrity Collapse (SIC): a model remembers a constraint yet fails to enforce it in later answers, a failure mode they distinguish from ordinary forgetting. They evaluate seven pinned model snapshots across five constraint categories and five dialogue depths, showing that compliance decay is concentrated in symbolic rules and in one notably brittle model, and they argue convincingly that single-turn instruction-following metrics are insufficient for deployed assistants.

Museum description generation is usually framed as a data-to-text task where richer metadata improves grounding, yet heterogeneous and unevenly documented catalogue records break that assumption. This paper introduces a two-phase framework—observational across 10,250 artefacts and controlled across 500 objects—to isolate how metadata quantity, field type, and image access shape what LLMs include, omit, and infer. The central finding is that field identity, not just field count, determines faithfulness: context fields are omitted while content fields trigger unsupported elaboration, and images can increase inference rather than grounding. The authors argue that aggregate reference similarity masks these failures and motivate field- and claim-level evaluation.

LLM-as-a-Judge is widely used for multilingual evaluation, yet prompting strategies are usually validated only in English. This paper tests whether those strategies transfer across judge models and languages by benchmarking nine prompting strategies with four open-weight judges on 27 languages and four tasks. The central finding is that judge identity dominates prompting effects, with strategy effectiveness reversing across models and collapsing in low-resource languages, underscoring that prompt design must be validated per judge and language.

This paper investigates whether giving LLMs access to personal user memory actually improves responses from the user's perspective. The authors collect roughly 1,000 pairwise preferences from 87 participants judging responses to their own ChatGPT queries, with and without memory conditioning. Their central finding is that memory-augmented responses are not consistently preferred and can backfire, while off-the-shelf evaluators systematically overestimate the value of personalization. To close this gap, they introduce a rubric-based memory-aware reward model $R_{\theta}(q,y,M)$ that learns from real user judgments to predict when memory use is appropriate, achieving 72.8% accuracy on held-out participants versus ~59% for zero-shot LLM judges.

This paper introduces a ground-level dataset of 121,536 ground–reasons pairs extracted from 86,464 French civil cassation rulings spanning 2016–2025. Using a deterministic, rule-based pipeline, the authors pair each legal complaint with the Court’s response and classify the pair into established doctrinal families, producing a descriptive grammar of civil cassation. The resource enables quantitative analysis of which grounds succeed, which legal texts are cited, and how both evolve over time, alongside an interactive consultation tool.

ColaBench tackles the problem of evaluating how personal agents use long-term interaction history to guide the acquisition and application of external knowledge. The core idea is a traceable benchmark built on an Evidence Graph $\mathcal{G}=(V,E)$ that links personal facts to their disclosures and to supporting documents, paired with MemKB, a retrieval method that connects state summaries and memory-document links. This matters because current benchmarks test memory recall and retrieval in isolation, leaving open whether agents can jointly recover historical conditions and apply newly retrieved rules to a specific user's situation.

Document-level LoRA adapters are pitched as modular knowledge units for retrieval and merging, yet standard open-book evaluation cannot tell whether an adapter has memorized a document or merely learned to read it well. This paper introduces a cloze probe that withholds the source text and finds that open-book scores are almost entirely generic reading behavior: a random adapter recovers 82% of a specialist's open-book advantage but only 2% of its cloze gain. The findings argue that adapter-bank evaluation needs a closed-book probe alongside open-book scores, and the authors' candid disclosure of four silent errors is a model of experimental hygiene.

RetiJev tackles the latency, cost, and privacy barriers of autoregressive LLMs in point-of-care ophthalmic decision support. The authors introduce RetiJev-Data, a 128k-question source-grounded corpus, and a family of 0.8B–9B compact models that score predefined candidate answers in a single pass instead of generating free-form text. This yields sub-200 ms inference per case on one GPU while reportedly surpassing a 27B LLM and a commercial decision model on guideline grading and clinical-action accuracy.

This paper investigates why user-stated preferences for AI responses often fail to predict their actual ratings. Using the public PRISM dataset, the authors show that the gap between stated and revealed preferences largely disappears when attributes are measured via aggregate human perceptions rather than simple text proxies, yet concrete features like length dominate ratings regardless of what users claim to value. The findings suggest that personalization systems should treat preference elicitation format as a design parameter matched to the decision at hand, not as a direct window into stable, latent utility functions.