Nothing here yet
Museum description generation is usually framed as a data-to-text task where richer metadata improves grounding, yet heterogeneous and unevenly documented catalogue records break that assumption. This paper introduces a two-phase framework—observational across 10,250 artefacts and controlled across 500 objects—to isolate how metadata quantity, field type, and image access shape what LLMs include, omit, and infer. The central finding is that field identity, not just field count, determines faithfulness: context fields are omitted while content fields trigger unsupported elaboration, and images can increase inference rather than grounding. The authors argue that aggregate reference similarity masks these failures and motivate field- and claim-level evaluation.
The paper asks whether offline audits can reliably certify that an agent update will be safe once deployed. It formalizes the post-audit mirage by constructing matched safe and harmful lifecycles that expose byte-identical public evidence to the verifier while an isolated scorer records opposite deployment truth. The authors prove a non-identifiability theorem at this audit interface, instantiate it in an exact benchmark spanning authorization, scheduling, and shared-capacity environments, and propose a fail-closed monitor that uses fresh group-labelled harm outcomes to decide only when the evidence supports it. The core message is that correctness checks on fixed observations cannot substitute for missing deployment information.
Existing computer-use benchmarks conflate planning, reasoning, and execution, making it hard to pinpoint exactly where agents fail at precise visual control. PointerBench introduces a diagnostic suite of 1,500 static screenshot–instruction pairs that isolates fine-grained GUI grounding into three tracks: spreadsheet geometry, character-level text targets, and professional-application controls. The work matters because it reveals that even top frontier models drop sharply on thin structural elements and text carets, while specialist grounding models lag further behind, exposing substantial headroom before agents can reliably manipulate dense professional interfaces.
Dual Exam proposes evaluating Multimodal Agents and World Models separately under controlled conditions, rather than as a coupled system. The framework holds environment responses fixed to test Agents and fixes action interventions to test World Models across physical, digital, social, and scientific domains. By diagnosing component-level failures rather than assigning blame from end-to-end outcomes, it provides a principled basis for studying reliable interaction at scale. The evaluations cover 1,088 Agent items and 400 World Model cases, though the authors explicitly caution that the resulting profiles do not directly predict coupled performance.
Large language models increasingly expose detailed reasoning traces that help users but also provide rich supervision signals for unauthorized distillation by competitors. This paper proposes Conflict-Aware Logit Adapters (CALA), a lightweight training-time defense that attaches a small residual adapter to a frozen teacher and uses a virtual proxy-student ascent step to identify fragile imitation positions. By encoding anti-distillation perturbations into the model rather than imposing them at decoding time, CALA aims to preserve teacher utility and fluency without the online overhead of prior decoding-time approaches such as Antidistillation Sampling (ADS).
Diffusion language models generate tokens in parallel, but few-step decoding makes it hard to decide which predictions to commit at each iteration without losing future conditioning opportunities. SetCommit is a training-free coordinator that sits between the model forward pass and the decoder state update, selecting commitment sets by balancing per-token reliability against a heuristic local joint risk and allowing budget-neutral exchanges of vulnerable earlier commitments. On LLaDA-8B and Dream-7B, the authors report quality gains up to 7.4 percentage points and geometric-mean incremental speedups of $1.44$--$1.74\times$ across reasoning and code tasks at common quality targets.
LLM-as-a-Judge is widely used for multilingual evaluation, yet prompting strategies are usually validated only in English. This paper tests whether those strategies transfer across judge models and languages by benchmarking nine prompting strategies with four open-weight judges on 27 languages and four tasks. The central finding is that judge identity dominates prompting effects, with strategy effectiveness reversing across models and collapsing in low-resource languages, underscoring that prompt design must be validated per judge and language.
This paper investigates whether giving LLMs access to personal user memory actually improves responses from the user's perspective. The authors collect roughly 1,000 pairwise preferences from 87 participants judging responses to their own ChatGPT queries, with and without memory conditioning. Their central finding is that memory-augmented responses are not consistently preferred and can backfire, while off-the-shelf evaluators systematically overestimate the value of personalization. To close this gap, they introduce a rubric-based memory-aware reward model $R_{\theta}(q,y,M)$ that learns from real user judgments to predict when memory use is appropriate, achieving 72.8% accuracy on held-out participants versus ~59% for zero-shot LLM judges.
Generative adaptive tutoring systems risk trapping marginalized learners in remedial loops when LLM graders misread linguistic variation as conceptual error. Kamalov et al. formalize this 'algorithmic drift' as a coupled dynamical system, prove a bifurcation threshold $\epsilon^\star = 1 - p^\star/\sigma(\omega)$, and propose a Temporal Fairness Stabilizer (TFS) that periodically forces difficulty probes to break the feedback loop. The work reframes educational fairness as a longitudinal property of closed-loop control rather than a static deployment audit.
Notation3 (N3) Logic extends RDF with graph terms—entire graphs used as terms in triples—to support statements about statements and rule-based reasoning. This paper fills a long-standing gap by providing the first comprehensive formal semantics for these graph terms, introducing an abstract syntax and a two-layer semantics (basic entailment followed by N3-entailment for built-in predicates). The work matters because N3 has lacked a standardized model-theoretic account, leaving implementations like EYE and Cwm without a common semantic foundation.
As LLMs move from isolated tools to networked populations, the structure of who interacts with whom becomes a key safety variable. This paper studies how network topology shapes convention formation and adversarial tipping in populations of up to $10^5$ LLM-derived agents playing the naming game. By placing policies extracted from four models on fully connected, heterogeneous, and sparse regular networks, the authors show that topology modulates convergence speed, can rescue stuck populations, and creates hub-shaped vulnerabilities that well-mixed estimates miss.
Optimisation-based decision support systems recommend schedules or plans by solving mixed-integer programs, but end users often ask "why not X?" contrastive questions that are hard to answer reliably with pure LLMs. This paper proposes LLMxMIP, a pipeline that extracts irreducible infeasible subsystems from a constraint model, organises them into a structured graph of reasons, and prompts an LLM to generate a textual explanation. The work matters because it marries rigorous constraint reasoning with natural language generation to address a real oversight requirement in automated decision-making.
Decision-support methods typically evaluate predefined strategies under uncertainty while neglecting the combinatorial architecture of available choices. This paper proposes multichoice modeling, a framework that jointly simulates interdependent decision configurations and uncertain futures, then uses variance-based sensitivity analysis and conditional distribution comparisons to identify influential choices and emergent strategies. Demonstrated on mortgage design, manufacturing upgrades, and M&A strategy, the work reframes decision support from evaluating fixed options to exploring and designing strategy configurations.
Frontend coding agents are usually graded only on whether the requested target gets built, but real engineering demands that the rest of the site keeps working. ProductWebBench evaluates this second obligation—website continuity—through 400 frozen tasks where a simulated user reveals requirements stage by stage and every cleared gate remains armed for the rest of the run. Across 8,737 runs from 13 models, 28.2% of Change attempts that pass every requested-work check still fail a continuity check, showing that scoring a patch against the target alone is insufficient.
SecLog-Bench introduces a benchmark for post-detection cybersecurity log reasoning that requires LLMs to infer attack progress from scaffolded multi-source evidence and produce structured, evidence-referenced reasoning chains. The authors construct 29 reproducible attack scenarios spanning CVE exploits and multi-hop attacks, aligning host audit logs, application logs, and network traffic into stage-labeled evidence windows. Evaluation of eight mainstream LLMs reveals modest attack-stage prediction (best accuracy 44.44%) and unreliable reasoning (best SRF1 54.87%), demonstrating that final-label accuracy alone misrepresents true model capability.
This paper introduces a ground-level dataset of 121,536 ground–reasons pairs extracted from 86,464 French civil cassation rulings spanning 2016–2025. Using a deterministic, rule-based pipeline, the authors pair each legal complaint with the Court’s response and classify the pair into established doctrinal families, producing a descriptive grammar of civil cassation. The resource enables quantitative analysis of which grounds succeed, which legal texts are cited, and how both evolve over time, alongside an interactive consultation tool.
Writing efficient kernels for Huawei Ascend NPUs is notoriously hard because AscendC code is scarce in LLM training data and operator deployment requires coordinating host-side tiling with device-side kernels. CANNBot-CAKE presents a full-stack agentic workflow that turns a natural-language request into a compiled, packaged, and benchmark-validated CANN custom operator. Its key ideas are a multi-DSL router that uses an item-response model to choose among four code-generation tracks, an evolution mode that iteratively refines kernels under a roofline model, and a cross-operator wiki (CAKE Book) that accumulates verified experience. On 53 CANN-Bench operators on Ascend 950PR, CAKE reports 100% standard-case and 98.28% hidden-case pass rates, with a mean score of 78.63/100.
LLM-native advertising introduces new auction-design challenges because an ad’s click-through rate depends on its context passage and earlier ads steal clicks from later ones under a first-click cascade. This paper studies truthful mechanism design when the LLM answer and its passage order are fixed before bidding, showing that a constant-factor welfare approximation is compatible with truthfulness even under this editorial-order restriction. The authors propose click thinning—randomly omitting earlier ads to equalize click services across positions—which turns welfare maximization into a tractable linear-fractional matching problem and yields an elegant optimal tradeoff between approximation, truthfulness, and computational efficiency.
The paper diagnoses a critical split in how LLMs handle Theory of Mind: explicitly recognizing an agent's belief versus applying that belief to predict behavior or evaluate actions. It argues that aggregate ToM benchmarks conflate these skills and hide severe applied deficits, then tests whether the gap reflects a capacity limit or merely latent competence that can be recovered via prompting, in-context learning, or lightweight parameter adaptation. The work is motivated by the deployment of LLMs in socially sensitive roles where surface-level mental-state recognition is insufficient for trustworthy interaction.
Early-stage structural design is an iterative, creative process that balances form, force, and material. This paper proposes a proof-of-concept workflow connecting a large language model agent to a COMPAS-based structural modeling API via a Model Context Protocol (MCP) server, enabling natural-language generation and editing of semantic building elements. By producing explicit graph-based artifacts rather than generic meshes, the work aims to reduce modeling friction and move toward human-AI co-design in structural engineering.