Nothing here yet
Text-to-music alignment is usually tested with a single global similarity score that hides which specific musical attributes are missing or misaligned. This paper proposes MQAScore, a fine-grained evaluator that uses a large audio language model to verify each atomic concept—across genre, instrument, mood\/theme, vocal, and usage dimensions—via binary question answering. The work matters because it moves beyond opaque global scores toward interpretable, per-dimension diagnostics that can directly guide improvements in text-to-music generation systems.
Automated survey generation has relied on flat document retrieval, which misses the evolutionary structure of research fields. SurveyGraph tackles this by constructing a Methods-Entity Knowledge Graph (MEKG) that encodes typed method-level relations—such as extends, improves, and applies—across papers. The graph then serves as a unified backbone for core-paper discovery, graph-aware retrieval, hybrid outline generation, and evolutionary narrative synthesis, aiming to make survey generation more auditable and structure-aware.
Scene retrieval requires compact image representations that preserve geometric invariance without sacrificing discriminative power. This paper proposes aggregating multi-scale, second-order CNN statistics using a Fisher Kernel framework, yielding a global descriptor for scalable search. The core idea is to replace explicit visual vocabularies with Gaussian mixture modeling and to pool Fisher-encoded features across image resolutions. The authors evaluate this on standard INRIA Holidays and UKBench benchmarks, claiming improved mAP over classical aggregation baselines.
T2Quant addresses the challenge of quantizing Diffusion Language Models (DLMs), whose iterative denoising process makes activation precision sensitivity vary across both tasks and timesteps. The core idea is a label-free proxy—combining local KL divergence with downstream influence—to predict where extra precision helps most, then allocating mixed activation bit-widths across denoising steps via exact dynamic programming under a fixed average budget. This yields consistent accuracy gains of up to 6.7 percentage points over fixed-precision quantized baselines on three DLMs and six benchmarks.
Long-horizon coding agents often notice failures yet keep following flawed plans. RIPPLE tackles this "Critique-to-Action Handoff Gap" by internalizing recovery supervision into a single self-correcting policy. It decomposes recovery into reflection triggering, diagnosis, and corrective action, using a prefix-only teacher and stage-specific RL credit that gates trajectory advantages with local validity signals. The result is a mean improvement of about 4.8 points across four software-engineering benchmarks without an online critic at deployment.
StateTrack-Bench studies whether LLMs can keep obeying a persistent rule across multi-turn conversations. The authors define State Integrity Collapse (SIC): a model remembers a constraint yet fails to enforce it in later answers, a failure mode they distinguish from ordinary forgetting. They evaluate seven pinned model snapshots across five constraint categories and five dialogue depths, showing that compliance decay is concentrated in symbolic rules and in one notably brittle model, and they argue convincingly that single-turn instruction-following metrics are insufficient for deployed assistants.
Museum description generation is usually framed as a data-to-text task where richer metadata improves grounding, yet heterogeneous and unevenly documented catalogue records break that assumption. This paper introduces a two-phase framework—observational across 10,250 artefacts and controlled across 500 objects—to isolate how metadata quantity, field type, and image access shape what LLMs include, omit, and infer. The central finding is that field identity, not just field count, determines faithfulness: context fields are omitted while content fields trigger unsupported elaboration, and images can increase inference rather than grounding. The authors argue that aggregate reference similarity masks these failures and motivate field- and claim-level evaluation.
The paper asks whether offline audits can reliably certify that an agent update will be safe once deployed. It formalizes the post-audit mirage by constructing matched safe and harmful lifecycles that expose byte-identical public evidence to the verifier while an isolated scorer records opposite deployment truth. The authors prove a non-identifiability theorem at this audit interface, instantiate it in an exact benchmark spanning authorization, scheduling, and shared-capacity environments, and propose a fail-closed monitor that uses fresh group-labelled harm outcomes to decide only when the evidence supports it. The core message is that correctness checks on fixed observations cannot substitute for missing deployment information.
Existing computer-use benchmarks conflate planning, reasoning, and execution, making it hard to pinpoint exactly where agents fail at precise visual control. PointerBench introduces a diagnostic suite of 1,500 static screenshot–instruction pairs that isolates fine-grained GUI grounding into three tracks: spreadsheet geometry, character-level text targets, and professional-application controls. The work matters because it reveals that even top frontier models drop sharply on thin structural elements and text carets, while specialist grounding models lag further behind, exposing substantial headroom before agents can reliably manipulate dense professional interfaces.
Dual Exam proposes evaluating Multimodal Agents and World Models separately under controlled conditions, rather than as a coupled system. The framework holds environment responses fixed to test Agents and fixes action interventions to test World Models across physical, digital, social, and scientific domains. By diagnosing component-level failures rather than assigning blame from end-to-end outcomes, it provides a principled basis for studying reliable interaction at scale. The evaluations cover 1,088 Agent items and 400 World Model cases, though the authors explicitly caution that the resulting profiles do not directly predict coupled performance.
Large language models increasingly expose detailed reasoning traces that help users but also provide rich supervision signals for unauthorized distillation by competitors. This paper proposes Conflict-Aware Logit Adapters (CALA), a lightweight training-time defense that attaches a small residual adapter to a frozen teacher and uses a virtual proxy-student ascent step to identify fragile imitation positions. By encoding anti-distillation perturbations into the model rather than imposing them at decoding time, CALA aims to preserve teacher utility and fluency without the online overhead of prior decoding-time approaches such as Antidistillation Sampling (ADS).
Diffusion language models generate tokens in parallel, but few-step decoding makes it hard to decide which predictions to commit at each iteration without losing future conditioning opportunities. SetCommit is a training-free coordinator that sits between the model forward pass and the decoder state update, selecting commitment sets by balancing per-token reliability against a heuristic local joint risk and allowing budget-neutral exchanges of vulnerable earlier commitments. On LLaDA-8B and Dream-7B, the authors report quality gains up to 7.4 percentage points and geometric-mean incremental speedups of $1.44$--$1.74\times$ across reasoning and code tasks at common quality targets.
LLM-as-a-Judge is widely used for multilingual evaluation, yet prompting strategies are usually validated only in English. This paper tests whether those strategies transfer across judge models and languages by benchmarking nine prompting strategies with four open-weight judges on 27 languages and four tasks. The central finding is that judge identity dominates prompting effects, with strategy effectiveness reversing across models and collapsing in low-resource languages, underscoring that prompt design must be validated per judge and language.
This paper investigates whether giving LLMs access to personal user memory actually improves responses from the user's perspective. The authors collect roughly 1,000 pairwise preferences from 87 participants judging responses to their own ChatGPT queries, with and without memory conditioning. Their central finding is that memory-augmented responses are not consistently preferred and can backfire, while off-the-shelf evaluators systematically overestimate the value of personalization. To close this gap, they introduce a rubric-based memory-aware reward model $R_{\theta}(q,y,M)$ that learns from real user judgments to predict when memory use is appropriate, achieving 72.8% accuracy on held-out participants versus ~59% for zero-shot LLM judges.
Generative adaptive tutoring systems risk trapping marginalized learners in remedial loops when LLM graders misread linguistic variation as conceptual error. Kamalov et al. formalize this 'algorithmic drift' as a coupled dynamical system, prove a bifurcation threshold $\epsilon^\star = 1 - p^\star/\sigma(\omega)$, and propose a Temporal Fairness Stabilizer (TFS) that periodically forces difficulty probes to break the feedback loop. The work reframes educational fairness as a longitudinal property of closed-loop control rather than a static deployment audit.
Notation3 (N3) Logic extends RDF with graph terms—entire graphs used as terms in triples—to support statements about statements and rule-based reasoning. This paper fills a long-standing gap by providing the first comprehensive formal semantics for these graph terms, introducing an abstract syntax and a two-layer semantics (basic entailment followed by N3-entailment for built-in predicates). The work matters because N3 has lacked a standardized model-theoretic account, leaving implementations like EYE and Cwm without a common semantic foundation.
As LLMs move from isolated tools to networked populations, the structure of who interacts with whom becomes a key safety variable. This paper studies how network topology shapes convention formation and adversarial tipping in populations of up to $10^5$ LLM-derived agents playing the naming game. By placing policies extracted from four models on fully connected, heterogeneous, and sparse regular networks, the authors show that topology modulates convergence speed, can rescue stuck populations, and creates hub-shaped vulnerabilities that well-mixed estimates miss.
Optimisation-based decision support systems recommend schedules or plans by solving mixed-integer programs, but end users often ask "why not X?" contrastive questions that are hard to answer reliably with pure LLMs. This paper proposes LLMxMIP, a pipeline that extracts irreducible infeasible subsystems from a constraint model, organises them into a structured graph of reasons, and prompts an LLM to generate a textual explanation. The work matters because it marries rigorous constraint reasoning with natural language generation to address a real oversight requirement in automated decision-making.
Decision-support methods typically evaluate predefined strategies under uncertainty while neglecting the combinatorial architecture of available choices. This paper proposes multichoice modeling, a framework that jointly simulates interdependent decision configurations and uncertain futures, then uses variance-based sensitivity analysis and conditional distribution comparisons to identify influential choices and emergent strategies. Demonstrated on mortgage design, manufacturing upgrades, and M&A strategy, the work reframes decision support from evaluating fixed options to exploring and designing strategy configurations.
Frontend coding agents are usually graded only on whether the requested target gets built, but real engineering demands that the rest of the site keeps working. ProductWebBench evaluates this second obligation—website continuity—through 400 frozen tasks where a simulated user reveals requirements stage by stage and every cleared gate remains armed for the rest of the run. Across 8,737 runs from 13 models, 28.2% of Change attempts that pass every requested-work check still fail a continuity check, showing that scoring a patch against the target alone is insufficient.