Empty Commitments: When Agents Promise What Their Runtime Cannot Deliver
This paper identifies empty commitments: promises that LLM agents make about future actions their runtime cannot actually execute. Unlike broken promises, these are structurally impossible from the moment of speaking because the agent lacks the required tools, scheduling, memory, or delegation affordances. The authors define a formal taxonomy of failure types and propose a measurement protocol, though no empirical results are presented.
The paper makes a crisp conceptual contribution by distinguishing empty commitments---promises impossible given the agent's runtime configuration---from broken promises that fail only in later execution. Its taxonomy of capability, temporal, and agentive failures, together with the anchoring condition, provides a precise vocabulary for a real deployment problem. However, the paper is purely definitional: it proposes a measurement protocol but presents no experimental results, ending only with open questions and the acknowledgment that "A multi-model evaluation is under way." At four pages, the work raises more questions than it answers.
The strongest element is the formally grounded conceptual framework. The felicity condition---that "a commitment is felicitous iff some execution path under $\Phi_A$ performs $\alpha$ when $\varphi$ holds, within $\delta$; otherwise it is empty"---is clear and decidable from the agent's configuration and a single turn. The anchoring distinction between calling an enabling tool in the same turn versus promising without one captures a subtle but important failure mode, and the outcome taxonomy is well-designed, explicitly treating over-refusal as a cost metric so that "over_refusal is the cost side, so a fix that makes the model refuse everything does not count as a fix."
The most serious limitation is the total absence of empirical results. The paper defines a measurement protocol across five setups (C0--C4) and two environment variants, but reports no outcome distributions, rates, or comparisons. Every metric is hypothetical. The authors candidly note that "The protocol is single-turn with mocked tools, and its LLM-based labels need human validation," which undermines the reliability of automated measurement even before experiments begin.
The distinction from prior work on broken promises is conceptually clean but under-argued without data showing that empty commitments are actually prevalent in deployed systems. The doability matrix and deterministic labeling rules are described at too high a level for independent implementation.
There is essentially no evidence in the paper beyond conceptual definitions. The authors do not empirically demonstrate that empty commitments are common, that their protocol successfully detects them, or that stating the environment reduces failures. Comparisons to related work on overclaiming, prospective memory, and promise-breaking are brief but appropriately position the paper as studying "the prior question: was the promise possible at all?" However, without any reported experiments, these distinctions remain rhetorical rather than grounded.
Reproducibility is severely limited. No code, prompts, or mock tool specifications are released. The five setups are sketched in Table 3 but tool schemas, persona prompts, and the "short paragraph of facts about the runtime" added in the stated variant are not provided. The labeling pipeline relies on a setup-blind detector and a judge that return five booleans per response, but neither component is specified in sufficient detail to rebuild. The authors' admission that "its LLM-based labels need human validation" compounds the problem.
A chatbot that says "I will remind you tomorrow" will not run again until the user writes. We call such a promise an empty commitment: a promise of an action after the current turn that nothing in the agent's tools or runtime can carry out. Unlike a broken promise, its emptiness follows from the agent's configuration alone; no later trajectory is needed. We define empty commitments on top of commitment semantics, with three failure types, an anchoring condition for promises that a tool could make real, and a response-level outcome taxonomy. We then describe a measurement protocol: follow-up requests run in five setups that add one persistence affordance at a time, with the environment either left implicit or stated.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.