Beyond Answer Confidence: A Controlled Audit of Self-Knowledge in a Black-Box Decision Model
Decision models like Jev return probability distributions to drive routing, abstention, and human review, yet average calibration cannot distinguish whether uncertainty stems from chance or from missing knowledge. The authors audit this distinction in a black-box decision model through roughly 575,000 API calls across paired interventions that vary supplied information while holding items fixed, plus targeted follow-up questions about the case and the model’s own familiarity. Their core finding is that confidence tracks correctness on familiar material but fails at knowledge boundaries—assigning up to $p_{\max}=0.80$ to salient options with no basis and growing overconfident past an observed temporal boundary—while apparent self-knowledge largely evaporates once surface cues are controlled. The work matters because it provides a practical, behavioral checklist for black-box knowledge audits and shows that querying case properties directly can be more reliable than reading a single output distribution.
The paper makes a strong empirical case that Jev’s top-answer confidence is operationally blind to missing knowledge: with no relevant information it assigns up to $p_{\max}=0.80$ to salient options, and past an observed knowledge boundary it exceeds accuracy by $0.21$–$0.33$ while recalibration on earlier months does not close the gap. The audit is unusually careful, holding items and ambiguity fixed while varying supplied evidence, comparing future chance events with fixed unknown facts, and adding text-only baselines and cue-controlled variants. Its central limitation is scope: it is a case study of one closed model version, and several salient controls were added post-hoc on the same news panel, so the evidence for general claims about decision models or black-box audits is suggestive but not conclusive.
The controlled experimental design is the paper’s greatest strength. The intent dose–response shows Jev follows evidence when given swapped examples ($90$%/$98$% mapping fidelity), and the open-model comparison demonstrates that salience bias is not intrinsic to decision models. The decomposition of follow-up questions is also sharp: asking whether an outcome is “settled” achieves AUROC $1.00$, and whether evidence is “enough” reaches $0.95$, well above the $0.85$ obtained from confidence on the same items. The authors are commendably transparent that the “known” signal is largely carried by surface cues once name form and temporal expressions are controlled.
The audit is restricted to a single version of a single closed model (jev-1.13.0, late September 2026), so generalizations to decision models or LLMs remain speculative. The knowledge boundary is defined as an observational accuracy change point in dated news, not a verified training cutoff, and topic or style shifts could confound the temporal contrast.
Post-hoc exploratory controls—look-alike names, date removal, recalibration bounds—reuse the same dated-news panel, raising multiple-comparison concerns and leaving the temporal findings unconfirmed on a fresh panel. In addition, date removal can alter proposition identity, so the reported response changes do not cleanly separate superficial date-reading from temporal knowledge retrieval.
The evidence strongly supports the claim that confidence fails as a readout of missing knowledge. The authors fairly distinguish their work from concurrent evaluations of Jev as a judge: they ask not whether confidence predicts correctness, but whether it reports why uncertainty exists. The comparison to the open GLiNER2.5-Decide model—nearly uniform under no information—effectively rebuts the objection that all decision models must be salience-driven. However, the dismissal of “known” as showing “no demonstrated advantage” over confidence after cue removal is slightly overstated: the paired difference is $+0.02$ with dates removed, and the $90$% interval ($-0.01$, $0.06$) does not formally demonstrate equivalence, so the null might more cautiously be described as unconfirmed.
Reproduction is partially blocked because Jev is a closed API: the authors requested jev-latest and obtained jev-1.13.0, but future access to that exact version is not guaranteed and the vendor acknowledges that answers to related questions need not be coherent. On the positive side, the code is publicly available, fifteen public datasets are pinned to specific revisions with listed licenses, and experimental ledgers report dates and call counts. Hyperparameters are sparse but explicit: each cell was queried three times and the mean distribution was analyzed. Because several key controls were exploratory and run after the main studies, independent replication would ideally need a fresh temporal panel and a preregistered protocol for surface-cue controls.
Decision models return probabilities intended for routing, abstention and automated action. Calibration makes those probabilities useful on average, but does not establish whether low confidence reflects chance or missing knowledge, nor whether confidence falls when a model moves beyond what it knows. We audit this distinction in Jev, a decision model, with over 15 public datasets and 6 generated task families, with paired interventions that vary the information supplied for a fixed item. Jev's confidence is calibrated on familiar closed-choice tasks but fails as an indicator of missing knowledge: with no answer-relevant information it assigns up to 0.80 to a salient option, and on news beyond an observed knowledge boundary it exceeds accuracy by 0.21--0.33, a gap that recalibration on earlier months does not close. Targeted yes/no questions give sharper readouts of the case: whether an outcome is settled (AUROC 1.00) and whether the evidence suffices (0.95, against 0.85 for confidence on the same items). Asking whether Jev knows the answer appears to flag fabricated entities and post-boundary news (0.91), but with realistic names or with dates removed it shows no advantage over answer uncertainty. Black-box knowledge audits therefore need explicit controls for surface cues. Code: https://github.com/Syntheme/beyond-answer-confidence.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.