Are AI Decision Models Biased? A Case Study of Jev
Do fast, structured AI decision models exhibit the same cognitive and social biases that plague human intuitive judgment? This study subjects Jev—a commercial "System One" model that returns probabilities over fixed choices rather than generating text—to 41,500 trials across cognitive reflection, classical bias paradigms, and a demographic pricing task. The core contribution is showing that pristine accuracy on famous benchmark items can collapse under minor rewordings or answer-order changes, and that probability scores often shift even when the top choice does not, which matters deeply when those probabilities drive downstream automation.
This preprint delivers a carefully executed, large-scale stress test of a proprietary decision model. Its central finding—that Jev passes famous bias items while failing structurally similar variants and showing strong answer-order and probability-level sensitivities—is methodologically important and practically urgent. The work is somewhat bounded by its reliance on a single model version and a narrow set of scenarios per bias, but the authors are transparent about these limits. Overall, it makes a convincing case that evaluating structured-decision models only on top-line accuracy can dangerously overstate their reliability.
The paper's sharpest strength is its insistence on looking beyond the selected answer to the full probability distribution and to subtle task variations. The conjunction robustness check is a standout: showing that Jev answers the canonical Linda problem correctly in all 500 calls but gets an adapted version wrong in every single call, and then extending this to 30 scenarios where error rates swing wildly by wording and order, powerfully undermines any assumption that the model understands the underlying rule.
The biggest empirical limitation is that most paradigms rest on just one original and one edited prompt, so claims about general bias susceptibility are extrapolated from tiny samples. The authors acknowledge this directly. A second concern is the lack of any baseline comparison—whether against standard LLMs, earlier Jev versions, or human respondents—making it impossible to gauge whether these fragilities are specific to Jev or generic to the interface. Finally, the continuous indices used in anchoring and social bias rely on Score outputs that TypeSafe itself warns have limited numerical calibration, raising questions about the interval validity of the effect sizes.
The evidence supports the paper's main claims for the specific prompts tested, and the authors generally contextualize their results fairly against prior work. They correctly note that Deußer et al. (2026) found unchanged accuracy under option rotation in their MMLU probe, while their own cognitive-bias tasks show clear order effects—especially in the zero-risk edited task, where selecting the larger-reduction program dropped sharply when the candidate list was reversed. The absence of a head-to-head replication on identical items across model families, however, limits the strength of comparative conclusions.
The authors commit to openness by stating that all data and code is available on GitHub, and they document the exact model version, date range, and response counts. Reproduction is nevertheless hindered by Jev being a proprietary, paid API with undisclosed sampling parameters such as temperature, and by the fact that original and edited variants were collected in separate runs, leaving open possible confounds from service-state drift. Independent replication will require both funding for API calls and careful control for run-level variation.
Do AI models designed for fast, structured decisions exhibit the cognitive and social biases observed in human judgment? We evaluate Jev, a model described as a System One decision-maker, using 41,500 responses across cognitive reflection tasks, five classical cognitive-bias paradigms, and a pricing experiment testing ethnic and gender bias. At the level of selected answers, the results are largely encouraging. We confirm previous findings that Jev performs extremely well on cognitive reflection problems. We also observe that it generally avoids the characteristic choice patterns associated with several well-documented human biases, and exhibits only small differences among demographic groups in the tested pricing scenario. However, this apparent robustness conceals important vulnerabilities. Its judgments are sensitive to irrelevant numerical information and the ordering of response options. Moreover, several cognitive-bias manipulations substantially affect returned probabilities even when selected answers remain unchanged. Most strikingly, Jev answers a well-known probability problem correctly in every response but violates the same underlying probability rule in every response to an adapted version. We investigate this discrepancy using 15,000 additional responses across 30 conjunction tasks. Jev violates the rule in 54\% of responses, with accuracy varying dramatically across formulations and answer orders. These findings suggest that evaluating decision models solely on familiar benchmarks and selected answers may substantially overstate their reliability in practical applications. They also identify vulnerabilities for developers to address and practical considerations for users integrating such models into automated decision-making systems.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.