AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack
LLM email agents increasingly handle sensitive inboxes, yet conventional phishing benchmarks only test whether models classify emails as malicious, not whether they actually obey harmful requests inside them. This paper introduces AgentPhishing, a benchmark that scores both detection failure and compliance with malicious requests, and PhishOpt, an adaptive optimizer that evolves target-specific email-generation skills using agent feedback under a realism constraint. The core finding is that agents which safely handle static, pre-existing phishing emails can be driven to near-zero detection and high compliance when attacks are adapted to their behavior, exposing a major gap in current safety evaluation.
The paper makes a compelling case that static phishing accuracy is a poor proxy for agent safety, but its strongest claims rest on an optimization objective with a known inflation bug, small-batch evaluation, and model-family overlap between judge and optimizer. The AgentPhishing benchmark design is sound in principle, yet the empirical results deserve scrutiny because the most vulnerable agents sit at the edge of mode-collapse thresholds and the reported scores are mild over-estimates when detection falls to zero.
The cross-agent heatmap is the most persuasive result: susceptibility varies far more by target agent than by source skill, with Sonnet showing perfect resistance while Qwen and Kimi collapse under the same inputs. The persuasion ablation is also cleanly executed, demonstrating that replacing Commitment and Consistency framing with Authority plus Urgency restores perfect detection and zero compliance on two targets. The paper is also refreshingly candid about its limitations.
A methodological flaw inflates the adversary objective for the most vulnerable agents. The F1 term in $J=0.5(1-D)+0.3C+0.2(1-F_1)$ becomes undefined when detection $D$ falls to zero, and the implementation coerces the missing value to $0$, awarding the full $0.2$ credit. The authors admit this produces a mild over-estimate for qwen3.7-plus at its best epoch (Appendix F.2). Separately, diversity diagnostics reveal that the strongest attack corpus for qwen3.7-plus clears the mode-collapse threshold by only $0.0063$, meaning one more optimization step could plausibly have crossed into template memorization (Appendix E). The realism judge is Opus 4.8, the same family as the optimizer, raising questions about evaluator independence.\n\nThe evaluation scope is also narrower than the safety framing suggests. All experiments use small batches (~35 scam emails per skill) and score only action selection and drafted replies, not actual external execution, so measured compliance does not establish completed real-world harm. The held-out additional-agent set mixes different model families without controlling for system-prompt or tool-access differences, making it unclear whether the universal skill exposes a general vulnerability or simply targets weaker configurations.
The comparison between pre-existing and adaptive attacks is stark and well-supported: all six agents perfectly detect and refuse every public-corpus phishing email, yet four of nine held-out agents drop below 10% detection and above 45% compliance under the universal skill. However, the paper does not run AgentBait head-to-head against PhishOpt on a shared target pool, leaving the practical delta between predefined and adaptive strategies unquantified. The persuasion analysis is correlational, and the authors correctly caution that pooled associations do not establish independent causal effects; the ablations on two targets provide helpful evidence but do not generalize to all agents.
The authors release the evaluation pipeline, evolved skill files, metric definitions, and analysis code, but withhold raw generated emails as a friction-raising misuse mitigation. Hyperparameters are itemized (model versions by role in Table 1, objective weights, and prompt constraints), and the full generator prompts appear in Appendix F. Major blockers to independent reproduction include access to future-dated or unreleased model versions (GPT-5.4, Claude-Sonnet-5, etc.), an unstated total API query budget, and lack of disclosed random seeds or retry logic beyond a passing mention of retry hardening.
Computer-use agents increasingly handle private information and act on users' behalf across desktop and web applications, including email. Phishing therefore poses privacy and security risks that depend not only on whether agents recognize malicious messages, but also on how they respond. Existing phishing datasets were developed primarily for conventional detection, leaving agent susceptibility to persuasive malicious requests insufficiently characterized. To expose vulnerabilities that pre-existing datasets may miss, we introduce AgentPhishing, a benchmark for evaluating whether persuasive phishing attacks can evade detection and induce LLM email agents to comply with malicious requests. We develop PhishOpt, which optimizes reusable email-generation skills against individual agents under fixed benchmark constraints and a realism constraint. We analyze the generated emails using Cialdini's persuasion principles to identify strategies associated with agent failures and use these findings to construct a universal generation skill. Our evaluation shows that agents can handle pre-existing phishing emails safely yet remain susceptible to attacks adapted to agent behavior, while in an AgentPhishing evaluation of nine agents, four exhibit phishing-detection rates below $10\%$ and malicious-request compliance rates above $45\%$. These results highlight the limits of inferring agent safety from conventional phishing evaluations and the need to assess how agents respond to persuasive malicious requests.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.