Simulated Control Evals
Control evaluations test whether safeguards can stop a deliberately harmful AI agent, but they typically run in separate settings that may not resemble real deployment. This paper introduces simulated control evaluations (SCEs), which use a language model to generate tool responses in place of live execution, allowing evaluators to test attacks on a deployment's own tasks and codebases without causing harm. The central finding on LinuxArena is that SCEs predict safety metrics far more accurately than executing attacks in the mismatched BashArena control setting, suggesting that deployment-specific task coverage can outweigh the loss of execution fidelity.
The paper makes a compelling and practically important argument that deployment-specific simulation outperforms mismatched execution for control evaluation transfer. On LinuxArena, SCEs achieve an $\mathrm{RMSE}$ of 3.6 percentage points versus BashArena's 10.1 at a 5% audit budget, and they match executed behavioral distributions with over tenfold lower $D_{\mathrm{KL}}$ divergence. Still, the result is demonstrated on one benchmark pair, relies on a trusted simulator, and leaves open how the accuracy generalizes to deployments that differ substantially from LinuxArena.
The quantitative gains are large and multi-faceted: SCEs explain $78\%$--$85\%$ of between-environment variance in attack success, monitor scores, and safety, and they do so with fewer than $5\%$ as many task-pair draws as the deployment set. The behavioral-transfer framework—clustering attack strategies, attacker reasoning, and monitor reasoning and comparing distributions via forward KL—is a meaningful methodological advance beyond aggregate metrics. The monitor-briefing experiment also demonstrates concrete utility; simulated strategies improve detection of held-out executed attacks more than BashArena strategies and approach the gain from same-environment execution.
The most serious limitation is the trusted-simulator assumption; the authors explicitly warn that "an untrusted simulator that attempts to manipulate the results of the evaluation could collude with the attack policy to conceal successful attacks and inflate estimated safety," yet they provide no implemented solution. Aggregate accuracy also masks severe outliers: "The five largest errors account for about 50% of the summed squared error across the 92 side tasks." Finally, the authors undermine their own discriminability diagnostic by showing that "optimizing against this diagnostic has not improved transfer: self-critique made trajectories harder to distinguish but worsened control metrics," which leaves simulator improvement without a reliable proxy.
BashArena serves as a fair but forgiving baseline because it differs in both tasks and environments, so beating it primarily shows that deployment-task specificity matters rather than that simulation universally beats execution. The variance-decomposition methodology is sophisticated—correcting for run-to-run and task-sampling variance—and the headline variance-explained figures are striking. However, these estimates hinge on an attack-success judge that "overestimates their average success rate by 3.7 percentage points," and the paper notes that in some environments "the judge overestimates success by nearly as much as the combined simulator and judge do," making it difficult to disentangle simulator error from judge bias.
There is no mention of released code, data, or model checkpoints in the provided text, which is a major obstacle to independent verification. Reproduction is further impeded by the use of model snapshots such as "the simulator is gpt-6-astra" and "the attack policy is gpt-5.6-sol at medium reasoning effort," which appear to be internal or future versions and may not be publicly accessible. Hyperparameters like reasoning effort, trajectory length limits, and audit budgets are reported, but exact prompts, codebase snapshots, and the held-out task-pair splits are not provided, so teams outside Redwood Research would face substantial friction in replicating the LinuxArena SCE pipeline.
Control evaluations measure whether safeguards can prevent an AI model from causing undesired outcomes even when it is intentionally trying to do so. These evaluations typically use purpose-built control settings and extrapolate their results to a model's deployment. However, the effectiveness of safeguards in deployment can depend on features of environments and tasks that control settings do not represent. To broaden where control evaluations can be run, we introduce simulated control evaluations (SCEs), which use a language model to simulate tool responses in place of an environment. We evaluate SCEs on LinuxArena, where execution provides ground-truth outcomes. We compare SCE prediction errors with those from a separate control setting, BashArena, and with variation in control metrics within LinuxArena. For independently sampled evaluation and deployment task sets, SCEs predict safety with a root mean squared error of 3.6 percentage points across five monitors at a 5% audit budget, compared with 10.1 for BashArena. SCEs closely reproduce distributions of attack strategies, attacker reasoning, and monitor reasoning, with over tenfold lower KL divergence from execution than BashArena. SCEs explain 78-85% of the between-environment variance in attack success, monitor scores, and safety. This accuracy is sample efficient: SCEs predict safety with less error than BashArena using fewer than 5% as many task-pair draws as the deployment task set. Strategies discovered through simulation also improve monitor performance on held-out LinuxArena attacks, outperforming BashArena and approximately matching improvement from executed strategies. These results suggest that deployment-specific SCEs can be more predictive than control evaluations executed in separate control settings.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.