Auditing the Hand-Written Inputs of Learned Access Control
Access-control systems increasingly combine mined rules with learned classifiers, relying on a small trusted shield and a region map to route decisions. This paper audits the two hand-built artifacts that sit between verified code and measured ML but are themselves unchecked: the hard-deny set read from policy prose and the region map produced by the mining pipeline. Through exhaustive soundness checks on public ABAC policies, blind LLM elicitation, and an exact decomposition of hybrid routing risk, it shows that prose-derived deny sets are frequently unsound, standard miners emit the wrong kind of rule set for hybrid routing, and the routing benefit cannot be read from a single loss metric. The results matter because these silent failures propagate the authority of a trusted shield into wrong denials and degenerate routing.
The paper exposes a genuine and under-appreciated gap in learned access-control architectures: even a verified shield is only as good as the hand-written deny set it enforces, and a measured learner is only as good as the region map that feeds it. Its central empirical finding—that clauses faithfully transcribing explicit restrictions are unsound at rates of 0.50–0.70—is sobering, but the evidence is limited to two public policies with explicit restrictions and a small panel of elicitors. Likewise, the routing analysis yields a valuable decomposition, yet the conclusion that a study reporting one loss can reach either conclusion underscores that the paper's sharpest contribution is diagnostic rather than prescriptive.
The formalization of hard-deny soundness and the exhaustive enumeration gate (rather than sampling) is rigorous and cleanly implemented, and the provenance stratification isolates the structural scope-mismatch mechanism from elicitation noise. Proposition 3's exact decomposition of the hybrid-versus-learner effect into covered mass times a rule–learner risk contrast is elegant, and the authors are admirably transparent that the proof is one line and claims no depth.
The empirical base for the headline unsoundness claim is narrow: the mechanism stratum is empty on three of five policies, so the 0.50–0.70 rate rests on only 23 clauses from two public artefacts. The LLM elicitor panel is also homogeneous (all Anthropic models), and the synthetic policy used to exercise the full composed system is explicitly acknowledged as supporting no robustness claim. Consequently the failure mode is shown to exist and be reproducible, but not to be prevalent across the policy types the community actually deploys.
Beyond diagnosis, the practical guidance is fragile. On Workforce every selection rule trades false non-grants for false permits, and on E-Document the only configuration that favours rules on both losses survives only one of four learner configurations and a band of noise levels. The practitioner is left with a clear reporting requirement—audit the hand-written inputs—but without a reliably constructive recipe for building a sound deny set or a non-degenerate region map.
The authors fairly acknowledge that the soundness check is a decision problem these tools already solve and that the rare-item connection is not a discovery. The positioning against Margrave, Zelkova and Cedar is accurate, and the negative finding about miners is appropriately scoped to a Xu–Stoller-style reference arm. The evidence is therefore internally consistent, but it does not extend to proving that no miner could emit an abstaining rule set, nor that class-aware selection would remain non-degenerate on policies with balanced base rates.
The authors commit to a strong reproducibility standard: an open analysis package regenerates every number from the public policy files and checks each quoted value against regenerated results. However, no persistent URL or DOI appears in the provided text, and the LLM elicitors are pinned to specific commercial versions (Claude Opus 5.5, Sonnet 5.5, Haiku 5.5) that may not remain accessible. Hyperparameters, splits, and noise levels are otherwise exhaustively reported.
Learned access control is typically deployed behind two hand-built artefacts: a hard-deny set enforced by a small trusted shield, and a region map that decides which requests mined rules answer and which the learner answers. Verifying the shield or measuring the learner validates neither. We audit both on the five public ABAC Lab policies. First, we formalise hard-deny soundness and check it exhaustively over relations of up to 794,250 tuples. Clauses that faithfully transcribe an explicit restriction are unsound at rates of 0.50-0.70 across four elicitors - an author and three blind language-model elicitors - because prose scopes restrictions to a section while the grant relation is a disjunction over all rules: 97% of counterexamples are granted by other rules. Shipped unchecked, these deny sets would withhold 16-78% of everything the policies grant; certified, they withhold nothing. Vacuity, a second silent failure, also occurs. Second, hybrid routing needs a two-class, abstaining rule set that ABAC miners do not emit, and ranking a mixed-effect candidate pool under a cap is one-class in all 24 cells for the two commonest ranking keys. Third, for additive losses the hybrid-versus-learner effect factorises into covered mass times a rule-learner risk contrast that must be reported on both false-non-grant and false-permit loss: on two real policies, most selection rules give opposite verdicts on the two losses, and the one favourable on both holds for one policy, one learner configuration and a band of noise levels. A released artefact regenerates and checks every reported number.
Pick a starting point or write your own. Challenges run in the background, so you can keep reading while the AI investigates.
No challenges yet. Disagree with the review? Ask the AI to revisit a specific claim.