Compliance Guard Models Ignore the Rule They Are Meant to Check

A new study introduces a benchmark that crosses two governing rules with two scenarios so that neither alone predicts the correct label, and tests this against current compliance guard models and activation probes used to flag regulatory violations in language model outputs. Across every guard and activation probe tested, detection accuracy stays unchanged when the governing rule is deleted, permuted, or substituted for its opposite, a failure the authors call rule blindness. This includes a policy-conditioned guard that correctly cites the governing clause in its output yet barely changes its verdict when that clause is swapped for a permissive counterpart. Step by step reasoning was the only approach among those tested that escaped the failure. The authors also propose a training-free activation readout, the Internal Compliance Score, calibrated from ten labelled pairs, and report that it does not beat a pre-registered baseline criterion, with a simple bag-of-words model matching its generalisation exactly; it still proved useful for auditing four deployed guard models, an 8B zero-shot judge, and thirteen benchmarks at low cost, though gains from using it to rank candidate responses disappeared under an adaptive white-box attack. The protocol and benchmark are being released so rule blindness can be tested in future probe and guard claims; this is laboratory research with no disclosed real-world incident yet tied to it.

rss · arXiv cs.AI · Aug 18, 04:00

「Background」 Organisations deploying language models increasingly rely on compliance guard models and internal activation probes as automated checks that outputs conform to written rules covering data protection, healthcare, financial regulation, and platform policy. This monitoring is treated as a legal and audit control on the assumption that a detector's verdict actually depends on the stated rule rather than on incidental features of the scenario being judged, an assumption that had not previously been isolated and tested by a benchmark designed to rule it out.

「Who Is Exposed」 Organisations that rely on guard models or activation probes as evidence of regulatory compliance for deployed language models are in scope, particularly where these tools are cited to auditors or regulators as automated controls. To check exposure, teams should identify whether their compliance monitoring stack uses fast classifier-style guards or activation-based probes rather than full step by step reasoning checks, since the paper found only the latter escaped rule blindness; they should also check whether verdicts have ever been validated against counterfactual rule swaps rather than only against labelled scenarios under the original rule. The finding covers four deployed guard models and an 8B zero-shot judge tested in this study, not the entire market, so applicability to a specific product depends on its detection architecture.

「Mitigation」 No fix is available for the underlying rule blindness in the guard models and probes tested; the authors' proposed Internal Compliance Score also failed its own pre-registered bar against a trivial baseline, so it is not a validated remedy. The released counterfactual protocol and crossed-rule benchmark can be used as a compensating control to test whether an organisation's own compliance detectors actually respond to the governing rule before those detectors are relied on for audit purposes, and step by step reasoning approaches appear more resistant to this specific failure mode based on the tests reported.

Tags: #AI guardrails, #regulatory compliance, #activation probes, #benchmark evaluation, #LLM safety