Decoy Defense Poisons Abliteration Attacks on Open-Weight Models

A pre-registered study demonstrates that safety alignment in open-weight language models can be removed in minutes via abliteration, a technique that projects a refusal-mediating direction out of model weights, with no durable release-time defense known to prevent it. The authors propose 'decoy hardening,' which trains models so that once refusal is stripped, most answers to hazardous operational requests become confident but falsified decoys rather than refusals. Tested on seven models from five families (9B-122B, dense and mixture-of-experts), six of seven passed a pre-registered efficacy gate, with 0.51-0.90 of attacked-state responses to held-out prompts being decoys (0.27-0.84 attributable to the defense) while staying within registered benign-behavior and capability budgets; the seventh, smaller model failed the gate. The paper also reports that repeated sampling (K=64 consensus) can reconstruct usable procedures on a meaningful fraction of prompts for the weakest defended model, and that on an external CBRNE-adjacent red-team benchmark the defended 122B model is wrong on 0.82-0.86 of matched-quality answers versus at most 0.10 for the undefended model, with no label-free way to distinguish falsified from correct answers.

rss · arXiv cs.AI · Aug 19, 04:00

「Background」 Open-weight model releases rely on the assumption that safety alignment applied before release provides some durable protection against misuse, since downstream users can inspect or modify the weights but are expected to face friction in removing safety behavior. Abliteration undermines this assumption by allowing an attacker to strip refusal behavior directly from the weights in minutes, which is why the paper treats release-time alignment as fundamentally non-durable and proposes deception as a fallback rather than prevention.

「Exposure」 This concerns organisations that release or redistribute open-weight models and rely on built-in safety alignment as a control against hazardous misuse, particularly for chemical and biological hazard categories evaluated here. Exposure depends on model size and architecture, since the defense was validated on seven models from 9B to 122B parameters across dense and mixture-of-experts designs, with one smaller model failing the efficacy gate; results may not generalize to untested architectures or sizes. The defense also does not address in-context jailbreaks and only protects the initially released defended weights, so any fine-tuning or further modification by downstream users falls outside the demonstrated protection.

「Mitigation」 The paper's own defense, decoy hardening, is the mitigation under test: it does not prevent abliteration but poisons the payoff of a successful attack by causing most post-attack answers to be falsified rather than refused, at measured rates of 0.51-0.90 across six of seven tested models. The authors are explicit that this is a partial, epistemic mitigation with no independent way to separate falsified from correct answers, that repeated sampling can restore some usable information on the weakest model, and that it offers no protection against in-context jailbreaks or against weights altered beyond the initial release.

Tags: #open-weight models, #safety alignment, #abliteration attack, #deceptive defense, #model security