Narrative-Wrapped Prompts Bypass Guardrails in Small LLMs, Study Finds

A study analyzed the latent activation trajectories of three small language model families, Phi-3, Qwen2.5, and Gemma-2b, under adversarial prompts that wrap harmful requests in benign narrative framing such as creative writing. The researchers identify an 'Intent Horizon', a depth of roughly 15-20% of total layers, at which the model's early representation of harmful intent collapses into a representation indistinguishable from a safe query once the request is recast as fiction. At later layers the camouflaged attacks are reported to evade detection by standard classifiers at a rate below 20%, while early-layer activations retain a detectable 'harm signature'. The paper proposes a probing defense, Latent Intent Verification, and reports it outperforming standard guardrails by 20-50% across the tested architectures on the PKU-SafeRLHF dataset, without retraining the underlying model. The work is a research study on small open-weight models; it has not been shown to generalize to larger production LLMs.

rss · arXiv cs.AI · Aug 24, 04:00

「The assumption being tested」 Most deployed LLM safety stacks rely on refusal-based alignment and input/output guardrails that inspect a prompt or completion at the surface level rather than the model's internal representations. This is trusted because refusal training is comparatively cheap to apply and has visibly reduced compliance with directly stated harmful requests, but it assumes the model's internal sense of 'this is harmful' persists through generation rather than being reframed away by context.

「Who this affects」 The demonstrated attack and defense apply specifically to the three small open-weight model families tested, Phi-3, Qwen2.5, and Gemma-2b, under a lab evaluation using the PKU-SafeRLHF dataset. Organizations deploying these or architecturally similar small models with only output-side or refusal-based guardrails, and no inspection of internal activations, are the ones directly in scope; teams should check whether their safety layer relies solely on classifying final text rather than probing intermediate representations. The finding does not establish that widely deployed large production LLMs share the same layer-depth collapse behavior, so exposure outside small open models is unconfirmed.

「What reduces the risk」 The paper's proposed Latent Intent Verification probe is reported to reduce bypass rates substantially without retraining, but it is a research prototype validated on a limited set of small models and one dataset, not a deployed or independently confirmed fix. Until broader validation exists, teams relying on refusal-only guardrails for narrative or role-play style prompts should treat that layer as insufficient on its own and consider supplementary activation-based or context-aware review where feasible.

Tags: #LLM safety, #jailbreak, #alignment, #interpretability, #guardrail bypass