Meta Llama Model Family: Internal Safety Probes Fail Against Sophisticated Jailbreaks
Research reveals critical vulnerabilities in the safety architecture of Meta's Llama model family, where adversarial "wrapping" techniques exploit an inference gap between internal model activations and actual content generation. These linguistic wrappers cause internal safety probes to erroneously signal "safety" even as harmful outputs are generated, degrading harmful intent detection AUROC from 0.936 to 0.803. Furthermore, the rise of "abliteration"—the surgical removal of refusal mechanisms from model weights—renders prompt-based defenses and runtime guards like Llama Guard obsolete. To counter these threats, defenders must shift from prompt-level monitoring to forensic weight-level auditing using metrics such as Z-sum thresholding and Weight-Recovery Energy to identify unaligned model artifacts.