Abstract

Abliteration, the removal of refusal capabilities from large language models by projecting weight matrices orthogonal to an extracted refusal direction, has emerged as a prominent safety concern through its ability to bypass post-training alignment using only a small set of contrastive prompts. We find that existing defenses commonly overlook the cause of abliteration; that is, how _easily_ the refusal direction can be extracted. To hinder this process, we introduce a weight-editing method that obscures the refusal signal by applying rank- _k_ updates to residual stream writer matrices while replacing refusal-inducing activations with random aliases and correcting downstream reader matrices to preserve the models original behavior. On Llama-3-8B, AMRA improves post-abliteration refusal scores by 2 _._ 16 points over the undefended baseline with less than 0 _._ 5 percentage points of MMLU degradation. On Gemma-2-9B, it improves the post-abliteration refusal by 14 _._ 70 points over the baseline while keeping harmful output rates similar to the baseline, albeit at a greater utility cost.