LLM Security Fine-Tuning Evasion
Abstract
LLMs are increasingly fine-tuned for specific security classification tasks, and the resulting models are typically evaluated on held-out examples from the same distribution as their training data. We show that this evaluation practice misses a class of vulnerability introduced by fine-tuning itself: a model trained on a corpus where specific indicator tokens are strongly correlated with maliciousness can learn token-level semantics that preserve canonical accuracy while failing under behavior-preserving transformations. In our PowerShell setting, these transformations include alias substitution, command reconstruction, string construction, execution indirection, and case mutation. To understand the mechanism, we study a natural base/fine-tuned model pair, Llama-3.1-8B-Instruct and Foundation-Sec-8B-Instruct, on matched PowerShell classification cohorts. Using causal interventions, we localize the classification circuit to a small late-attention bundle and show that this route is inherited from Llama rather than created by fine-tuning. Fine-tuning concentrates and semantically specializes this inherited structure, adding associations between canonical command indicators and malicious classification. Those associations improve baseline classification behavior but create transformation-sensitive attack surfaces: a three-tier evasion benchmark finds Foundation-Sec misses across multiple transformation categories that Llama does not share. We derive a practical pre-deployment monitoring method that combines a linear probe from base-model activations at the classification boundary with an indicator-token sign test to cheaply track post-fine-tuning drift, identify command families where canonical indicators change role, and prioritize targets for red-team variant generation. More broadly, these results caution against treating small task-specific fine-tunes as straightforwardly safer security classifiers: specialization can convert inherited model structure into brittle indicator rules that preserve held-out accuracy while expanding the evasion surface. Robust AI-enabled security will require specifying the full transformation space of the task and monitoring semantic drift through fine-tuning.