FlagThis

Abstract

Prompt injection is widely treated as a defect to be patched through better training, filtering, or guardrails. We show that, for a broad class of prompted neural architectures, perfect prompt-injection prevention cannot be guaranteed within the shared representational pipeline itself. We formalize prompted systems as Prompted Action Models whose outputs include not only token generation but also control-authoritative actions such as refusal decisions, tool authorization, policy routing, and memory-write operations. We then define a security property, Semantic-Faithful Control (SFC), requiring that control-authoritative behavior depend only on the semantic content of untrusted input, not on semantically irrelevant variations in its representational encoding. Our analysis has three parts. First, we derive a provenance-recovery impossibility result: when trusted instructions and untrusted content are processed through shared representations, the Bayes-optimal error of provenance recovery is governed by the total variation distance between the corresponding representation distributions, and is nonzero whenever those distributions are not disjoint. Second, we show that in standard shared-attention architectures, untrusted content enters control-relevant computation through the same valueaggregation pathway used to determine control-authoritative outputs. Third, combining these facts with a finite-coverage invariance argument, we prove that no mechanism operating solely within the shared representational pipeline can guarantee perfect Semantic-Faithful Control.