Abstract

Large language models (LLMs) embedded in production systems face prompt injections, trojans, and manipulation of automatic quality metrics. This thesis develops a set of models, methods, and algorithms for evaluating and improving the robustness of LLMs to adversarial input sequence variations: prompt injection, trojans (backdoor attacks), and attacks on agentic LLM systems. We propose _Rstab_ ( _f_ ), a formal metric of generative robustness based on bounded divergence measures, in particular the JensenShannon divergence, between per-step output distributions under small input perturbations. For localized attacks, we prove that the vulnerability of a deterministic decision operator _h_ is bounded by its non-robustness: _V_ ( _h_ ) __ 1 _ Rclass_ ( _h_ ), where _Rclass_ ( _h_ ) is the expected probability that _h_ preserves its decision under small input perturbations. For non-localized attacks, we propose an empirical calibrated model. For LLM-as-a-Judge systems, we separate instructive and content attacks and develop ASA, an adaptive evolutionary attack that combines token-level mutations with semantic paraphrasing. Under black-box access, ASA reaches an attack success rate (ASR) of up to 73.8%, and the transfer success rate between open models reaches 62.6%. On Trojan Detection Challenge 2023 data for Pythia-1.4B, we quantify an asymmetry of trojan detection: surrogate triggers reach a reverse-engineered attack success rate (REASR) of __ 0 _._ 99, while the recall of the true triggers is __ 0 _._ 17 against a baseline of __ 0 _._ 14. On SaTML CTF 2024, we systematize four classes of bypasses of multi-layer defenses. Committees of 57 heterogeneous models reduce the ASR for Gemma-3-4B by 4755 percentage points, from 73.8% to 19.3% with 7 models. The Condorcet jury theorem justifies the growth of committee accuracy when the members err independently and each has an accuracy above 0.5. For Gemma-3-4B, which does not meet this condition, the gain comes from the heterogeneity of the committee. Multi-layer defense reduces the ASR from 90% to 1525%. For agentic systems based on the Model Context Protocol (MCP), we propose AttestMCP, which attests tool calls with HMAC-protected packets at an overhead below 0.1 ms per call, and the Commit Boundary isolation pattern. On the MCPBench benchmark of 847 scenarios, our defenses reduce the average ASR from 53.7% to 12.4%. The methods are implemented in the registered software suites JudgeGuard and TrojanArmor and in the MCPSec module. JudgeGuard is deployed in production at Viasat.