FlagThis

Abstract

Large language models (LLMs) exhibit a well-documented gap between latent capability and consistent activation. Czares [1] proposes the _router hypothesis_ : that models _route problems to cached structural patterns rather than deriving answers compositionally_ . That work reports that injecting structural priors via cheatsheets raises performance dramatically on in-distribution data, yet collapses even below the zero-shot baseline when evaluated on an out-of-distribution (OOD) test setand that a more heavily engineered cheatsheet (AN45c) that peaks locally underperforms a simpler predecessor (AN38) under official distribution shift. We ask whether this phenomenon is _cross-domain_ . We reproduce the experimental design of [1] in the domain of _source-code security vulnerability detection_ , evaluating three frontier LLMs (GPT-OSS-120B, Llama-3.3-70B, Gemma-4-31B) across three vulnerability categories (two CWEs CWE-798, CWE-284 plus one non-CWE anti-pattern, N+1) spanning syntactic, contextual, and semantic vulnerability types on a synthetic corpus of 348 labeled pairs (278 training, 70 held-out evaluation), and then transferring the cheatsheet-augmented prompts to real-world CVE (Common Vulnerabilities and Exposures) data from the VUDENC benchmark (CWE-89, CWE-22). Our main findings replicate and extend those of [1]: **(F1)** Structural priors reduce routing failures on synthetic data, lifting the semantic-vulnerability recall from 20.0 % to 100.0 % across all three models. **(F2)** Zero-shot performance degrades along a semantic complexity gradient (syntactic _>_ contextual _>_ semantic) for GPT-OSS-120B, and partially for Gemma-4-31B; Llama3.3-70B shows an inverted pattern on N+1. **(F3)** The same cheatsheets that saturate synthetic performance _amplify_ distribution-shift collapse on real CVE data: for CWE-89, GPT-OSS-120B __ drops from 100 % synthetic F1 to 48.9 % on VUDENC ( 51.1 pp), compared with 61.5 % zero-shot ( __ 29.4 pp). **(F5)** Iterative recalibration of the routing priors using real error patterns produces a v2 cheatsheet that performs _worse_ than v1 on the same real distribution (41.7 % vs 48.9 % vulnerable-class F1 for CWE-89 with GPT-OSS-120B), mirroring the AN45c underperformance relative to AN38 reported by [1]. Together these results provide evidence that the _cross-distribution trade-off surface_ documented by [1] generalizes to code security, and that the router hypothesis is cross-domain. We discuss the implications for prompt-engineering-based detection pipelines and argue that the structural nature of the collapse motivates distribution-aware training rather than prompt calibration as a more promising remedy. Code, cheatsheets (v1/v2), and evaluation scripts are available at `https://github.com/bytepro-ai/bitcoder-v2-research` .