Abstract

Static application security testing (SAST) tools help developers find vulnerabilities before deployment. However, false positive reports result in substantial triage efforts. This paper studies whether CodeQL false positives in Java security analysis are recurring, explainable patterns that can be reduced by refining the analysis itself. We run CodeQLs Java security query suite on 167 CVE instances spanning 110 projects, focusing on the ten queries with highest false positive rates. From these queries, we manually review 500 sampled false positive paths/locations and construct a source-level taxonomy of false positive causes. The five high-level categories of false positive causes are Missed Path Constraint or Sanitization (36.6%), Benign Execution Context (29.4%), and Missing Trust Boundary Modeling (27.6%), followed by Imprecise Concurrency Modeling (5%) and Imprecise Sink Modeling (1.4%). Using insights from our empirical analysis, we implement CodeQL refinements that model recurring false positive patterns to enable their detection and filtering at the query level. Our refinements remove 81.8% of reviewed false positives. On the full selected-query dataset, they remove 15.8% of reported paths and locations while retaining 7/8 true positives, showing that many false positives can be reduced directly in the analysis, but also that fixed refinements often depend on project-specific context. To address this generalization gap, we evaluate whether agentic coding tools can adapt refinement patterns to new project-specific contexts. With our refinement patterns as templates to extend, they succeed on 56% and 62% of tasks, respectively, with query compile-pass rates above 90%. However, without this guidance, the success rates drop to 28% for both tools, while the compile rate drops to 30-36%. Overall, our results support a refinement-oriented SAST workflow in which recurring false positives are modeled in CodeQL queries and then automatically adjusted to different project contexts, reducing repeated triage.