The Limitations of LLMs in Autonomous Vulnerability Discovery and Prioritization

Current research from IBM, the NDSS Symposium, and Boston University's PEAC Lab indicates that Large Language Models (LLMs) are fundamentally insufficient for autonomous vulnerability discovery and risk-based prioritization. While LLMs demonstrate pattern recognition capabilities, they suffer from high false-positive rates and a systemic lack of architectural context, preventing them from understanding how vulnerabilities interact with specific deployment environments. This creates an "automation paradox," where the volume of unverified LLM-generated findings increases the manual verification workload for Application Security (AppSec) professionals. Furthermore, models demonstrate a critical failure in reasoning about actual exploitability, making them unreliable for determining the real-world risk of identified security flaws.

• Research Overview: The LLM Capability Gap
  • Disconnect between pattern recognition and nuanced security reasoning.
  • Inability to incorporate environmental or architectural context into findings.
  • Significant failure in predicting actual exploitability and risk likelihood.
• Technical Methodology: Evaluation Frameworks
  • IBM Research: Development of comprehensive benchmarks for security reasoning.
  • NDSS Symposium (2026): Technical analysis of LLM-based detection reliability.
  • BU PEAC Lab: Rigorous testing of LLM reasoning within security contexts.
• The Automation Paradox: AppSec Operational Impact
  • High false-positive rates driving increased manual verification toil.
  • LLM outputs acting as a noise multiplier rather than a force multiplier.
  • Deterioration of triage efficiency due to unverified AI-generated alerts.
• Key Findings: Reasoning & Exploitability Failures
  • Models struggle to distinguish between theoretical flaws and exploitable vectors.
  • Lack of semantic depth required to assess real-world runtime impact.
  • Failure to align with established exploitability prediction models.
• Industry Implications: Defensive Response
  • Necessity for robust "Human-in-the-loop" (HITL) verification protocols.
  • Requirement for context-aware models integrated with deployment metadata.
  • Shift in strategy from autonomous discovery to AI-augmented triage.