arXiv (Computer Science - Cryptography and Security) • 2h
HexStrike-AI: Evaluating the Limits of LLM-Driven Security Tool Orchestration
HexStrike-AI utilizes the Model Context Protocol (MCP) to orchestrate over 150 cybersecurity tools, enabling LLM agents to perform autonomous penetration testing. Research utilizing the picoCTF benchmark demonstrates a solve-rate increase from 55.4% to 72.0% through targeted tool refinements. However, significant performance variance (2.1x) persists between different client implementations of the same model, indicating that orchestration logic is as critical as model reasoning. While augmenting capabilities, this framework introduces systemic risks, including the potential for autonomous zero-day discovery and the risk of agent hijacking, where the orchestration layer is compromised to execute malicious payloads.