FlagThis

Abstract

Nowadays, the autonomous execution of cyberattacks capable of causing substantial realworld harm is widely regarded as one of the critical red lines that frontier AI systems must not cross in many international consensus statements on AI safety. Within this broader red-line scenario, autonomous penetration represents a core enabling capability and subtask: the ability of large language model (LLM)-powered AI systems to independently conduct adversarial operations against a target server without human intervention, identify and exploit security vulnerabilities, and obtain unauthorized access or control, thereby potentially compromising the confidentiality, integrity, and/or availability of computer systems. A growing body of work, including evaluations conducted by OpenAI and Anthropic, has sought to assess the autonomous penetration capabilities of LLM-powered AI systems without human intervention. However, existing evaluations often employ opaque methodologies, rely on unrealistic or overly simplified penetration-testing scenarios, or provide LLMs with excessive prior knowledge and task-specific guidance. As a result, they do not accurately capture the extent to which modern AI systems can autonomously perform this core capability within broader high-impact cyberattack scenarios. To address these limitations, we construct a new autonomous penetration evaluation framework consisting of two components: target servers and agent scaffolding. Specifically, on the target-server side, we design two levels of target environments based on the number of secure services without known vulnerabilities deployed alongside a vulnerable service: Tier 1 (one secure service) and Tier 2 (three secure services), resulting in a total of 300 target servers. Meanwhile, the agent scaffolding adopts a general-purpose agent architecture equipped with a set of general-purpose cybersecurity tools and is provided only with the IP address of the target server, without any target-specific prior knowledge or handcrafted penetration instructions. We evaluate 19 open-weight and proprietary LLMs, and our experimental results demonstrate that current models achieve penetration success rates ranging from 10.7% to 69.3%. Moreover, we observe that autonomous penetration capability continues to improve alongside advances in overall model capability, highlighting the increasingly severe misuse risks posed by highly capable frontier models. We hope our findings serve as a timely warning about the urgent need to better understand and rigorously evaluate the risks posed by frontier AI systems, as well as the importance of establishing effective governance mechanisms for the autonomous penetration capabilities of such systems.