FlagThis

Abstract

Claw-like AI agents (e.g., OpenClaw) are always-on processes running inside the users environment with persistent access to credentials, files, tools, and external services. To deliver this functionality, they take on system-level responsibilities such as installing packages, maintaining long-lived state, scheduling subtasks, and mediating I/O. This depth of access, together with their widespread adoption, makes the consequences of any security failure far more severe than those of other agents. Despite this, existing evaluation benchmarks focus on the security of the models responses and tool calls, leaving the cross-component failure modes of Claw-like agents largely unmeasured. To address this gap, we adopt a computer-system perspective as a motivating analogy: we treat a Claw-like agent as an agentic computer system whose gateway runtime performs an OS-like mediation role, whose Skills resemble user-installed applications, and whose in-process Plugins resemble loadable extensions that execute with runtime privileges. Each agent component has a classical counterpart whose protection mechanism, refined over decades of classical cybersecurity research, is missing on the agent side. We use this perspective to develop SafeClawArena , a benchmark of 406 adversarial tasks across four attack surfaces (Skill Supply-Chain Integrity, Persistent State Exploitation, Cross-Boundary Data Flow, and Indirect Prompt Injection). Each task is executed inside a containerized replica of a real agent platform with canarymarked credentials and evaluated by automated taint tracking across nine output channels. We evaluated three platforms (OpenClaw, NemoClaw, and SeClaw) and five frontier LLMs. The highest overall attack success rate reaches 70%, and malicious Plugins succeed in 100% regardless of the underlying LLM since they are unhardened. Moreover, although platform-level hardening is effective, its efficacy varies between different LLMs: SeClaw (a streamlined variant of OpenClaw with added security defenses) cuts GPT-5.4s attack success rate from 70% to 22%, in part through a utilitysecurity tradeoff (removing attack-surface features such as the Skill-bundled Plugin loader) rather than purely through active defenses, while Claude-Opus-4.6 already sits near a 22% security floor on every platform and gains almost nothing from hardening. The results expose the inadequacy of current defenses and point to possible directions for the future defense design of Claw-like agents. Code and data are available at https://github.com/sunblaze-ucb/ SafeClawArena .