FlagThis

Abstract

Persistent personal agents combine long-term memory with access to users external environments, enabling personalized foreground assistance and proactive background execution. This integration also creates a new path to compromise: untrusted external content can be silently written into persistent memory and later reused as trusted state. We study this threat as stealth memory injection, where a remote black-box adversary delivers a single email payload that must induce the agent to inject poisoned memory, remain hidden to its response to the user, and affect future behavior. We introduce WHISPERBENCH, a 108-case benchmark spanning five risk categories and both fact and preference poisoning, that uses a real IMAP/SMTP workflow and an authentic email agent skill, to conduct the full-cycle evaluation on the attack effect of the stealth memory injection. To enable this black-box attack under single-email delivery and no runtime feedback, we propose MEMGHOST, a one-shot payload generation framework that uses an environment proxy to emulate persistent-agent execution and an objective proxy to convert memory adoption and conversational stealth into dense rubricbased rewards, then trains the attacker policy with supervised fine-tuning and reinforcement learning. Across 56 held-out test cases, MEMGHOST achieves 87.5% end-to-end success on OpenClaw with GPT-5.4 and 71.4% on Claude Code SDK with Sonnet 4.6, while also transferring across personal-agent architectures (NanoClaw and Hermes Agent), memory backends (filesystem and vector-based Mem0), and remaining effective against inputlevel, model-level, and system-level defenses. These results suggest that persistent memory can turn ordinary external processing into a practical pathway for long-term agent compromise.