Daydreaming: LLM Skill Stealing

Arxiv pdf 2026-08-01T00:00:00
arXiv Paper — PDF not available. Only the Executive Summary is available here. To read or download the full paper, visit the arXiv abstract page.

Abstract

market sells such capabilities the way software is sold as a service, a setting we call Skill-as-a-Service ( SkaaS ) where the vendor hosts the skill on its own agent and the customer pays per task or by subscription. Software-as-a-service withholds the program and charges for what it computes while SkaaS withholds the expertise and charges for what it judges. Existing vendors already sell hosted access in domains such as law, autonomous medical coding, and security operations, sometimes charged per completed task. Moreover, these vendors treat the hidden skill as a protected asset, as their terms of service forbid reverse-engineering the service or using outputs to build a competing one, and stealing attacks motivated by this kind of commercial market have appeared for single prompt settings such as domain-specific system prompts. Agent skills bundle instructions, reference data, and executable helpers that let a general agent perform specialized tasks. Hosted providers can keep these files secret while selling access to task results, making the skill itself a valuable target. Existing disclosure defenses can block requests that ask for the skill or reproduce its text, but they cannot block customers from submitting the ordinary tasks the service is built to complete. We present Daydreaming , an executiononly attack that steals a multi-file skill through black-box task interactions. The victim is never asked to reveal the skill or grade a reconstruction. Instead, Daydreaming adaptively creates crafted tasks whose results distinguish possible hidden behaviors. It tests individual behaviors, uses attacker-controlled shadow agents to choose a design, and completes each file using stored victim results and local execution checks. We formalize three nested threat levels of access as Differential, Trace, and Output, and focus on Output, where the attacker sees only the final response and returned files. Across 7 skills and 4 victim models, Daydreaming recovers 86 . 8% of original skills capability at Output, outperforming SigLeak by almost 4 . It produces installable skills using a median of 32 victim calls per skill even with disclosure defenses enabled. These results show that hiding skill files and filtering direct disclosure do not, by themselves, prevent functional reconstruction through normal use.

Loading executive summary...

LINK COPIED TO CLIPBOARD