FlagThis

Abstract

metadata to be a harmless productivity assistant while containing prompt injection, credential leakage, data exfiltration, or insecure command execution within its instruction or script layers [18, 19]. Conversely, a skill may claim to support a specific capability in its description, yet fail to implement it within its instructions or resources. This reflects a broader documentation-implementation consistency problem, where natural-language descriptions can diverge from the executable or procedural artifacts they are intended to summarize [20, 28, 36]. In Agent Skill packages, such inconsistency can be described as _cross-layer misalignment._ In this case, user-facing claims may diverge from the behaviors supported, requested, or enabled by the underlying instructions, resources, and executable components. Large language model (LLM) agents are increasingly extended through Agent Skills, reusable artifacts that package natural-language metadata, procedural instructions, and execution-time resources for runtime use. As open-source skill marketplaces expand, users and agents increasingly rely on brief metadata to select third-party skills, making it difficult to detect inconsistencies between a skills description and its true behavior, a problem we call _cross-layer misalignment_ . To address this issue, we propose **Progressive LoadingAware Hierarchical Contrastive Learning (PL-HCL)** , an LLMbased framework that detects misalignment by modeling the layered structure of Agent Skills and learning cross-layer consistency. Using a normalized corpus of over 264,000 open-source skills and a human-verified challenge set, PL-HCL improves Macro-F1 from approximately 0.45 for unadapted baselines to 0.870.89 across evaluated LLM backbones. This approach offers an effective screening tool for users and operators, as well as design principles for detecting inconsistencies in layered digital artifacts. This mismatch warrants attention because trust decisions regarding Agent Skills are frequently made before the package is fully inspected or executed. Marketplace descriptions, repository metadata, and skill names influence the selection process. However, the underlying instruction and resource layers that dictate behavior may not be accessible until later loading or execution stages. Consequently, the trustworthiness of Agent Skills cannot be assessed solely by examining the security of a single layer or by evaluating task success rates after execution. Past research has shown that agent and skill evaluation often focuses either on security vulnerabilities in specific artifacts or on downstream task performance, while cross-layer consistency between surface claims and deeper package evidence remains a distinct evaluation target [6, 9, 17, 18, 21]. Instead, it is necessary to determine whether a consistent behavioral contract exists across the different layers prior to execution.