FlagThis

Abstract

Adversary emulation plans specify multi-step attacker procedures at the level of MITRE ATT&CK techniques, privilege requirements, and observable telemetry. Translating such plans across operating systems is necessary for cross-platform defender evaluation, and large language models (LLMs) make automated translation tractable. They also introduce a qualityassurance problem: a translation that renames tools while retaining source-platform constructs yields no usable coverage for defenders of the target platform. Binary question-based scoring tends to overestimate how faithful these translations are, because it measures countable properties rather than structural, observable, or rule-level equivalence. Graph-Based Structural Evaluation (GBSE) addresses this gap. Each procedure is modelled as a directed attributed graph, and fidelity is computed as a normalized Graph Edit Distance (GED) across four progressively stricter node-matching layers: technique (L0), tactic (L1), telemetry class (L2), and Sigma logsource (L3). Applied to the full 29-step ALPHV/BlackCat Windows __ Linux plan, with a genuine native-Windows control reconstructed step-for-step from the conversion record and a Linux variant taken unmodified from the LLM output, the framework finds: technique and tactic structure is preserved across the OS boundary (GED=0, Simstruct=1 _._ 000); telemetry fidelity drops to Simstruct=0 _._ 897 (GED=3), driven by three steps that emit an unmapped observable class or drift in telemetry; and independent Sigma-layer matching recovers to Simstruct=1 _._ 000 across all 29 steps. Every state classifies as Medium Fidelity (best composite _S_ =0 _._ 674), and the deployment gate ( _S_ 0 _._ 80, requiring technical realism __ 0 _._ 990 against the measured 0 _._ 43) is unreachable at current evaluation quality. The layer scores are numerically identical to those obtained under an earlier same-procedure design, which confirms that the layers decompose cleanly along the OS-abstraction axis. The framework includes a bipartite-GED implementation, a telemetry-intent parser that derives structured observable classes from free-text annotations, and a validated library of 49 Sigma detection rules (19 Linux, 30 Windows) that gives complete ATT&CK technique coverage of the procedure and passes the Sigma specification validator with zero findings. A supplementary analysis recovers a genuine technique-level divergence (for example RDP-based external access reassigned to unencrypted exfiltration, and credential-store access reassigned to remote-system discovery) that a procedure-aligned view necessarily suppresses. All numerical results in this paper were reproduced from the reference implementation and asserted against the recorded pipeline outputs.