FlagThis

Abstract

show near-native performance alongside strong resistance to PCIe snooping and HBM dump attacks, demonstrating that inference-time model exfiltration can be made substantially less practical in real-world deployments. Large foundation models deployed on third-party and shared accelerator infrastructure face a practical risk of model exfiltration that existing defenses do not fully address [30]. In common serving deployments, the model provider controls the VM or bare-metal serving stack and trusts the drivers and runtime of its own accelerators, but does not control the surrounding hardware substrate: the hostGPU interconnect, the accelerator fabric, and neighboring infrastructure components remain outside the tenants trust boundary. Prior work has demonstrated that both attack surfaces are practically exploitable. Hermes [39] shows that a passive observer with access to the PCIe bus can achieve lossless DNN reconstruction from transferred packets alone. TunnelS [36] shows that a non-participating host with driver-level access can exfiltrate HBM contents at high throughput by exploiting PCIe underutilization, without interrupting inference. Beyond these direct hardware paths, co-tenant VMs reachable over the shared frontend or management network can access memory-mapped interfaces or improperly segmented RDMA regions without requiring physical co-location.