Programmable Backdoors in Vision-Language Models VLMs
This research identifies a critical vulnerability in Vision-Language Models (VLMs) enabling "programmable" backdoors through an any-to-any caption-control framework. Unlike static backdoors, attackers utilize a heuristic poisoning strategy to instill a general "trigger-as-instruction" rule, decoupling poisoning from target selection. By employing feature-space trigger steganography via norm-controlled perturbations, adversaries can synthesize stealthy visual triggers at inference time to force the model to generate arbitrary, previously unseen target captions. This mechanism bypasses classical defenses—including pruning and fine-tuning—while maintaining the model's original utility on clean datasets, allowing for covert and arbitrary semantic control of multimodal outputs.
Adversarial Clothing and GaP Patches Targeting Clearview AI and Amazon Rekognition
The emergence of Universal Physically Transferable Adversarial Patches (GaP) enables the bypass of black-box facial recognition systems, specifically targeting the computer vision (CV) pipelines used by Clearview AI and Amazon Rekognition. By exploiting vulnerabilities in Convolutional Neural Networks (CNNs) and Transformer-based image classification, GaP patches manipulate physical-to-digital transferability mapping to disrupt feature extraction. This results in significantly higher False Rejection Rates (FRR) and allows users to evade identity matching. The technical vector involves introducing adversarial noise into the physical environment that translates to high-confidence misclassifications within the target model's latent space.