1Password Research: The Risk of FLAWED AI-Generated Patches in ChatGPT and Claude
Research by 1Password, led by Keith Hoodlet, demonstrates that frontier LLMs such as ChatGPT-5.5 and Claude Opus 4.8 frequently generate "Fix-Like Artifacts with Embedded Defects" (FLAWED) when addressing complex vulnerabilities. These models often produce fragile patches that block specific Proof-of-Concept (PoC) inputs rather than remediating the underlying architectural root cause. This failure mode resulted in a 53.9% failure rate during testing, with 49.3% of patches leaving exploitable attack paths open. The research highlights critical risks in automated remediation workflows, where AI-generated fixes may pass syntactic checks while remaining vulnerable to alternative exploitation vectors, potentially creating a false sense of security for CISOs and engineering teams.