FlagThis

Abstract

Large language models (LLMs) and LLMbased coding agents are now used to generate code from natural-language specifications, yet ensuring such code is both functionally correct and secure remains a challenge. We present DUALGAUGE, the first fully automated framework for jointly evaluating correctness and security of specification-only code generation, supported by DUALGAUGE-BENCH, a languageagnostic benchmark of 307 coding tasks each paired with functional and security tests derived from the same specification. Evaluating 10 representative LLMs across Python, C++, and JavaScript, we find that functional correctness substantially overestimates reliable code generation: even the strongest model remains below 15% joint security-functionality success in every language. Common model-side factors scale, extended thinking, quantization, instruction tuning, and code specializationdo not reliably improve joint performance, suggesting secure-and-correct code generation does not simply emerge from stronger coding capability. Evaluation of 3 leading agentic coding systems (Codex, OpenHands, and Claude Code) shows that iterative scaffolding provides no advantage over direct (LLM-based) generation on specification-only tasks. A qualitative audit reveals failures concentrate at the output contract boundary and in guards that exist but are insufficientpatterns that only joint benchmarking reliably exposes.