FlagThis

Abstract

IoT firmware vulnerability detection remains challenging due to the heterogeneity of firmware ecosystems, resource-constrained platforms, and limitations in existing benchmark quality. Many available datasets are synthetic or generalpurpose and do not provide human-verified, contaminationscreened annotations. As a result, evidence regarding cross-corpus generalization across training data sources, model architectures, and curriculum strategies remains limited. To address this gap, this paper introduces IoTVulBench, a human-verified benchmark for cross-corpus firmware vulnerability detection. IoTVulBench-Core was constructed from GitHub repositories, validated by three expert reviewers, and evaluated using a contamination-screened held-out target across five model architectures, two tuning methods, and three curriculum strategies, together with ensemble, distillation, and robustness analyses. Models trained on IoTVulBench achieved the highest Matthews Correlation Coefficient (MCC) among the single-source datasets where others were matched by random undersampling, reaching 0.58 compared with 0.44 for PrimeVul and 0.39 for D2A. Staged curriculum learning further increased MCC to 0.69, while a diversity-optimized ensemble achieved 0.73, representing a 0.42 MCC improvement over the strongest reference comparator, a static analyzer with an MCC of 0.31, and a 0.29 MCC improvement over the strongest single-source training dataset, PrimeVul. At a 0.5% false-positive rate, the model missed only 21% of vulnerabilities, compared with 71% for the strongest reference comparator. It also retained 86% of its performance under identifier renaming while demonstrating strong calibration and largely faithful explanations. These findings indicate that domain-matched training data and curriculum design, rather than model scale alone, are key drivers of generalization in firmware vulnerability detection. The results provide both a benchmark for future research and deployment-ready configurations for practical IoT security applications.