FlagThis

Abstract

Adversarial robustness is commonly evaluated with predefined attack ensembles, such as AutoAttack, at a single perturbation budget and on a selective choice of perturbation norms. We argue this formulation is fundamentally limited. First, robustness perturbation curves may intersect or decay at different rates across models, making single-rankings unstable. Second, current ensembles provide no evidence of optimality, leaving an unknown gap to worst-case performance. Third, fixed attack configurations provide no systematic control over the trade-off between attack strength and evaluation cost. To address these limitations, we introduce a unified evaluation framework based on a comprehensive pool of minimum-norm attacks and robustness-perturbation curves across $L_0, L_1, L_2,$ and $L_\infty$ norms.