FlagThis

Abstract

Our prior work introduced COVA, a synthetically generated multi-turn conversational smishing dataset of 3,201 labeled conversations, establishing baseline detection benchmarks across eight models. While XGBoost with TF-IDF features achieved the best performance (72.5% accuracy, 0.691 macro F1), transformer models underperformedattributed to input truncation and insufficient training data. We present COVAX, an expanded dataset of 10,985 conversations spanning eight elder-targeted scam categories, produced by an improved generation pipeline addressing contamination, label mismatch, stagedirection bleed, and prompt-design failures from the first iteration. Retraining all classifiers on the expanded dataset yields the central finding of this work: Longformer now surpasses XGBoost on all evaluation metrics (79.71% accuracy, 0.7786 macro F1 vs. 78.43% and 0.7563), directly confirming that transformer models require larger conversational corpora to realize their contextual advantages. We additionally document a quality lifecycle including a 12.7 improvement in label correction rate (3.9% vs. 49.8%), an architectural intervention reducing virtualkidnapping artifact rates (67.1% 46.5%), and a per-scamtype outcome analysis showing scam categories modulate results in mechanism-consistent ways. A pre/post-cleanup sensitivity analysis confirms that dataset refinement recovers genuine labelrelevant signal across all three classifier architectures.