FlagThis

Abstract

Selecting appropriate machine learning (ML) configurations for malware detection is a complex, multi-criteria problem: model choice, feature engineering strategy, and update mechanism must jointly satisfy operational constraints that vary substantially across deployment contexts. This paper proposes a Framework for Decision-making to build ML-based Malware detection systems (FDM), a structured methodology that formalises this selection process through the Weighted Configuration Compatibility Score (WCCS), a multi-criteria scoring function that maps five quantifiable operational parameters (platform constraint, resource budget, response latency, update frequency, and detection sensitivity) to ranked recommendations across nine configuration dimensions. To validate the framework, four experiments were conducted on three datasets (a private Windows API call dataset covering eight malware families, the public Malimg image benchmark of 9,339 samples from 25 families, and a public Android static API dataset of 15,036 applications). Key results include: (i) XGBoost achieved the best accuracy-to-resource ratio in binary classification (97.46% test accuracy, < 70 MB RAM), outperforming LSTM and BiLSTM which consumed up to 2.8 GB; (ii) in multi-class classification, classical models (XGBoost 79.03%) outperformed recurrent deep models (BiLSTM 72.27%), reversing the binary ranking; (iii) class-incremental learning with EfficientNetB0 maintained 99.13% accuracy with only 0.65 percentage-point accuracy degradation across eleven incremental steps; (iv) transfer learning reduced training time by an average of 2.14 on image-based malware data with no significant accuracy cost; and (v) autoencoder pre-processing yielded a 14 training speedup at a cost of only 0.86 pp accuracy. These findings collectively confirm that the optimal ML configuration is context-dependent, validating the FDMs core premise and demonstrating its practical utility for cybersecurity practitioners.