Leveraging AutoML for Advanced Network Traffic Analysis and Intrusion Detection by Enhancing Security with a Multi-Feature IDS Dataset
Open Access DepositedThis research explores how AutoML enhances network-based intrusion detection by reducing false alarms, eliminating manual tuning, and enhancing model adaptability. AutoML automates key machine learning processes such as preprocessing, algorithm selection, feature engineering, and hyperparameter tuning, thereby improving the efficiency and detection accuracy of IDS. The research employs an enhanced, multi-feature version of the CSE-CIC-IDS2018 dataset. Comprehensive preprocessing was undertaken to eliminate issues like missing values, duplicate records, mislabeling, and irrelevant features. To address severe class imbalance, a multi-phase resampling strategy was employed, combining undersampling, SMOTE, and SMOTEENN, which improved the representation of the minority class. Three AutoML platforms AutoGluon, AWS Autopilot, and Azure AutoML were evaluated using stratified sampling with consistent validation and testing sets. Model performance was measured using metrics such as accuracy, balanced accuracy, precision, recall, F1-score, false positive rate (FPR), and Matthews Correlation Coefficient (MCC). All frameworks achieved exceptional results, with accuracy exceeding 0.99999, balanced accuracy above 0.99980, F1-score, precision, and recall above 0.99999, and MCC above 0.99995, alongside near-zero false positive rates. Statistical validation using the non-parametric Mann-Whitney U test confirmed that the proposed AutoML models achieved a statistically significant improvement in accuracy (p = 0.002), a statistically significant reduction in FPR (p = 0.004), and a statistically significant reduction in model development time (p = 0.014) compared to benchmark studies. ROC and precision-recall curves yielded perfect AUC and AP scores of 1.00. The results show that the proposed AutoML models make intrusion detection better, increase accuracy, shorten development time, and make it easier to scale IDS deployment. These findings highlight AutoML's practical value for organizations looking to rapidly implement adaptable and accurate intrusion detection solutions.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.