AN ENSEMBLE MACHINE LEARNING FRAMEWORK FOR AUTOMATED CYBERSECURITY INCIDENT CLASSIFICATION USING STRUCTURED METADATA

Abstract

This study addresses the growing need for accurate and automated cyber incident classification systems to support timely decision-making in response to increasing cyber threats. It develops and evaluates an ensemble machine learning framework for multi-class classification using structured cybersecurity incident metadata. A dataset of 93,144 incidents, categorized into six attack types, was analyzed using four supervised learning algorithms: Decision Tree, K-Nearest Neighbors (KNN), Random Forest, and Extra Trees. To address class imbalance, the Synthetic Minority Oversampling Technique (SMOTE) was applied. The dataset was split into 80% training, 10% validation, and 10% testing, and model performance was evaluated using accuracy, precision, recall, and F1-score, alongside confusion matrix and feature importance analysis. Results show that ensemble models outperform non-ensemble approaches, with the Extra Trees classifier achieving the highest performance (95.4% accuracy and 95.2% macro F1-score), followed by Random Forest (94%), Decision Tree (88%), and KNN (81%). Performance remained consistently high across all attack categories, with most misclassifications occurring between closely related classes. Feature importance analysis revealed that over 80% of predictive performance was driven by variables such as user agent, source IP, destination IP, and temporal features. The study concludes that lightweight and interpretable ensemble models can achieve high classification accuracy comparable to more complex methods while remaining computationally efficient. The proposed framework offers a practical and scalable solution for real-world cybersecurity incident classification and supports improved incident response strategies.