Refining malware detection with enhanced machine learning algorithms using hyperparameter tuning

Abstract

The aim of this research is to investigate and demonstrate the advantages and limitations of various machine learning techniques for malware classification, specifically focusing on portable executable (PE) files. The study addresses common challenges in machine learning, such as overfitting and underfitting, by employing ensemble methods and pre-processing techniques, including feature selection and hyperparameter tuning. The primary objective is to enhance classifier performance in distinguishing between malicious and benign PE files. Through a comparative analysis of machine learning methodologies such as random forests, decision trees, and gradient boosting, the study highlights the superiority of the random forests algorithm, achieving an impressive accuracy rate of 99%. By thoroughly evaluating the strengths and limitations of each algorithm, the research provides valuable insights into effectively handling diverse malware categories. This paper underscores the significance of ensemble methods, feature engineering, and pre-processing in improving classifier performance for malware classification, specifically in the context of portable executable files.