A Contextualized Deep Neural Network Model for Classifying IT Incidents Based on Severity

Abstract

IT service management operational efficiency is significantly dependent on the accuracy and cost of IT tickets classification. Whilst several machine learning techniques have made significant progress in the task of IT tickets classification, Classifying tickets by severity remains inherently complex, costly and multifaceted. It requires a deep understanding and interpretation of contextual information in words, phrases, and sentences. Additionally, the correlation between the classification process and operational costs impacts the efficacy and effectiveness of IT operations. This paper introduces the Severity Misclassification Impact (SMI) as a novel metric of quantifying the operational costs of misclassification errors by accounting for the varying costs associated with the different types of errors due to misclassifications. To highlight the need for SMI in contextual classification, this paper proposes a novel blended model approach comprising Bidirectional Encoder Representations from Transformers (BERT), Term Frequency-inverse Document Frequency (TF-IDF), Long Short-Term Memory (LSTM), and Light Gradient-Boosting Machine (LightGBM). By conducting extensive experiments on real-world IT Tickets, it demonstrates that incorporating SMI as an evaluation metric enables a robust way to assess a classification model's potential. This approach offers a nuanced understanding of its effectiveness in minimizing operational costs of misclassification, hence significantly improving IT tickets classification.