Enhancing Machine Learning and Deep Learning Models for Depression Detection: A Focus on SMOTE, RoBERTa, and CNN-LSTM

Abstract

Depression is a major public health concern, affecting millions worldwide, and necessitates early, accurate detection for timely intervention. This study focuses on enhancing machine learning (ML) and deep learning (DL) models for improved accuracy in depression detection using the Counsel Chat Dataset. To address the challenges of class imbalance, we employed advanced preprocessing techniques, including the Synthetic Minority Oversampling Technique (SMOTE), alongside model fine-tuning and architectural optimizations. Our results demonstrated significant performance improvements, particularly with transformer-based models and hybrid architectures. RoBERTa, a transformer-based model, achieved an accuracy of 91.55%, an F1-score of 0.91, and a recall of 92.10%, outperforming state-of-the-art approaches. Similarly, CNN-LSTM attained an accuracy of 91.67% with a 95% CI of (0.8987, 0.9312), while XGBoost achieved the highest accuracy among ML models at 93.06%, with a 95% CI of (0.921, 0.941). Statistical tests validated the superiority of these models, with p-values of 5.48e-13 for RoBERTa and 3.41e-16 for XGBoost. These findings underscore the pivotal role of data augmentation and preprocessing in creating balanced datasets and enhancing the predictive capabilities of AI models for depression detection.