Using LMS Log Data to Identify At-Risk Students: A Systematic Review of Machine Learning Approaches and Bibliographic Analysis

Abstract

Purpose: This study evaluates the effectiveness of machine learning algorithms in predicting student dropout using Learning Management System (LMS) log-in data. Design/ Methodology/ Approach: The study used a systematic literature review and bibliographic analysis. The search encompassed papers from the Scopus database up to October 2024. Initially, 100 articles were identified. After applying exclusion criteria, including removing editorials, letters, comments, and conference papers, 61 studies were chosen for the final review. Performance criteria such as accuracy, precision, recall, and f1-score were employed to assess these studies. Research Limitation: Several limitations were acknowledged, including potential publication bias due to the inclusion of only peer-reviewed articles, variability in educational contexts and LMS platforms, and heterogeneity in machine learning methods and performance metrics. Findings: Random Forest emerged as the most commonly used machine learning algorithm for identifying at-risk students, followed by Convolutional Neural Networks. In the analysed research, Random Forest outperformed all other algorithms, achieving a 99% accuracy rate in predicting at-risk students. Students' assessment scores emerged as the most significant feature in the model performances, followed by students' participation in a session. Practical Implication: It is noted that most researchers do not report on significant features/variables or the contribution of features to the model’s performances. Social Implication: These significant features or variables are essential for institutions employing a blended learning approach, as they provide insights into where to allocate limited resources most effectively. Originality/Value: This study contributed to the pool of knowledge on how Machine learning techniques have been employed with Learning Management System log-in data to predict student dropout.