DOI: 10.26466/opusjsr.1948539 ISSN: 2791-9781

Early identification of at-risk students in online learning environments: A learning analytics approach using machine learning models

Verda Gizem Oğul, Yavuz Selim Balcıoğlu
Show PDF Cite
Online learning environments generate temporally ordered traces that can support early academic-risk screening, but credible claims require an explicit prediction point and strict control of post-outcome information. This study evaluates whether information available by day 28 of an Open University module presentation can identify students who subsequently fail or withdraw. Raw tables from the Open University Learning Analytics Dataset (OULAD) were reconstructed at student–module–presentation level after timestamped Virtual Learning Environment (VLE) and assessment records had been truncated at the prediction point. The day-28 risk set comprised 27,522 enrolments from 24,832 unique students; 44.1% subsequently failed or withdrew. Seven classifiers were compared using student-grouped five-fold validation, randomized hyperparameter optimization, and a student-disjoint held-out test set. Gradient Boosting achieved the highest grouped-validation ROC-AUC and yielded test ROC-AUC=0.790 (95% CI: 0.777–0.801), PR-AUC=0.770 (95% CI: 0.753–0.786), accuracy=0.724, precision=0.728, recall=0.597, and F1=0.656. Performance increased from day 14 (ROC-AUC=0.747) to day 56 (0.835), demonstrating an earliness–performance trade-off rather than uniformly high early accuracy. SHAP analysis identified completion of assessments due by the cutoff, assessment performance, prior education, and VLE engagement as influential predictors. A score-excluded specification retained meaningful discrimination (ROC-AUC=0.762). Three-class sensitivity analysis showed that failure and withdrawal were considerably harder to distinguish than success. The findings support cautious use of day-28 risk scores for prioritizing supportive outreach, while emphasizing calibration, subgroup monitoring, human oversight, and local validation.

More from our Archive