Multimodal Performance Prediction in Online Learning Using Bi-LSTM and Mixture of Experts

Educational records alone may not capture changes in students’ classroom affective states. This study proposes Bi-LSTM-SCPM, a student comprehensive performance model (SCPM) integrating bidirectional long short-term memory (Bi-LSTM) with mixture of experts (MoE) regression. The dataset comprised 14,645 facial images from 71 students across three stages of 12-week computer-assisted translation courses. ResNet50 achieved 93.8% accuracy in positive–negative facial state classification. Bi-LSTM integrated stage scores derived from instructor-designated key periods and Bayesian smoothing into a student-level affective score (ZHCJ). ZHCJ was combined with educational information and prerequisite course performance to predict final grades. Across 30 matched student-level splits, MoE achieved an R2 of 0.943 ± 0.002 and an MAE of 1.144 ± 0.128 points, compared with 0.927 ± 0.002 and 1.274 ± 0.149 points for XGBoost. Both differences remained significant after Holm adjustment (p < 0.001). ZHCJ also outperformed summary statistics and direct stage score inputs under both predictors. The setting ϖ = 2 and k = 20 performed best within the nine-setting grid. These findings support the predictive value of integrating temporal affective representations and educational information within the study cohort.