Kuramoto-regularized deep learning framework for automated student behavior analysis in smart classrooms

This research work presents an integrated computer vision framework for automated analysis of student behaviour in real classroom environments, offering a practical solution to a pressing issue. A significant problem with existing research on predicting user behaviour is that most methods rely on a single instance image for simplicity. Some works, however, utilise group images/videos, which lag in capturing socially coupled behaviours. Student behaviour in the classroom has fundamental dynamics, and we need a model to understand it in the context of crucial collective alignment. To synchronise the phases of behaviours, we introduce the Kuramoto function from physics as a synchrony regularizer for the deep learning model, allowing it to synchronise the nonlinear complex phases while learning about the group of people. The proposed pipeline combines robust object detection with deep sequential classification to recognise fine-grained behaviours such as writing, listening, reading, or discussing. In the detection stage, You Only Look Once Version 8 (YOLOv8) and Faster Region-based Convolutional Neural Network (Faster R-CNN) models are trained to localise students accurately under occlusion, clutter, and varying classroom conditions. Cropped student regions are then passed to a Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) CNN + LSTM + ATTENTION classifier that captures both spatial features and temporal dependencies while highlighting salient frames through an attention mechanism. To further enhance temporal consistency and interpretability, novel regularisers, (i) temporal smoothness for reducing behavioural fluctuations, (ii) Kuramoto synchrony that adopts a non-linear dynamics model of phase alignment to encourage coherent behavioural patterns in a group of students, and (iii) attention entropy helps to prevent degenerate focus on single frame, which are incorporated during training. Experimental results on the UK dataset demonstrate that the pipeline achieves high detection precision (0.961) and recall (0.975) with YOLOv8 and strong behaviour classification performance (validation accuracy: 97.31%), surpassing baseline CNN- or LSTM-only models. This research contributes a scalable, interpretable, and deployment-ready methodology for smart classrooms, reducing reliance on manual observation and enabling real-time behavioural insights to support teaching effectiveness and student engagement monitoring.