Multimodal Engagement Assessment in Children During Invented Story Paradigm With a Social Robot

This study presents a multimodal framework for assessing engagement in school‐age children during an “invented story” paradigm with the NAO social robot. Engagement is conceptualized as a multidimensional construct encompassing cognitive, affective, and behavioral components. Seventy‐two children (ages 7–9) participated in one‐on‐one sessions in a school setting, in which they were asked to invent and narrate a story to the robot. An automated gaze‐labeling strategy, combining Gaze360 with K‐means clustering, was developed to quantify attention‐related metrics in real‐life conditions. Additional features included heart rate (HB) measured at discrete interaction phases and task‐performance indicators such as word count, latency, and interaction duration. Engagement scores provided by independent human observers were used as a reference for validation. Correlation analyses revealed significant relationships between observer‐rated engagement and multiple sensor‐derived features, particularly gaze behavior and task‐related measures. Furthermore, supervised classification methods (support vector machines, K‐nearest neighbors, and random forests) achieved up to 81% accuracy in distinguishing high‐ versus low‐engaged children. These results highlight the effectiveness of combining sensor‐derived and behavioral metrics for scalable engagement assessment. The proposed approach holds promise for broader applications in educational and therapeutic settings, especially for populations with atypical developmental profiles. Overall, the study demonstrates the utility of embodied storytelling and automated multimodal analysis as tools for understanding and fostering child–robot engagement.