Multimodal Analysis of Patient–Nurse Interactions During Inpatient Discharge Education: A Pilot Comparison of Bedside and Virtual Nursing Modalities

Background: The ongoing nursing shortage, COVID-19 pandemic, and advances in healthcare technology have accelerated the adoption of Virtual Nursing Care Models (VNCM) in inpatient settings. Although virtual nurses improve staffing efficiency, there is limited evidence on how patients interact and communicate during virtual and bedside nurse discharge education. Objective: This study examined differences in patient attention, emotional expression, and communication patterns during bedside and virtual nurse discharge education using multimodal eye-tracking and vocal emotion analysis. Methods: A quasi-experimental design was used with 12 hospitalized adults (≥30 years) in medical-surgical units. Participants received standardized discharge education from either a bedside nurse (n = 7) or a virtual nurse via in-room television (n = 5). Eye-tracking data were collected using Pupil Invisible glasses and analyzed using iMotions, vocal emotion features were analyzed using audEERING. Eye-tracking metrics included dwell time, fixation dwell, fixation count, fixation duration, and time to first fixation (TTFF). Vocal measures included activation, valence, dominance, and neutrality. The analyses were descriptive and exploratory. Results: Patients in both modalities directed most of their visual attention to discharge materials, with higher dwell time, greater fixation counts, and faster TTFF compared to the nurse and TV screen. In the bedside condition, attention was more tightly concentrated on the materials, reflecting the shared and guided viewing setup. In contrast, the virtual condition showed more distributed attention across the screen and printed materials, suggesting greater self-directed visual exploration. Voice analysis showed higher activation in the virtual condition, indicating increased emotional arousal; however, this was accompanied by a reduction in interactive communication. Conclusion: These findings suggest a trade-off between guided, material-focused attention in bedside settings and more self-directed distributed processing in virtual environments. The study findings imply the need to support both information delivery and patient interaction in virtual nursing models.