Gaze-based music generation system for users with severe speech motor impairment

Users with Severe Speech Motor Impairment (SSMI) face difficulties interacting with digital systems due to limited speech and motor control. To address this, we present an eye gaze-based system that enables users with SSMI to generate short music using eye gaze control. The system combines a webcam-based eye-tracking with a music generation system that transforms user-selected images into music. A user study was conducted with 15 participants with SSMI across 74 sessions. The results show that the participants were able to interact with the system using gaze-based control, with an average engagement time of 61.38 seconds per session. Eye-tracking calibration accuracy varied across participants, reflecting individual differences in gaze control. These findings demonstrate the feasibility of gaze-based generative music interaction as an accessible assistive system for users with SSMI.