Dual-Stream Fusion of Eye-Tracking and ECG Signals for Fatigue Detection in Remote Tower Air Traffic Controllers

Fatigue detection in remote tower air traffic controllers is important for maintaining operational safety under sustained visual monitoring and high cognitive workload. This study proposes MFD-Net, a dual-stream multimodal fusion framework using eye-tracking and electrocardiogram (ECG) signals. The model separately encodes eye-tracking and ECG-derived temporal inputs, incorporates an ECG-derived RMSSD expert feature, and performs lightweight late fusion for fatigue-state classification. Under the mixed-subject random-window protocol, MFD-Net achieved an Accuracy of 85.20%, a Recall of 83.33%, and an AUC of 0.9337. Because overlapping windows from the same participant and scenario could appear in both training and test sets, this result should be interpreted as a potentially optimistic within-distribution estimate. Under the stricter zero-shot leave-one-subject-out (LOSO) protocol, performance decreased substantially, with an Accuracy of 70.95±21.59%, a Recall of 22.98±36.30%, and an AUC of 0.6025±0.2984. This low zero-shot Recall indicates limited subject-independent fatigue-detection capability. Lightweight target-subject calibration and sequential probability aggregation improved adaptation and temporal stability, although the calibration results should be interpreted cautiously because random target-subject windows were used for fine-tuning. These findings suggest that eye-tracking and ECG fusion are promising under controlled conditions, while practical deployment requires deployment-oriented calibration protocols, recall-oriented optimization, and further real-world validation.