ECG-Based Emotion Recognition Using Beat-Level Complementary Feature Fusion

Physiological-signal-based emotion recognition has received attention in human–computer interaction. Electrocardiograms (ECGs) are readily acquired, and short windows contain repeated beat morphology and beat-wise variations that single-window encoding struggles to separate. Multiscale morphology modeling and training-sample diversity remain limited. We therefore propose Complementary Feature Fusion Dual-Path (CFF-DP), an ECG emotion recognition framework using beat-level complementary feature fusion, with three components: (1) a dual-path framework, where the morphology-stable path constructs representative beats with window-adaptive Gaussian weights, while the morphology-difference path combines beat-wise encoding, positional encoding, and additive attention; gated fusion integrates representations; (2) adaptive dilated convolution (ADConv), which extracts multiscale beat-morphology features using shared kernels and input-dependent scale weights; and (3) deviation-based beat-oriented augmentation (DBOA), which adjusts real-noise injection probability and target signal-to-noise ratio according to morphological deviation. CFF-DP achieved 43.39% mean Macro-F1, close to the best comparator, with the fewest multiply–accumulate operations in five-seed WESAD three-class leave-one-subject-out (LOSO) evaluation, although recognition mainly distinguishes stress, with limited amusement discrimination. With short-gap calibration and testing within the same recording, fine-tuning using 40 s per class achieved 76.72% Macro-F1, exceeding six comparators. Binary DREAMER LOSO retained WESAD hyperparameters: valence Macro-F1 exceeded six comparators, whereas arousal fell below four; both remained below uniform random baselines, indicating limited recognition under current experimental conditions. The framework combines computational efficiency with within-record personalization advantages, although cross-subject recognition remains limited.