Multimodal Deep Learning Approach Using Laser Doppler Flowmetry, Fluorescence Spectroscopy, and Clinical Data for Multi‐Class Mental Health and Severity Classification

ABSTRACT

Conditions affecting mental health, namely stress, anxiety, and depression, are on the rise in various populations, emphasizing the need for early evaluation and ongoing surveillance of these states. Mental health evaluation typically relies on subjective questionnaires and fails to assess underlying physiological mechanisms. This study describes an automated framework for multi‐class mental health classification and level of severity assessment using multimodal data. The model X‐DyFGate proposed in this paper uses Laser Doppler Flowmetry, Fluorescence Spectroscopy, and clinical factors. It is built using residual sensor encoders, cross‐attention fusion, transformer representation learning, dynamic feature gating, and a compact bottleneck layer. This pipeline moves beyond simple feature concatenation. Its novelty lies in its hierarchical, attention‐based fusion of low‐level (sensor) and high‐level (clinical) data, coupled with a dynamic gating mechanism that learns the importance of each modality for a specific individual's prediction. This personalizes the model, a key step toward precision psychiatry. The dataset used in this work consists of 1078 samples across six mental health categories with associated severity levels. The proposed model provided an accuracy of 99.38% multi‐class classification, while the one‐dimensional Convolution Neural Network model accuracy is 94.44%. For severity level classification, the proposed model provided 95.67% accuracy for stress, 95.67% for anxiety, and 97.53% for depression, whereas the one‐dimensional Convolution Neural Network model produced an accuracy of 67.28% for stress, 90.12% for anxiety, and 61.11% for depression. The proposed model is further validated on two more publicly available datasets, and it performed well on those two datasets. Despite promising performance, the proposed framework involves relatively high computational cost and has been validated on a limited, homogeneous dataset. Furthermore, comprehensive evaluation on diverse real‐world clinical data and deeper interpretability analysis remain areas for future investigation.