Developing an artificial intelligence‐based headache diagnostic model from a clinic patients' dataset
Masahito Katsuki, Yasuhiko Matsumori, Shin Kawamura, Kenta Kashiwagi, Akihito Koh, Senju Tachikawa, Fuminori Yamagishi- Neurology (clinical)
- Neurology
Abstract
Objective
We developed an artificial intelligence (AI)‐based headache diagnosis model using a large questionnaire database in a headache‐specializing clinic.
Background
Misdiagnosis of headache disorders is a serious issue and AI‐based headache diagnosis models are scarce.
Methods
We developed an AI‐based headache diagnosis model and conducted internal validation based on a retrospective investigation of 6058 patients (4240 training dataset for model development and 1818 test dataset for internal validation) diagnosed by a headache specialist. The ground truth was the diagnosis by the headache specialist. The diagnostic performance of the AI model was evaluated.
Results
The dataset included 4829/6058 (79.7%) patients with migraine, 834/6058 (13.8%) with tension‐type headache, 78/6058 (1.3%) with trigeminal autonomic cephalalgias, 38/6058 (0.6%) with other primary headache disorders, and 279/6058 (4.6%) with other headaches. The mean (standard deviation) age was 34.7 (14.5) years, and 3986/6058 (65.8%) were female. The model's micro‐average accuracy, sensitivity (recall), specificity, precision, and F‐values for the test dataset were 93.7%, 84.2%, 84.2%, 96.1%, and 84.2%, respectively. The diagnostic performance for migraine was high, with a sensitivity of 88.8% and c‐statistics of 0.89 (95% confidence interval 0.87–0.91).
Conclusions
Our AI model demonstrated high diagnostic performance for migraine. If secondary headaches can be ruled out, the model can be a powerful tool for diagnosing migraine; however, further data collection and external validation are required to strengthen the performance, ensure the generalizability in other outpatients, and demonstrate its utility in real‐world settings.