Diagnosis and analysis of plasma phosphorylated tau 217 in a memory clinic cohort: limitations of Generative Pre-trained Transformer-5
Yat Fung Shea, Yi Ling Wong, Tsz King Wong, Pak Lam Chan, Ka Keung Yam, Felix Chi Kin Wong, King Pui Florence Chan, Yi Wah Eva Cheung, Henry Ka Fung Mak, Sau Man Mary Ip, Patrick Ka Chun Chiu, Ching Wan LamAbstract
Introduction:
No published studies have evaluated the application of Generative Pre-trained Transformer-5 (GPT-5) in the analysis of memory clinic patient clinical notes or the interpretation of plasma phosphorylated tau (P-tau) 217 values. We compared Alzheimer’s disease (AD) probability estimates generated by GPT-5 before and after incorporating plasma P-tau 217 with pre- and posttest probabilities.
Methods:
This was a retrospective study comprising 74 patients from a memory clinic in Queen Mary Hospital, Hong Kong. Final diagnoses were made by physicians, supported by medical history, physical examination, neuroimaging and amyloid positron emission tomography. Extracted clinical data included cognitive, functional and neuropsychiatric assessments. Pretest AD probabilities were derived from a published meta-analysis, while posttest probabilities were calculated using a Bayesian approach. These values were compared with those estimated by GPT-5. The diagnostic performance of GPT-5 and physicians was assessed using accuracy and Kappa coefficient, with final diagnosis as reference.
Results:
There were 40 amyloid-positive (A+) and 34 amyloid-negative (A−) patients. In A+ patients, Bayesian posttest probabilities were higher than GPT-5 estimates (median 97.0% vs. 80.0%,
Conclusion:
Our findings show that GPT-5 has limitations in analysing clinical information of real-life memory clinic patients.