Utilization of Large Language Models in Prehospital Triage: A Focus on ChatGPT’s Performance According to Pre-KTAS

This study evaluated the applicability of a large language model (LLM) as a decision-support tool for prehospital triage based on the Prehospital Korean Triage and Acuity Scale (Pre-KTAS). A simulation-based comparative study used 100 Pre-KTAS scenarios to compare two system groups, a non-trained LLM and a trained LLM (an LLM provided with Pre-KTAS educational materials), with a control group of 10 active-duty 119 paramedics. Each LLM evaluated all 100 scenarios in 30 independent repetitions. The outcomes included accuracy, classification consistency, agreement with paramedics, over- and under-triage patterns, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUC). Data were analyzed using Welch’s analysis of variance, Games-Howell post hoc tests, intraclass correlation coefficients (ICC), kappa statistics, chi-square tests, and receiver operating characteristic analysis. Overall accuracy was 0.69 ± 0.10 for the non-trained LLM, 0.81 ± 0.03 for the trained LLM, and 0.80 ± 0.03 for paramedics. The trained LLM and paramedics were significantly more accurate than the non-trained LLM (p < .001), with no significant difference between them. The trained LLM showed the highest classification consistency (ICC (3,1) = 0.940), the lowest under-triage rate (6.1%), and the highest sensitivity (0.95) and NPV (0.97). Paramedics showed the highest specificity (0.96) and PPV (0.93). The AUC values were 0.915 for the non-trained LLM and 0.934 for both the trained LLM and paramedics. These findings suggest that a Pre-KTAS-trained LLM may support prehospital triage decisions, although field validation and institutional and ethical guidance are required before clinical use.