Harnessing the Potential of LLMs in Hospital Management: Insights Into Medical Data Inquiry
Jiaxi Li, Yu Zhang, Jing Zhao, Shengyang He, Danni LiABSTRACT
Background
Considering the rapid expansion of hospital operations and the increasing digitisation of medical data, there is a pressing need for efficient and intelligent methods to process and analyse large‐scale medical data.
Methods
We integrated the QLoRA algorithm with ChatGLM2‐6b, Llama2‐7b, and Llama2‐13b models, fine‐tuning them on a local SQL dataset to optimise query performance. Prompt‐Engineering with ChatGPT was further applied for effective SQL execution.
Results
Original open‐source models showed almost no SQL generation capability (overall EX ≈ 0 for ChatGLM2‐6B and Llama2‐7B; 0.04 for Llama2‐13B). QLoRA fine‐tuning substantially improved performance, with QLoRA‐Llama2‐13B achieving the best results among open‐source models (overall EX 0.41 ± 0.030). Proprietary models demonstrated significantly stronger performance. Zero‐shot ChatGPT‐3.5 achieved moderate accuracy (EX 0.44 ± 0.007), which improved to 0.94 ± 0.017 with few‐shot prompting. GPT‐4.1 further improved performance, reaching 0.78 ± 0.038 in the zero‐shot setting and 0.96 ± 0.011 with few‐shot prompting. The few‐shot GPT‐4.1 results were comparable to those of database engineers (EX 0.97 ± 0.017), with no significant difference ( p = 0.42).
Conclusion
Fine‐tuned LLMs and few‐shot GPT‐4.1 demonstrate substantial improvements in SQL query execution, providing a robust framework for efficient medical data analysis and informed hospital decision‐making.