Extraction of structured data from coronary angiography reports using large language models
J Martins Neves, D Ferreira, J Cravo, M Vilela, D Cazeiro, C Silva, I Araujo, J Pedro, S Esteves, S Morgado, F Valdeira, C Soares, F J Pinto, M Nobre MenezesAbstract
Introduction
Coronary angiography and angioplasty reports contain essential anatomical, physiological, and procedural data for research, audits, and quality assessment. However, as unstructured text, extracting variables —such as lesion location, FFR/iFR values, and procedural details—is slow and hard to scale. This limits dataset size, delays research, and hinders quality monitoring. Although Large Language Models (LLMs) show promise in medical text processing, their performance on Portuguese cardiology reports is unknown. This study evaluates whether open-source, locally deployable LLMs can convert free-text angiography reports into structured data for research and quality improvement.
Methods
A total of 1,314 reports (2012–2023) were analyzed. Two tasks were assessed: extraction of FFR/iFR values per vessel and classification of procedure type. The reference standard was established through manual review by cardiology trainees. Llama3, Mistral and Medllama2 models were tested using zero-shot and one-shot prompting and compared with a REGEX baseline. Performance was measured using global concordance, sensitivity, positive predictive value (PPV) and F1-score, with strict exact-value matching for physiological indices.
Results
Of the 1,314 reports, 681 described angiography alone and 633 included angioplasty, totaling 591 FFR and 1,190 iFR values. MedLlama2 was excluded for inconsistent structured output. For procedure classification, REGEX achieved the highest global concordance (0.992) and PPV, while Llama3 one-shot showed the best sensitivity (0.998) and an F1-score of 0.984. For FFR/iFR detection , Llama3 zero-shot demonstrated the strongest performance, with global concordance of 0.948 and sensitivity of 0.872, surpassing both Mistral variants and REGEX. For exact numerical extraction, Llama3 one-shot reached the highest concordance (0.906). REGEX remained highly specific but less sensitive in detecting index presence, reflecting limitations with varied phrasing.
Conclusion
Open-source LLMs can accurately structure Portuguese coronary angiography reports, reducing the need for large-scale manual extraction. Llama3 models reliably detect FFR/iFR measurements and show high concordance in exact value extraction, outperforming REGEX and other open-weight models in linguistically complex tasks. This substantially decreases the number of reports requiring human review and enables dataset generation while preserving data privacy. Overall, LLM-based extraction emerges as a practical, deployable solution for research, quality monitoring, and audit, with clear potential to accelerate data-driven cardiovascular care.ResultsFor image description, please refer to the figure legend and surrounding text.