Reliable and scalable hybrid extraction pipeline with selective human-in-the-loop assessment for clinical digital fingerprinting enabling real-time prediction in cardiology
M Eberhardt, P Hempel, J Spauschus, J Fischer, D Krefting, C Schmidt, S Brandenburg, G RiedemannAbstract
Background
Systematic outcome modelling in cardiology is limited by the lack of high-quality, scalable extraction of structured variables from heterogeneous clinical documentation. In routine care, key information for phenotyping and risk stratification remains embedded in unstructured narratives, while structured documentation is often difficult to achieve because clinical information is complex and context-dependent; manual review and extraction of relevant data are resource-intensive and difficult to scale.
Purpose
To develop a privacy-preserving, on-premise hybrid pipeline for multi-document clinical digital fingerprinting to support real-time risk prediction in routine TAVI care.
Methods
We developed an on-premise workflow integrating: (i) rule-based (regex) extraction for 74 deterministic standardised parameters, (ii) a locally hosted LLM ensemble consisting of 3 general-purpose foundation models for 43 context-dependent procedural and narrative variables across echocardiography, TAVI, and discharge documentation, (iii) an LLM verifier for post-extraction consistency checks, and (iv) selective human-in-the-loop (HITL) assessment in case of ensemble disagreement. Extracted variables were harmonised into a patient-level digital fingerprint (117 structured variables) covering cardiac structure/function, valvular haemodynamics, and procedural/peri-procedural domains. The pipeline was evaluated in 240 consecutive TAVI patients using routine TAVI pathway documentation. Performance was assessed against a manually corrected gold standard annotated by two clinical reviewers, with discrepancies resolved through discussion.
Results
Rule-based modules achieved 100% value agreement for all standardised parameters. The selectively assessed LLM ensemble achieved 97.2% accuracy for context-dependent variables extracted from narrative clinical documentation. Integration of the LLM verifier improved ensemble accuracy from 89% to 97% by flagging inconsistent outputs. Selective HITL assessment was required in 8% of ensemble outputs. Fingerprints were complete in 87% of patients; incompleteness was driven by absent source documentation rather than extraction errors (manual abstraction ceiling: 87%). Multiple clinical scores could be calculated, with completeness primarily limited by source documentation; (e.g., CHA₂DS₂-VASc 91% and EuroSCORE 89% of patients).
Conclusion
A reliable and scalable hybrid extraction pipeline combining rule-based and LLM-based extraction with selective HITL assessment enables high-quality structuring of multi-document cardiology text into clinically meaningful digital fingerprints. This supports privacy-preserving, on-premise phenotyping in routine care and provides a practical foundation for real-time clinical risk prediction and decision support.Figure ESC Abstract.png