Evaluation of a human-in-the-loop pre-procedural clinical conversational AI calling system for cardiac catheterisation
D Pineda, A Vengrenyuk, M Chang, M Jahaj, Y Vengrenyuk, B Oliver, A Hooda, G Dangas, J Bander, A Rhee, B Kaplan, P Francaviglia, B Darrow, S Sharma, A KiniAbstract
Background
Pre-procedural preparation for cardiac catheterisation requires substantial nursing time for patient outreach, logistical instruction, and clinical screening. Artificial intelligence (AI) may help automate routine tasks, but safe implementation requires maintained clinical oversight.
Purpose
To evaluate the safety, reliability, and patient engagement of a human-in-the-loop AI calling system, Sofiya, integrated into pre-procedural cardiac care.
Methods
Sofiya, an AI-enabled conversational agent, was incorporated into the pre-catheterisation workflow to provide logistical instructions and collect selected clinical information, including allergy history and anticoagulant use. All interactions remained under registered nurse oversight, with nurse attestation and escalation for clinical concerns or patient request. Outcomes were compared across an initial 90-day implementation phase and a subsequent 90-day operational phase.
Results
Among 1,606 patient encounters, the AI-supported workflow achieved an overall 87.1% successful call outcome. System performance improved over time, with AI hallucinations requiring a nurse call decreasing from 0.7% to 0.0% and reliability-related issues decreasing from 6.0% to 3.0% in the later phase. Self-service resolution, where a nurse decided not to call the patient, increased from 36.6% to 42.6%. Escalation to human providers remained available for clinical nuance and occurred in 12.0% of calls. Patient-requested human intervention increased from 0.5% to 1.1%. Patient satisfaction remained high at 94.9%, although only patients who reached the end of the script were surveyed.
Conclusion(s)
A human-in-the-loop AI calling system can support pre-procedural cardiac workflows while preserving clinical oversight and patient access to nursing staff. This model was associated with high patient satisfaction, improved operational reliability over time, and effective delegation of routine pre-procedural tasks.Results Table