Enhancing cardiovascular consultation training with a multi-agent virtual patient framework: a multi-model comparative evaluation study

Background

Incomplete cardiovascular history-taking and suboptimal risk-factor assessment are primary contributors to delayed diagnosis and inconsistent guideline adherence. While Large Language Model (LLM)-based virtual patients (VPs) offer a scalable solution for standardized training, single-agent systems often suffer from "hallucinations," role inconsistency, and instability in complex cardiovascular scenarios, limiting their utility in high-stakes clinical education.

Purpose

To develop and evaluate a Multi-Agent (MA) virtual patient framework designed to enhance structured cardiovascular consultation quality, minimize diagnostic hallucinations, and improve clinical reasoning in simulated cardiology training.

Methods

We developed a cardiology-specific MA virtual patient system composed of orchestrated sub-agents responsible for clinical information retrieval (RAG-supported), personality modeling, and response generation. The system was benchmarked against single-LLM baselines across seven backbone models using a standardized dataset of 560 cardiology consultation scenarios.

Evaluation was conducted in two stages:Large-scale script-based benchmarking (N=560): Performance was evaluated using checklists derived from ESC guideline-relevant domain (symptom characterization, risk-factor coverage, red-flag recognition, and differential diagnosis).Pilot interactive validation: Four medical postgraduates conducted blinded consultations with five unique cardiology virtual patients to assess response accuracy, role consistency, and perceived educational impact.

Results

In script-based evaluation (N=560), the MA framework consistently outperformed single-LLM systems. High-capacity MA models achieved diagnostic accuracy >97% and a hallucination rate <2%, whereas single-model stability fluctuated significantly (accuracy <95%). Role-playing fidelity scores, assessed by blinded evaluators were markedly higher for MA systems (38.0–39.7/40) than for baselines (36.3–38.7/40), particularly in complex differential diagnosis. Pilot interactive validation confirmed high usability, with all participants reporting enhanced clinical reasoning awareness and supporting its integration into cardiology curricula.

Conclusion

A multi-agent virtual patient architecture significantly improves the stability, accuracy, and clinical fidelity of cardiovascular consultation simulations. By substantially reducing diagnostic inconsistency and clinically irrelevant output, and ensuring guideline-aligned interactions, this digital health approach provides a robust, scalable strategy to standardize early-stage cardiovascular evaluation and improve diagnostic safety.