Reinforcement learning with GSPO enables vision language model (VLM) to detect structural heart disease from 12-lead electrocardiography
W C Wong, C Liu, C Y Leung, H L Li, R T C Sun, Y C Zhang, L Yu, J W K Ho, C K WongAbstract
Background
Convolutional neural networks (CNNs) enable the detection of structural heart disease (SHD) from 12-lead electrocardiography (ECG). However, the subtle ECG features identified by CNNs are generally not interpretable by humans and are not translatable into new diagnostic markers for clinicians to adopt.
Purpose
To develop vision-language models (VLMs) that detect SHD from ECG data and leverage chain-of-thought (CoT) analysis to evaluate the human interpretability of the model's reasoning.
Methods
We converted 79,449 ECGs from 50,125 patients in a public dataset into 12-lead ECG image representations. Binary SHD labels were derived from paired echocardiographic findings. We fine-tuned the open-sourced Qwen3-VL-8B model with low-rank adaptation (LoRA) using two approaches coupled with visual question answering (VQA): supervised fine-tuning (SFT) and reinforcement learning (RL) with group sequence policy optimization (GSPO). (Figure 1) We compared the resulting Qwen3-VL-8B-GSPO model against state-of-the-art (SOTA) closed-source VLMs, including GPT-5.2 and Gemini 3 Pro. Predictive accuracy was evaluated on an external cohort. For explainability assessment, board-certified cardiologists independently reviewed the model-generated CoT reasoning.
Results
Without fine-tuning, SOTA closed-source VLMs more accurately detected conditions with well-established ECG findings, such as pericardial effusion (GPT-5.2: 86.5%; Gemini 3 Pro: 71.2%; Qwen3-VL-8B: 17%) and left ventricular hypertrophy (LVH; GPT-5.2: 79.6%; Gemini 3 Pro: 79.8%; Qwen3-VL-8B: 34.7%). For SHDs not typically considered detectable by human interpreters, such as valvular heart disease, baseline VLM performance was similar across models. Fine-tuning Qwen3-VL-8B with SFT improved accuracy for LVH (34.7% to 86.0%) and left ventricular systolic dysfunction (LVSD) (19.3% to 44.0%), but accuracy for other SHD classes deteriorated. Conversely, RL with GSPO enabled accuracy improvements across most SHD classes, exceeding SOTA closed-source VLMs in LVH, LV/RV dysfunction, and tricuspid/mitral regurgitation despite a significantly smaller model size. Comparable accuracy was observed using external validation set for LVH and LV/RV dysfunction. (Figure 2) CoT inspection revealed that GPT-5.2 performed poorly in detecting valvular diseases because it reasoned that such conditions "should not be detectable on ECG," leading to a significantly higher refusal rate compared to Qwen3-VL-8B-GSPO (0% vs. 28.3%, p= 0.0000291). There was a high rate of hallucinated ECG findings in both models, such as citing non-existent QRS, ST segment, and T-wave changes (60.4% vs. 43.4%, p=0.926).
Conclusion
RL via GSPO enables smaller VLMs to achieve comparable or superior SHD diagnostic accuracy compared to SOTA closed-source models. The high rate of hallucinations in the CoT reasoning suggests that VLM-generated insights are not yet suitable for human-interpretable clinical use.