Numeric versus embedding pipelines for optimized clinical risk prediction
Aaron Ge, Jialong Wu, Xueyao Wu, Jeya Balaji Balasubramanian, Varun Nautiyal, Rishi Jayakumar, Chy Murali, Angela Lee, Andrew Nguyen, Matthew Allen, Chang Shu, Clayton H Brown, Shuo Chen, Katarina Zeder, Jonas De Almeida, Florence X Doo, Bradley A MaronAbstract
Objectives
To determine whether numerical and large language model embedding-based representations of the same structured electronic health record data fail on the same patients or on distinct patient groups.
Materials and Methods
We compared 2 XGBoost pipelines across 6 intensive care unit prediction tasks in 22 591 MIMIC-III patients: a numerical model (NM) trained on 458 engineered features and a semantic model (SM) trained on text embeddings of serialized versions of the same variables. We evaluated discrimination, calibration, linear-probe encoding fidelity, descriptive SHAP attribution, patient-level error overlap, subgroup-discovered failure phenotypes, and data-architecture meta-features.
Results
The NM outperformed the SM for 5 of 6 tasks, including in-hospital mortality (area under the receiver operating characteristic curve [AUROC] 0.9010 vs 0.8351; P < .001), whereas the SM outperformed the NM for 30-day readmission (0.6618 vs 0.5900; P < .001). False positives were largely non-overlapping (80.7% of mortality false postivies and 87.6% of readmission false positives were unique to 1 model), but false negatives were shared more often (55.0% and 55.2%). Embeddings encoded measurement frequencies more reliably than physiological values. Semantic model mortality errors concentrated in coagulopathic, hypoalbuminemic, hepatic, and high-density records; NM readmission errors were dominated by creatinine-centered phenotypes in sparse, imputed, low-volatility records.
Discussion
Numerical representations preserved quantitative physiology, whereas semantic embeddings emphasized monitoring intensity and syndromic care complexity.
Conclusion
Representation choice shapes not only aggregate performance but also which patients are misclassified, supporting hybrid architectures over winner-take-all replacement.