Multimodal AI for Same-Session Estimation of PHQ-9 Screen-Positive Status from Counselor-Conducted Fixed-Structure Interviews: A Preregistered Internal Validation Study of Incremental Value and Calibration

Background/Objectives: Multimodal mental health assessment integrates acoustic, facial, and linguistic signals. We tested whether Gaussian mixture model (GMM) posterior summaries added value beyond participant-level factor analysis (FA) means for same-session estimation of Patient Health Questionnaire-9 (PHQ-9) screen-positive status. Methods: This preregistered secondary analysis included counselor-conducted fixed-structure interview results from 103 participants (83 adults, 20 pediatric participants; 24 screen-positive). The pediatric outcome was a study-specific operational label because criterion validity was not established for the administered form and cutoff. Repeated nested cross-validation compared FA+GMM and FA, both including a cohort indicator, with a design-only benchmark. Average precision (AP) was the primary metric; the no-skill baseline was 24/103 = 0.233. Results: In the primary analysis, mean AP was 0.302 for FA, 0.282 for native-scale FA+GMM, and 0.332 for the design-only benchmark. Mean paired FA+GMM-minus-FA differences were −0.020 for AP, −0.031 for the area under the receiver operating characteristic curve, and +0.0016 for Brier score. Conditional bootstrap comparisons with the design-only benchmark were imprecise. Calibration slopes after cross-fitted recalibration were 0.104 for FA and 0.093 for FA+GMM. Post hoc training-fold standardization increased FA+GMM AP to 0.300, close to FA. Refitting with a pediatric threshold of 11 did not alter the incremental-value conclusion. Conclusions: GMM summaries did not demonstrate incremental value under the evaluated conditions. Standardization attenuated the native-scale deficit, identifying implementation sensitivity as a model-selection consideration. Modest discrimination and unstable calibration constrain individual probability interpretation. These results support evaluating added representation complexity against simpler, design-aware benchmarks and assessing calibration alongside discrimination.