Integrating Three-Parameter Logistic IRT Models and Confirmatory Factor Analysis for Multidimensional Assessment of Academic Performance and Associated Factors in University Leveling Programs

This study integrated Item Response Theory (IRT) models with ordinal survey instruments to establish a baseline psychometric framework and identify multidimensional factors associated with academic achievement among first-semester leveling students (N = 1558 pre-test; N = 1676 post-test) at the Escuela Politécnica Nacional, Ecuador. A dual-component methodology was employed in this study. Initially, an 80-item ordinal survey was utilized to assess eight latent constructs, yielding substantial validation metrics through Confirmatory Factor Analysis (CFA). Secondly, structured diagnostic assessments in core STEM and language subjects were calibrated using three-parameter logistic (3PL) IRT models via Expected A Posteriori (EAP) estimation. Results demonstrated high internal consistency (r = 0.93 between IRT and raw scores), with mean IRT-scaled ability θ¯ = 10.45 (SD = 3.51) on a 1–20 scale. Estimated item parameters yielded a mean discrimination of a¯ = 1.92 and a centered mean difficulty of b¯ = 0.05. The Orlando–Thissen S−X2 goodness-of-fit test, applied at a significance threshold of p < 0.01, identified 19 items (23.75%) whose observed response patterns deviated significantly from model predictions, with the majority concentrated in the physics and chemistry content domains. Factor scores and performance outcomes were statistically contrasted against 24 categorical demographic variables, revealing differential performance patterns across student subgroups. This research provides validated psychometric instruments, reproducible IRT-LMS integration protocols, and empirical evidence supporting targeted interventions to strengthen university transition.