Distinguishing Contemporaneous Rule Recovery from Financial Risk Forecasting in Higher Education Institutions: A Controlled Synthetic-Panel Experiment
Yu Chao, Nur Fazidah Elias, Yazrina Yahya, Ruzzakiah Jenal, Mo FanHigh classification performance in financial-risk early-warning research may reflect recovery of a constructed contemporaneous rating rule rather than prediction of an independent future outcome. This study distinguishes these two forms of evidence through a controlled synthetic-panel experiment situated in the context of higher education institutions (HEIs). Task 1 is a contemporaneous positive-control audit in which the four components defining a deterministic three-class rating are supplied to the classifiers. Task 2 uses predictors measured at year t−1 to classify the rating at year t through strict rolling-origin evaluation under prespecified Weak, Moderate, and Strong temporal-persistence conditions. In Task 1, the four fitted classifiers achieved held-out Macro-F1 values of 0.9968–1.0000, demonstrating near-complete recovery of the disclosed rating rule but providing no prospective forecasting evidence. In Task 2, the best mean annual Macro-F1/Macro-AUC increased from 0.480/0.696 under Weak persistence to 0.549/0.747 under Moderate persistence and 0.633/0.824 under Strong persistence. The Weak–Moderate–Strong ordering was observed for both primary metrics across all four fitted classifiers, while higher model complexity provided no consistent advantage within the disclosed synthetic mechanism. These findings confirm contemporaneous rule recoverability and sensitivity to deliberately embedded temporal persistence only within the controlled experiment. They do not establish predictive validity, transportability, or decision benefit in real HEIs. The study contributes a reproducible task-to-claim approach that aligns target construction, predictor–target overlap, information timing, rolling-origin evaluation, probability quality, and model performance with the inferences that the resulting evidence can legitimately support. Real-world validation would require source-indexed longitudinal HEI data, independently adjudicated post-origin outcomes, verified information-availability dates, external testing, and prospective decision evaluation.