From Hashed Records to Graduation Risk: Evaluating Foundation Models and Tabular Baselines on Longitudinal Higher-Education Data

In this paper, we explore the ability of generative models to integrate deterministically hashed representations of information in the context of higher education. We compare BERT, RoBERTa, and GPT-2 base configurations for predicting undergraduate graduation within the next four academic years using de-identified institutional records with values supplied as SHA-256 hashes and represented as token sequences. We construct a sample of 155,933 student-terms for 17,605 students. Each student appears in one split: 2012–2017 entry cohorts for training, 2018 for validation, and 2019–2021 for testing. XGBoost achieves the strongest discrimination on the held-out test set (AUROC 0.874), with logistic regression close behind (0.864). With all fields included, a GPT-2 hybrid model has the highest foundation-model AUROC, significantly exceeding RoBERTa and BERT (0.852 vs. 0.816 and 0.815, respectively); GPT-2 fine-tuned to directly generate the outcome label achieves 91.0% exact-match accuracy, but its AUROC (0.824) is lower than GPT-2 Hybrid’s. These findings suggest that generative models can extract and integrate non-semantic information embedded within hashed datasets, but their effectiveness in doing so remains limited relative to classical approaches.