Rigorous nested cross-validation and external generalization-gap analysis of a machine-learning classifier for U-251 glioblastoma cytotoxicity

Glioblastoma remains difficult to treat, and drug-repositioning strategies require efficient methods to evaluate chemical libraries. Here, we developed a machine-learning binary classifier for compound activity against U-251 glioblastoma cells using a corrected single-center dataset of 236 compounds evaluated under a standardized cell-viability assay. Compounds were labeled as active or inactive using a predefined viability criterion (<50% vs. ≥ 50%) within a single-point screening framework using 10 µM as the upper screening concentration. To reduce optimistic bias and information leakage, we used repeated nested cross-validation in which preprocessing, feature selection, and hyperparameter optimization were performed within the training folds. ExtraTrees was selected based on Matthews correlation coefficient, showing moderate internal performance (ROC-AUC, 0.76; balanced accuracy, 0.66; MCC, 0.36). Structural analyses using nearest-neighbor Tanimoto similarity and Bemis–Murcko scaffolds indicated that the training set was scaffold-diverse with limited series bias. However, when the locked model was applied to an independent external set of 53 compounds, all compounds were classified as inactive, resulting in zero sensitivity, balanced accuracy of 0.5, and MCC of 0. The external ROC-AUC based on continuous prediction scores was 0.37, indicating poor threshold-independent ranking performance. Nearest-neighbor Tanimoto similarity did not explain prediction correctness or rescue active-compound recovery. Y-randomization supported the presence of a non-random internal signal, whereas scaffold-grouped cross-validation suggested that internal ROC-AUC was not solely driven by scaffold leakage. Overall, this study provides a reusable dataset and reproducible workflow, and highlights a substantial generalization gap between internal validation and external prediction in small chemical machine-learning datasets.