Grouped Validation and Conformal Calibration for Clustered Cross-National Data with an Application to Harmonized Learning Outcomes

Clustered cross-national panels pose two problems: observation-level validation can be optimistic when the same economy appears in training and evaluation, while standard split conformal coverage requires exchangeability that pooled cells from heterogeneous economies do not justify. We study prediction, attribution, and interval calibration using 1565 Harmonized Learning Outcomes results from 144 economies. Performance is estimated by nested cross-validation grouped by economy and a temporal split; explanations are computed on evaluation folds using Shapley values and permutation importance. XGBoost attains R2=0.703, versus 0.593 and 0.595 for linear baselines, and R2=0.819 temporally. Under a country-held-out conformal design, three calibration rules are compared on identical predictions. At 90% nominal coverage, pooled calibration yields a cell-weighted and country-weighted coverage of 0.872 and 0.794 with a mean width of 118.9; equal-economy-calibration yields of 0.930 and 0.875 at 148.5; and subsampling yields of 0.961 and 0.937 at 180.4. Across 50 repeated calibration splits with re-selected XGBoost hyperparameters, the width ordering of the rules holds in every replicate, the cell-weighted coverage gain from pooled to equal-economy calibration is positive in 49, and the width increase averages 22 points; in the primary analysis, selecting the model class inside each development partition returns the same models in every fold. The observed coverage for lower-middle-income economies is below nominal under the first two rules and persistently so across calibration splits, although the economy-bootstrap interval under equal-economy calibration does not exclude the nominal level. Attribution, conditional on the included indicators and fitted models, emphasizes mortality under age five and adult schooling. These results show that cluster-aware validation and both coverage functionals materially affect predictive reliability.