Beyond Predictive Accuracy: Reference-Aligned Evaluation of Explainable AI in Distribution Networks
Luis Fernando Tipán Tipán VergaraFeature attributions can describe a fitted predictor while differing from a declared engineering reference. This study develops a reference-aligned audit that keeps predictive performance, attribution agreement, estimator precision, and reference evidential status separate. Two public distribution-network datasets provide exact additive controls: 53,567 Swiss meter records and 240 Ecuadorian utility-month records. Under a common finite-background marginal game, the Swiss multilayer perceptron achieved R2=0.99979 with normalized attribution distance 0.037, whereas gradient boosting and random forest retained R2>0.97 with distances of 0.480 and 0.706. In Ecuador, exact enumeration of all 16 coalitions yielded R2=0.99846 and a distance of 0.399 for the MLP, despite perfect four-feature rank agreement. Replacing the three billed-and-delivered outflow variables jointly in a two-group game preserved this result (0.405). Clustered-bootstrap associations between prediction error and zero-sum attribution discrepancy were weak for the MLP in Switzerland (ρs=0.238, 95% CI [0.025,0.440]) and Ecuador (ρs=0.071, 95% CI [−0.361,0.460]); these estimates do not establish independence. Company-balanced analysis exposed a CNEL-Sucumbíos failure hidden by pooled performance, while sample-size and multi-seed audits preserved the qualitative attribution-distance ordering. An all-meter voltage audit did not qualify observational regressions as physical ground truth. The conclusions are restricted to declared additive references and attribution games and do not constitute general physical validation.