Data defects and AI-mediated answer reliability: Experimental evidence from Chinese municipal transportation and education/public-service open data

This study examines how defects in Chinese municipal transportation and education/public-service open data affect AI-mediated answer reliability. The two-stage experiment evaluated 24 datasets, 144 questions, and 4032 trials. Stage 1 tested five theory-informed defect families at ordered perturbation levels with two model configurations; Stage 2 used new datasets and questions with four configurations. A subsequent 1152-trial harmonized validation tested field-name masking, record loss, and unlabeled scale shifts across all 24 datasets with the two model families common to both stages. In Stage 2, all three defects reduced strict-pass performance across all four configurations. In the harmonized validation, all six model-by-defect effects were negative in both data-setting groups; evidence was strongest for field-name masking and record loss, while scale shifts were more heterogeneous. The findings connect published-data properties to answer reliability and demonstrate a controlled approach for studying the robustness of AI-mediated access to public data.