Data Quality and Predictive Analytics in Official Statistics: An Asymmetric Hybrid Estimator for Poverty Estimation Under MNAR Measurement Error

Background: Can predictive algorithms trained on survey data affected by systematic measurement error meet official statistics quality standards? This study examines the tension between design-based inference and algorithmic modelling in poverty estimation, focusing on regressive income underreporting in the Italian shadow economy. Methods: A Monte Carlo microsimulation protocol of 1000 replications is calibrated on ISTAT and EU-SILC 2025 statistics. The shadow economy is formalised as Missing Not At Random measurement error with spatial heteroscedasticity across three Italian macro-regions. Five estimators—design-based, algorithmic, and hybrid—are compared and audited through the QOSU framework. Results: The GREG retains asymptotic design-unbiasedness on undistorted data but exhibits bias up to +29% on observed data, showing that calibration is insufficient under non-ignorable error. Uncalibrated machine learning estimators display systematic bias via shrinkage or amplification, termed statistical hallucination. The Asymmetric Hybrid, with a parameter-free quadratic weighting function, achieves GOLD in five of six domain–indicator combinations; a hold-out validation identifies a reliability threshold of 0.70 as the framework’s domain of validity. Conclusions: Integrating machine learning into official statistics requires an adaptive architecture in which each component’s contribution is modulated by the reliability of the information base. The QOSU framework provides a reproducible protocol for auditing algorithmic estimates under non-ignorable measurement error.