An Optuna-Optimized XGBoost Framework for Robust Non-Technical Loss Detection in Smart Grids

This study presents a robust, end-to-end machine learning framework for detecting Non-Technical Losses (NTLs) in smart power grids. The proposed Optimized Bootstrapped Gradient Classifier (OBGC) integrates k-Nearest Neighbor (kNN) imputation for missing data handling, Adaptive Synthetic Sampling (ADASYN) for class imbalance correction and Optuna-based hyperparameter optimization. The framework is evaluated using the State Grid Corporation of China (SGCC) dataset comprising 42,373 consumers over 1035 days (January 2014–October 2016), characterized by 25.6% missing values and a severe class imbalance (91.46% honest vs. 8.54% theft-prone consumers). To ensure robust evaluation, all experiments employ stratified 5-fold cross-validation on the training set, with a held-out 20% test set reserved for final performance assessment. Experimental results show that OBGC achieves an accuracy of 96.61%, precision of 96.18%, recall of 97.13%, and F1-score of 96.65%, substantially outperforming baseline XGBoost (accuracy 92.75%, F1 36.11%) and other state-of-the-art classifiers. An ablation study confirms that ADASYN is the most critical component; its removal causes recall to collapse from 97.13% to 16.87%, demonstrating the severe impact of class imbalance on theft detection. Optuna-based tuning contributes an additional 3.29 percentage-point gain in accuracy over the unoptimized baseline. The framework offers a transparent, reproducible, and operationally feasible solution for real-world NTL detection.