Feature engineering over complex architectures: A comparative analysis of machine learning models for short-term water demand forecasting

ABSTRACT

Flowchart summarizing the research workflow including data preprocessing, feature engineering, model development, validation strategy, and evaluation and analysis.

Short-term water demand forecasting plays a pivotal role in the efficient management and optimization of urban water systems. This study investigates the performance of six forecasting architectures (XGBoost, Random Forest, LightGBM, support vector regression (SVR), long short-term memory (LSTM), and Prophet) to assess whether complex deep learning methods yield superior results compared with advanced ensemble approaches driven by rigorous, domain-informed feature engineering. Leveraging approximately 35,000 hourly water demand observations coupled with ERA5-Land meteorological data, the forecasting task was formulated as a supervised learning problem. A rigorous feature engineering framework was proposed, extracting 27 informative predictors, including cyclical temporal encodings, 168-h autoregressive lags, dynamic anchor features, and weather-related regressors. Model evaluation revealed that tree-based gradient boosting algorithms fundamentally outperformed deep learning (LSTM) and structural (Prophet) models. XGBoost delivered the highest aggregate accuracy (R2 = 0.966, RMSE = 0.394), forming a statistically significant high-performance cluster with LightGBM, Random Forest, and SVR. Furthermore, a hierarchical Shapley additive explanations interpretability analysis demonstrated that the 1-week autoregressive lag and dynamic anchor features were the primary drivers of prediction, while meteorological variables acted as secondary microtuning parameters. Finally, introducing a novel residual-engineered hybrid architecture (e.g., Hybrid_XGB_XGB) yielded further statistically significant improvements, establishing a highly accurate and computationally efficient framework for operational forecasting.