Method for predicting eutrophication in water bodies based on dual experience pool TDDPG and DDPG-dual 3Q learning mode

ABSTRACT

A three-layer flow diagram of a deep reinforcement learning framework for water eutrophication prediction. It shows multi-factor water quality data input and preprocessing, three core models (DDPG-Double 3Q fusion, improved Double 3Q learning with LSTM and Transformer, and dual experience pool TDDPG), and output evaluation with prediction curves and metrics (MAE 0.012, RMSE 0.018, MAPE 3.5%, TLI 52.4).

Water eutrophication prediction remains challenging due to poor long-term feature retention, susceptibility to local optima, and difficulties in balancing smooth and abrupt time-series patterns. To address these issues, this study develops a two-stage reinforcement-learning forecasting framework in which Transformer-based temporal representation, auxiliary replay learning, and error-aware Q-value selection are jointly organized for multivariate eutrophication prediction. In the first stage, the TDDPG model replaces the conventional actor representation in DDPG with a Transformer-based temporal feature extractor and uses an auxiliary replay buffer to reduce the effect of strongly correlated sequential samples during policy learning. In the second stage, the DDPG-Double 3Q model treats the outputs and errors of several DDPG-based predictors as decision states, allowing the final prediction policy to select and refine candidate predictions under both gradual and abrupt water-quality variations. Experimental validation using multi-factor water quality monitoring data demonstrates that the proposed framework achieves an average improvement of 35% across key evaluation metrics – Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE) compared to baseline models such as ADDPG and RDPG. The results indicate that the framework improves prediction accuracy and training stability in the tested dataset, suggesting that reinforcement learning can provide a useful sequential decision-making formulation for multivariate eutrophication forecasting.