Reinforcement Learning-Assisted Multi-Objective Optimization for Multi-Time Scale Building Microgrid Scheduling

This paper adopts a hierarchical multi-time scale coordinated scheduling framework to jointly optimize occupant thermal comfort and operating cost in building microgrids. Operating cost measures system economic performance, while thermal discomfort is quantified by the time-weighted L2 norm of predicted mean vote (PMV). With other thermal comfort parameters held constant, PMV is evaluated using the indoor temperature obtained from a first-order RC building thermal model. In the day-ahead stage, the optimization is carried out based on day-ahead forecasts, with the operating cost and the thermal discomfort index minimized simultaneously. In the intraday stage, short-term forecasting information is used to update the day-ahead baseline schedule through rolling optimization. To ensure continuous scheduling, a power constraint based on a state-of-charge (SOC) trajectory corridor is further proposed for the battery energy storage system, thereby guaranteeing consistency between the initial and terminal SOC. The resulting bi-objective optimization problem is solved via a reinforcement learning-assisted multi-objective particle swarm optimization (MOPSO) algorithm. Compared with fixed-parameter MOPSO under the same runtime budget, the proposed algorithm increases hypervolume by 34.64% and reduces inverted generational distance by 87.14% on average. In a tested scenario, intraday rolling optimization reduces operating cost and thermal discomfort by 6.24% and 15.91%, respectively, relative to day-ahead scheduling alone.