Convergence of Machine Learning Methods for Feedback Control Laws: Averaged Feedback Learning Scheme and Data-Driven Methods

Abstract.

This work addresses the synthesis of optimal feedback control laws via machine learning. In particular, the averaged feedback learning scheme (AFLS) and a data-driven method are considered. Hypotheses for each method ensuring the convergence of the evaluation of the objective function of the underlying control problem at the obtained feedback laws toward the optimal value function are provided. These hypotheses are connected to the regularity of the value function and the stability of the dynamics. In the case of AFLS, these hypotheses only require Hölder continuity of the value function, whereas for the data-driven method, the value function must be at least [Formula: see text]. It is demonstrated that these methods are connected via their optimality conditions. Additionally, numerical experiments are provided by applying both methods to a family of control problems, parameterized by a positive real number, which controls the regularity of the value function. For small parameters, the value function is smooth, and in contrast with large parameters, it is nondifferentiable but semiconcave. The results of the experiments indicate that both methods have a similar performance for the case that the value function is smooth. On the other hand, if the value function is not differentiable, then AFLS has a better performance, consistent with the obtained convergence results.