A Minibatch Method for Solving Nonlinear PDEs with Gaussian Processes

Abstract.

Gaussian process (GP)–based methods for solving PDEs demonstrate great promise by bridging the gap between the theoretical rigor of traditional numerical algorithms and the flexible design of machine learning solvers. The main bottleneck of GP methods lies in the inversion of a covariance matrix, whose cost grows cubically concerning the size of samples. Drawing inspiration from neural networks, we propose a minibatch algorithm combined with GPs to solve nonlinear PDEs. A naive deployment of a stochastic gradient descent method for solving PDEs with GPs is challenging, as the objective function in the requisite minimization problem cannot be depicted as the expectation of a finite-dimensional random function. To address this issue, we employ a minibatch method to the corresponding infinite-dimensional minimization problem over function spaces. The algorithm takes a minibatch of samples at each step to update the GP model. Thus, the computational cost is allotted to each iteration. Using stability analysis and convexity arguments, we show that, in expectation, the iterates of the minibatch algorithm approach a nearly stationary point of the optimal recovery problem used to solve the PDEs with rate [Formula: see text], where [Formula: see text] is the number of iterations and [Formula: see text] is the batch size.