Unbiased fully nonlinear data assimilation. Part 1: The stochastic particle flow filter

Abstract

Data assimilation methods that can handle nonlinear dynamics and non‐Gaussian distributions effectively remain a significant challenge in high‐dimensional geophysical systems. To address this, we introduce the stochastic particle flow filter (SPFF), which enhances the recently developed deterministic particle flow filter (PFF) by incorporating an additional stochastic term, in the first part of a two‐part article. While the deterministic PFF has shown promise in high‐dimensional applications, it tends to underestimate ensemble spread when using a finite number of particles, particularly in sparsely observed scenarios. Our stochastic variant acts as a stochastic gradient Markov Chain Monte Carlo sampler that is unbiased for any ensemble size and converges asymptotically to the exact posterior distribution. Through experiments with the 1000‐dimensional Lorenz 96 model, we demonstrate that the SPFF achieves accuracy comparable with the existing PFF method with a matrix‐valued kernel and is able to capture multimodal distributions while providing a nearly unbiased ensemble spread.