Impact of vanishing regularization on the early time dynamics and valid prediction times of reservoir computing

We study the impact of the regularization coefficient λ on Reservoir Computer (RC) performance for chaotic time series prediction of the Lorenz system. We find that a larger λ results in a larger error of the first prediction step of the RC. The RC error initially evolves according to a rapid, nonautonomous expansion at very early times followed by the expected exponential, Lyapunov growth of the error. The maximum Lyapunov exponent of the prediction is close to the maximum Lyapunov exponent of the Lorenz system regardless of λ, whereas the error of the first prediction step and the rate of the nonautonomous growth are λ-dependent and are responsible for the λ dependence of the Valid Prediction Time (VPT) of the RC. Interestingly, we find that the VPT can exceed 30 Lyapunov times for vanishingly small λ, as we are predicting a noiseless system with a small sampling step and spectral radius. Moreover, we emphasize the importance of the numerical solver used to generate the Lorenz dataset and define a Valid Ground Truth Time (VGTT) during which the outputs of several common solvers agree. A VPT exceeding the VGTT is not meaningful, as a different solver could produce a different result.