Comprehensive Validation of Novel Deep Learning Architectures to Forecast Geomagnetic Substorms

Abstract

Magnetic substorms are disturbances in the terrestrial magnetosphere that can have significant space weather impacts, although forecasting their onset accurately remains an open problem. In this study, we develop multiple novel machine learning architectures based on convolutional neural networks, long short‐term memory networks, extreme gradient boosting, and their hybrid combinations for the probability prediction of magnetic substorm onsets. The best performing models came from hybrids in the following order: two‐dimensional convolution, bidirectional long short‐term memory, and finally extreme gradient boosting for binary classification probability prediction with an F1 score of 0.8411 across both classes. Notably, a novel modified stratified K‐fold cross‐validation benchmark is developed for these deep learning models; the F1 scores fell within 79%–85% across model architectures. The results demonstrate the ability of models to unambiguously forecast substorm onsets from upstream solar wind and IMF conditions. Two‐hr historical windows of solar wind velocity, proton number density, and three‐dimensional interplanetary magnetic field components, sampled at 1‐min cadence, are used to forecast the onset probability at the next minute after the window. The input data set was balanced using the SuperMAG list of substorm onsets between 2003 and 2022, with an equal number of non‐substorm intervals, to avoid bias. Principal component analysis showed significant overlap in the solar wind and IMF conditions prior to both substorm and non‐substorm intervals, indicating that these parameters alone may not be sufficient to forecast all substorms and that internal magnetotail preconditioning and magnetospheric processes are likely critical areas that should be explored further.