Three-Dimensional Deep Learning with Routine Brain Magnetic Resonance Imaging and Clinical Data for Identification of Secondary Progressive Multiple Sclerosis

Objectives: Secondary progressive multiple sclerosis (SPMS) is a natural transition from relapsing-remitting multiple sclerosis (RRMS) in many cases. However, whether and how these phenotypes differ on an individual basis is not fully understood, limiting timely diagnosis and management for SPMS. This study aimed to investigate how deep learning using 3-dimensional (3D) frameworks including VGG19, ResNet152, and DenseNet-121 helped differentiate SPMS from RRMS based on routine clinical datasets, and what brain areas mostly contributed to this differentiation using model explanation techniques. Methods: We examined 140 participants (70 each for RRMS and SPMS) as part of an ongoing study comprising prospectively collected clinical and imaging data from routine healthcare. The data was curated to improve consistency and completeness using different strategies and were then randomly split by subject into training (n = 120) and held-out testing (n = 20). The former was used for model development through five-fold cross validation. Deep learning used T1-weighted, T2-weighted, and FLAIR brain MRI, with optional clinical variables (n = 6). A 3D gradient-weighted class activation mapping (Grad-CAM) technique was applied to identify brain areas of significance followed by ablation studies for additional insight. Results: Among the 3D frameworks validated, VGG19 was deemed the best. Based on MRI and the best 3D VGG19 model, different data curation strategies showed largely similar results. Additionally, the models combining clinical variables with MRI achieved equivalent or slightly greater performance than MRI-only models, with an average testing area under the receiver operating characteristic curve of 0.84 when datasets were fused at the flatten layer, best at 0.92, versus 0.82 and 0.89. Model explanation indicated brain regions of significance in distinguishing SPMS from RRMS individuals, including bilateral frontal lobes, left occipital and temporal lobes, and cerebellum. Conclusions: Overall findings suggest the potential of 3D deep learning models such as VGG19 for distinguishing SPMS from RRMS using routine brain MRI and clinical data, which, along with 3D Grad-CAM, could facilitate discovery of new biomarkers underlying disease worsening.