Beyond Accuracy: Multi-Level Ordinal Assessment of Lumbar Spine Degeneration from Multiplanar MRI Using the RSNA 2024 (LumbarDISC) Dataset and Condition-Specific Models

Automated assessment of lumbar spine degeneration from magnetic resonance imaging (MRI) is commonly evaluated using aggregate classification metrics, potentially overlooking clinically relevant differences in severity misclassification, anatomical variability, and patient-level performance. This study proposes a Beyond-Accuracy Multi-Dimensional Evaluation Framework using the RSNA 2024 LumbarDISC dataset. The framework integrates six complementary dimensions: exact classification, ordinal severity, condition-specific analysis, spinal-level analysis, patient-level assessment, and reliability and robustness, alongside comparisons of imaging representations. To demonstrate its applicability, condition-specific models were developed for five lumbar degenerative conditions using ViT, MaxViT, ConvNeXt, and EfficientNet architectures with 2D and cross-level 2.5D-to-3D inputs. Experimental findings reveal substantial performance variation across conditions, spinal levels, and architectures, demonstrating that similar aggregate accuracy can conceal differences in ordinal error severity and patient-level error patterns. The evaluated 2D models generally outperformed their 2.5D-to-3D counterparts, suggesting that additional cross-level context does not necessarily improve classification. Overall, the proposed framework provides a structured approach to evaluating lumbar MRI models beyond aggregate accuracy, supporting more informative model comparison and clinically oriented assessments.