A lightweight RGB-D cross-modal learning network for rail defect detection

Abstract

Rail surface defects pose serious risks to railway transportation safety, and accurate real-time detection is essential for reliable railway maintenance. Existing RGB-only or depth-only methods are often limited by low contrast, uneven illumination, subtle defect patterns, insufficient annotated data, suboptimal cross-modal fusion, and high computational cost. To address these challenges, this study proposes BiMaL, a Bidirectional Modality-aware Lightweight framework for RGB-D rail surface defect detection. BiMaL consists of three stages: bidirectional cross-modal self-supervised pre-training, lightweight RGB-D fusion network construction, and importance-aware dependency-preserving structured pruning. First, a BYOL-based self-supervised strategy learns discriminative RGB and depth representations from unlabeled data, reducing reliance on manual annotations. Second, a compact fusion network integrates RGB texture cues and depth structural information through lightweight local complementary and global correlation pathways. Third, a channel-importance-aware pruning strategy compresses redundant encoder channels through dependency graph constraints while deliberately preserving key fusion and detection components for edge deployment. Experiments on the NEU RSDDS-AUG dataset show that BiMaL achieves a Structure Measure of 0.865, maximum F-measure of 0.901, maximum enhanced-alignment measure of 0.943, and mean absolute error of 0.061. The final model contains only 2.2 million parameters and reaches 101.8 frames per second. These results demonstrate that BiMaL achieves a favorable balance between detection accuracy and computational efficiency, providing a practical solution for real-time rail surface defect inspection on resource-constrained mobile platforms.