DOI: 10.3390/electronics15214957 ISSN: 2079-9292

Stage-Specific Semantic–Structural Descriptor Conditioning for RGB–Thermal Object Detection

Bing Liu, Li Bao, Yuxuan Gong, Yuanhao Huang

RGB–thermal detection benefits from both cross-modal feature calibration and the recovery of fine-scale structural evidence. However, spatial responses produced during these operations are often used only to regulate intermediate features. We propose a stage-specific semantic–structural descriptor-conditioning framework that retains calibration and discrepancy maps from P4 and structural-weighting and propagation maps from P2→P3. At P3–P5, the four maps are jointly encoded with pyramid features to generate separate, bounded adjustments for classification and regression. On FLIR Aligned, the framework achieves 46.83±0.10% mAP50:95 across six seeds, compared with 43.80±0.16% for the matched baseline (paired gain 3.03±0.15 points), using 7.51M parameters and 25.96 GFLOPs. It also outperforms a compute-matched ordinary P2 control by 1.72±0.10 points. Structural recovery provides the largest standalone gain. Under matched feature-path and prediction-head architectures, a six-seed four-way descriptor-group control shows a modest positive semantic–structural interaction (+0.066±0.031 points). Live descriptors exceed dual-neutral and cross-sample-permuted controls by 0.43±0.05 and 0.57±0.10 points. Independent retraining on M3FD yields a 3.17-point mAP50:95 gain. Stress tests further characterize sensitivity to spatially varying thermal misalignment.

More from our Archive