DOI: 10.3390/technologies14110720 ISSN: 2227-7080

Training-Free Post-Inference Fusion for RGB–Thermal Object Detection: An Evaluation with a Negative Result for Reliability-Aware Rules

Turki Saud Alotaibi, Ahmed Saud Alanizi, Qazi Emad Ul Haq, Tariq M. Khan

Late fusion can combine the outputs of separately trained RGB and thermal detectors without retraining them. We evaluate such training-free fusion, together with a rule-based reliability-aware controller that adapts per-frame thresholds and modality weights, on M3FD, FLIR-aligned, and FLIR ADAS v2. Box-level COCO metrics on the registered M3FD and aligned FLIR-aligned pairs are the primary results, and frame-level class-set metrics are secondary. At the operating threshold of 0.35, fixed fusion of two YOLOv8n detectors raised AP50 over the thermal detector from 0.319 to 0.377 on M3FD and from 0.623 to 0.666 on FLIR-aligned (from 0.724 to 0.758 over all detections down to a confidence of 0.001). Over all detections, the FLIR-aligned gain was 0.0198 to 0.0345 over three training seeds, 0.0168 when the detectors were trained for 60 instead of 30 epochs, and 0.0222 with larger YOLOv8m detectors (AP50 from 0.797 to 0.819); the 60-epoch and YOLOv8m results are single runs and do not show whether the gain decreases for better detectors. With YOLOv8m, fusion still raised AP50 but lowered box-level F1 at the operating threshold of 0.35 used for all systems, a reduction observed at this single operating point. On the unregistered FLIR ADAS v2 pairs, fusion lowered AP50, and its frame-level gain depends on how the frame label is defined. On the same detections, our single-score ProbEn-style rule was 0.092 AP50 below fixed fusion; published results obtained with other detectors and evaluation code are not directly comparable. The controller shifts the operating point toward slightly higher precision and lower recall; a threshold change of fixed fusion reproduces this shift; and at equal recall, fixed fusion is at least as precise, so we report the controller as a negative result. Fusion takes 0.4–1.0 ms per frame on a CPU for detections with a confidence of at least 0.35 and 8.8–9.6 ms when all detections are fused.

More from our Archive