DOI: 10.1371/journal.pone.0361401 ISSN: 1932-6203

IV-PAFNet: Infrared-visible cross-modal alignment and fusion network for nighttime UAV object detection

Cheng Yuan, Shichang Shang, Jinghao Zhang, Yang Zhang, Bin Zhou

Nighttime unmanned aerial vehicle object detection is challenged by degraded visible-light textures, cluttered infrared thermal responses, and local spatial discrepancies between infrared and visible images. To address these issues, we propose IV-PAFNet, an infrared-visible purification, alignment, and fusion network designed for object detection in nighttime and low-visibility conditions. IV-PAFNet consists of two primary components. The infrared signal purification and spatial alignment module applies learnable high-pass filtering and gated residual fusion to refine infrared responses, followed by visible-guided offset prediction and differentiable resampling to alleviate local cross-modal spatial discrepancies. The multi-scale fusion attention enhancement module performs channel-wise modality recalibration and spatial reliability weighting at multiple feature-pyramid levels, allowing the network to adapt the contribution of each modality to local imaging conditions. In the training-based DroneVehicle and generalization-based Camera-vehicle1 datasets, IV-PAFNet achieves 62.81% mAP@0.5:0.95 and 51.4% mAP@0.5:0.95, respectively. Additional controlled experiments investigate the distinct roles of infrared purification, visible-guided alignment, channel recalibration, and spatial reliability modeling, while statistical and computational analyses characterize run-to-run variation and inference cost. The code is available at https://github.com/iv-pafnet/IV-PAFNet .

More from our Archive