DAFormer: Enhancing Infrared-Visible UAV Perception via Degradation Aware Mixture-of-Experts Model

Infrared-visible image fusion, which enhances the perceptual deficiencies of a single modality, is commonly used for surveillance, detection, and tracking tasks. However, an important issue in the all-weather perception of unmanned aerial vehicles (UAVs) is the impact of distortion from the outdoor scene. Unlike ideal imaging, the captured images in practical outdoor environments are affected by weather and temperature such as rain and fog. This paper proposes DAFormer, an infrared-visible restoration fusion solution, which enhances the collaborative perception performance from degraded Infrared-visible images. The DAFormer comprises a dual-branch encoder embedded with the Degradation Aware Mixture of Experts (DA-MoE), a Switch Fusion Module (SFM), and a single-branch decoder. Specifically, DA-MoE decouples the distortion features across spatial and channel dimensions through residual learning, utilizing a dual-branch encoder to isolate cross-modal influences. Later, the SFM iteratively trains DAFormer by alternating between restoration and fusion paths, driving convergence toward restoration-fusion objectives. Experiments conducted in high-altitude monitoring and overhead surveillance scenarios demonstrate the superior ability to remove unpaired distortion across modalities, producing fused results that preserve crucial information and align with human visual perception. The DAFormer's code can be found at: https://github.com/suweijian1996/DAFormer.