Co-Fusion: Advancing 3D Object Detection with Collaborative Fusion

LiDAR–camera fusion for 3D object detection combines the metric geometry of LiDAR measurements with the semantic information in images. However, projected LiDAR depth is sparse and provides incomplete geometric guidance to the image branch, while the modality-specific branches follow different learning dynamics during multimodal training. We present Co-Fusion, a LiDAR-dominant collaborative fusion framework that extends the previously published Co-Fix3D detector. Co-Fusion introduces a LiDAR-depth guidance pipeline that standardizes projected LiDAR depth using dataset-level statistics and encodes the sparse measurements into features aligned with the image branch. A gated fusion network applies multiplicative modulation to the joint image–depth representation to produce depth-aware camera features. A three-stage parameter-transfer training schedule selectively transfers parameters from the LiDAR branch to initialize the camera and fusion branches. Experiments on the nuScenes 3D detection benchmark show that Co-Fusion reaches 74.1% mAP and 75.3% NDS in the single-frame multimodal setting without test-time augmentation.