Efficient 3D Semantic Occupancy Prediction via Integrated 2D-3D Feature Fusion

This paper presents a novel approach to 3D semantic occupancy prediction that leverages the integration of 2D and 3D features. Traditional 3D voxel representations, while detailed, are computationally intensive. Our method addresses this challenge by encoding 3D voxel features into a Bird’s-Eye View (BEV) representation, then decoding them back into voxels using a multi-layer perceptron (MLP). This fusion approach reduces computational resources compared to voxel-only methods while maintaining state-of-the-art accuracy. By leveraging multi-scale features and deformable attention mechanisms, our network achieves a mean intersection-over-union (mIoU) of 41.26% on the Occ3D-nuScenes dataset, outperforming recent methods. By introducing a 3D backbone and multi-scale voxel-to-BEV-to-voxel feature transformation, the model achieves improved mIoU with moderate computational overhead. Our results demonstrate the effectiveness of integrating 2D and 3D information for accurate 3D semantic occupancy prediction, providing a potentially useful scene representation for downstream autonomous driving tasks.