NS‐SLAM: High‐Accuracy Object‐Level Semantic SLAM With Neural Radiance Fields and Zero‐Shot Segmentation

Object‐level semantic simultaneous localization and mapping (SLAM) plays a vital role in enabling autonomous systems to perceive, understand, and interact with complex environments. However, conventional methods often suffer from low geometric modeling accuracy, strong reliance on predefined object categories, and poor generalization to unseen objects. To address these challenges, we propose NS‐SLAM, a novel framework that integrates neural radiance fields (NeRFs) with a zero‐shot segmentation foundation model (SEEM). NS‐SLAM achieves high‐precision object‐level reconstruction and localization via a hash‐encoded NeRF representation while leveraging SEEM to provide dense, pixel‐level semantic segmentation without requiring prior object knowledge. The semantic information is extracted by feeding high‐fidelity NeRF‐rendered views into the SEEM model, followed by a back‐projection strategy to associate 2D semantics with 3D geometry. Extensive experiments on the TUM RGB‐D dataset and a newly collected office dataset demonstrate that NS‐SLAM significantly outperforms representative baselines such as QISO‐SLAM and QuadricSLAM. Specifically, NS‐SLAM reduces the absolute trajectory error (ATE) root mean squared error (RMSE) by 8.2% on average and improves 3D modeling quality metrics (depth L1, peak signal‐to‐noise ratio [PSNR], and SSIM) by over 10%, while demonstrating robust zero‐shot semantic recognition capabilities, successfully identifying and segmenting both known and unseen object categories. These results validate the effectiveness of NS‐SLAM in enhancing both the precision and generalization capabilities of object‐level semantic SLAM, offering a promising solution for robotic applications in complex environments with unknown objects.