A Hierarchical Hybrid Framework for Real‐Time Dense Semantic Mapping With Semantic Rendering and Dynamic Keyframe Optimisation

ABSTRACT

Real‐time dense three‐dimensional (3D) semantic mapping plays a crucial role in robotics. However, existing explicit and implicit representations often entail a trade‐off among geometric precision, semantic richness and computational efficiency. To address this challenge, we propose a hierarchical hybrid mapping framework that unifies geometry, appearance and dense semantics within a shared multiresolution hash grid, enabling end‐to‐end optimisation via differentiable rendering. By caching per‐keyframe semantic masks and reusing them during incremental refinement, our method eliminates redundant semantic segmentation and significantly accelerates runtime. Additionally, an adaptive keyframe pruning strategy further ensures bounded memory usage. Extensive experiments demonstrate that our framework achieves real‐time performance with superior geometric and semantic accuracy.