Enhancing Socially Aware Construction Robot Navigation through LiDAR-Camera Sensor Fusion and Dynamic Object Tracking

Abstract

Dynamic object detection, tracking, and path planning are critical for enabling robots to navigate safely, efficiently, and socially in human-centered environments such as construction sites. Although light detection and ranging (LiDAR)-based simultaneous localization and mapping (SLAM) and occupancy-grid methods can detect motion, advanced three-dimensional (3D) vision techniques often depend on pretrained neural networks and require extensive postprocessing to identify dynamic objects. Sensor-fusion approaches that combine LiDAR accuracy with red, green, and blue (RGB) semantic information offer a compelling and lightweight alternative. This paper proposes an integrated framework that equips a quadruped robot with a LiDAR sensor and an upward-facing fisheye camera to perform 360° real-time dynamic object detection and tracking and to evaluate its impact on socially aware navigation. The system first extracts moving objects from a registered point cloud, then projects the detected objects to a cylindrical panorama, where 3D tracks are associated with real-time image-based detections to update a Kalman filter. To assess how perception quality influences navigation performance, we incorporate the resulting detections into a social force model (SFM) enhanced dynamic window approach (DWA) local planner and compare two perception pipelines: LiDAR-only detection versus LiDAR-camera sensor fusion. Experimental results in real-world testbeds with moving human workers show that the SFM-augmented DWA planner improves collision avoidance and social compliance while maintaining navigation efficiency and trajectory smoothness, with further gains when using sensor-fusion-based human detection and tracking. The proposed system demonstrates high precision and robustness in perception and planning, making it well-suited for real-world deployment on dynamic construction sites.