Robust Post-Processing for Marine GNSS/INS Integration: An Adaptive RTS Smoothing Approach via Huber M-Estimation

GNSS/INS integrated navigation systems play a critical role in marine navigation, providing high-precision position and attitude information for moving platforms. However, in complex marine environments—such as occlusions caused by offshore engineering platforms—GNSS signal attenuation frequently leads to a rapid degradation of positioning accuracy. To address this issue in post-processing applications, this paper proposes an Adaptive Rauch-Tung-Striebel Smoother (ARTSS)-based GNSS/INS integrated navigation method. The proposed method first performs forward filtering using an Error-State Extended Kalman Filter (ESKF). Subsequently, an adaptive equivalent weight is dynamically constructed using the Huber M-estimation cost function based on the forward filtering innovations. This adaptive factor is utilized to dynamically modulate the smoothing gain in the backward pass, thereby effectively suppressing the interference of measurement outliers. To verify the effectiveness of the algorithm, comparative experiments are conducted using real-world land vehicle and shipborne kinematic datasets. Three methods are evaluated: the standard ESKF, the fixed-interval backward smoothing (RTSS), and the proposed ARTSS approach. The loosely coupled solutions from the Inertial Explorer (IE) software serve as the reference truth. Experimental results demonstrate that the proposed algorithm achieves significant improvements in positioning and attitude accuracy during GNSS signal outages. Specifically, compared with the conventional ESKF and RTSS methods, the 3D position accuracy of the shipborne experiment is improved by 31.07% and 6.97%, respectively, while that of the land vehicle experiment is increased by 48.05% and 8.67%. Therefore, the method presented in this paper effectively mitigates the accumulation of forward filtering errors and significantly enhances the accuracy, stability, and reliability of the integrated navigation system in complex environments.