Spectral Similarity-Based Grouping Improves Laboratory Vis-NIR Prediction of Soil Organic Matter Under Soil Heterogeneity

Laboratory visible–near-infrared (Vis-NIR) spectroscopy is an efficient and practical approach for measuring soil properties. However, its predictive accuracy is often limited by spectral variability resulting from soil heterogeneity. Such variability poses challenges for traditional global spectral modeling methods in capturing the highly complex mapping relationships between spectral features and soil attributes. To address this issue, this study introduces a novel spectral similarity-based partitioned modeling strategy and compares it with a soil type-based grouping approach. The proposed method clusters soil samples according to their spectral characteristics to minimize within-group variance. Subsequently, individual inversion models are developed for each cluster, enabling more precise learning of stable relationships between specific spectral patterns and soil properties, thereby improving both model accuracy and interpretability. To validate this methodology, the study systematically evaluates the performance of five traditional clustering algorithms (TCAs) and five deep clustering algorithms (DCAs) using the Chinese Soil Spectral Library (CSSL) and the national soil spectral library of China, supplemented with local laboratory Vis-NIR data from Xinjiang, targeting soil organic matter (SOM). Results indicate that both spectral similarity-based and soil type-based grouping strategies significantly surpass conventional global modeling methods. Among them, the spectral similarity-based partitioning approach yielded the most notable improvements. Compared with traditional global modeling, it increased R2 from 0.65–0.71 to 0.81–0.85 (relative increase: 14.08–30.77%) and reduced RMSE from 2.49–3.24 g kg−1 to 1.07–1.39 g kg−1 (relative reduction: 44.18–66.98%). Compared with the soil type-based grouping strategy, it further increased R2 from 0.70–0.74 to 0.81–0.85 (relative increase: 9.46–21.43%) and reduced RMSE from 1.91–2.56 g kg−1 to 1.07–1.39 g kg−1 (relative reduction: 27.23–58.20%). These outcomes suggest that conventional soil taxonomic information alone is insufficient to capture critical spectral variations. In the spectral grouping stage, DCAs were found to identify and aggregate soil samples with high spectral similarity more effectively than TCA. In particular, the inversion model constructed using Automatic Encoding Clustering (AEC) delivered the best performance. Additionally, the study revealed considerable spectral diversity, even among soils of the same type. In the evaluated dataset, spatial proximity was not consistently associated with spectral similarity. These findings confirm that spectral similarity-based grouping and modeling substantially improve the predictive accuracy of soil property estimation, offering a more refined and effective modeling pathway for hyperspectral soil detection. All spectral data analyzed here are laboratory-based Vis-NIR spectra from the national soil spectral library of China. These findings are based on the Chinese Soil Spectral Library and the Xinjiang dataset and should be validated in other regions and for other soil properties before broad application.