Task-Oriented Evaluation of Eight Cropland Maps for Agricultural Monitoring in China

Differences in cropland definitions, spatial resolution, and mapping strategies complicate product selection. We harmonized and evaluated eight 2020 cropland and land-cover products across mainland China using hierarchically labeled probability samples. Fixed map predictions enabled comparisons across common task-specific and product-definition-matched reference labels. Evaluation covered spatial agreement, design-weighted classification accuracy, design-based area bias, cropland-type responses, and cross-scale performance. Only 26.8% of pixels in the cropland union were classified as cropland by all products. Within the consensus-cropland evaluation domain, CropLayer achieved the highest overall accuracy (0.961) and F1 score (0.844). CCropLand30 led 30 m products in F1, whereas the Esri land-cover product (ESRI-LC) had national area bias closest to zero (−2.26%). Most 30 m products showed commission-dominated errors and positive area bias. Product responses varied across cropland types, and reference labels altered performance estimates. Switching from unified-agriculture to product-definition-matched labels reduced ESRI-LC’s F1 from 0.765 to 0.682 and shifted area bias from −8.08% to +14.88%. Spatial aggregation reduced F1 for some fine-resolution products without consistently changing local cropland-proportion error. These results show that classification accuracy and area bias yield different rankings, supporting product selection according to application, cropland definition, landscape context, evaluation metric, and spatial support.