Near-Duplicate Contamination in a Widely Used Safety Helmet Detection Benchmark: Audit, Controlled Experiments, and a Duplicate-Aware Evaluation Split
Liyuan Chen, Zhe Wang, Zhixiong Li, Wei Li, Hao Zhu, Daguang HanPublic benchmarks drive research on vision-based safety monitoring, and the Safety Helmet Wearing Dataset (SHWD) has become the de facto standard for helmet-wearing detection. This paper reports a systematic near-duplicate audit of SHWD and measures how the contamination it uncovers affects the accuracy reported on the benchmark. A two-stage matching procedure—64-bit perceptual hashing with complementary thumbnail-correlation verification—shows that 2288 of 7581 images (30.2%) belong to 193 near-duplicate clusters, the largest containing 802 frames from one fixed camera, and that 421 of the 1517 official test images (27.8%; 22–29% across matching thresholds) have a near-duplicate counterpart in the training split, and a blind manual rating of 350 pairs finds that 100% of accepted pairs are frames of the same fixed camera (28% strict near-duplicates). Controlled experiments with YOLOv8n, three seeds each, quantify the consequence. With the test data fixed, removing the 1534 contaminating training images lowers mAP@0.5 on the contaminated test subset by 4.4 points, and a size-matched random control shows that the difference is not a data-volume effect; on the full official test set the inflation is 1.92 points relative to the deduplicated model and 1.35 points relative to the control. A cluster-disjoint experiment that holds the domain composition of the training set fixed shows that frames from the same cameras and sessions as the test frames add a further 0.6 points on those frames, so that most of the gap under the official split reflects the classroom imagery that the contaminating images supply rather than recognition of specific frames. Tracing images to their source shows that 97.1% of the test images imported from the SCUT-HEAD classroom-video dataset are contaminated, against 3.0–5.5% of the genuine helmet imagery, and that the effect is confined to the imported subset (+8.1 points there, −0.4 on helmet imagery). The inflation is reproduced with YOLOv8s (+2.1 points on the contaminated subset) and with RT-DETR-l (+2.8 points). Removing the contaminating training images—25.3% of the training set—leaves accuracy on the clean test subset unchanged within seed variability (−0.14 points). The duplicate lists, a duplicate-aware evaluation split, and the audit code are publicly available.