Difficulty Weighting and Semantic Centroid Regularization (SCR) for Raising the Performance Floor and Concentrating the Score Distribution of Small Models

Intelligent applications in the coal industry require models to read professional images such as safety tables and engineering drawings. Small multimodal models lack sufficient domain-specific knowledge, whereas large multimodal models are expensive to deploy. Black-box distillation with a commercial API as the teacher provides a feasible way to transfer the image–text understanding capability of large models to a locally deployable small model. However, conventional distillation usually optimizes only the average performance, which hardly meets the requirements of high-risk coal-mining scenarios in terms of the performance floor and the concentration of the score distribution. This paper, therefore, proposes two improvements that act only during training. The first is difficulty-weighted distillation (Diff), which uses the cross-entropy loss of the base student on the teacher-generated answer as a computable proxy for sample difficulty and oversamples hard samples so that the limited gradient budget is preferentially assigned to weak samples. The second is semantic centroid regularization (SCR), which aligns the last-token representations of the student toward the semantic centroids of their categories. The two components are combined to form the Diff-SCR training framework. The weights and the centroids are computed once and fixed before training, so the model structure is unchanged and no inference overhead is added. We distill Qwen3.5-0.8B using 1800 industry-specific images. The 223 test images are then independently evaluated by three human raters, and the mean of their scores for each image is used as the final score. Compared with ordinary distillation, Diff-SCR raises the average score from 73.37 to 80.82 and the average score among the lowest-scoring 25% of samples from 29.7 to 42.2, and reduces the share of complete failures from 2.2% to 0.4%. A 2×2 ablation shows that the two improvements are complementary: SCR alone already pushes the share of complete failures down to 0.9% and the average score among the lowest-scoring 5% of samples up from 7.8 to 9.1. The results show that, without increasing the inference cost, reallocating the training gradient resources improves the average performance while markedly raising the performance floor and compressing the dispersion of the score distribution.