Multi‐Model Fruit Quality Assessment With Coarse YOLOv11 Presence Filtering, Vision Transformers and the Segment Anything Model

ABSTRACT

Manual inspection of fruit quality is slow and subjective, and it does not scale to high‐throughput sorting. This paper presents a multi‐model deep learning pipeline for coarse fruit‐presence filtering, ripeness classification and pixel‐level spoilage segmentation. YOLOv11n is used only as an image‐level presence filter and is not claimed to localise fruit; four classifiers (EfficientNetV2‐S, ConvNeXt‐Base, DeiT‐Small and Swin‐V2‐Tiny) predict one of nine ripeness classes across three fruit types; Grad‐CAM heatmaps expose what each classifier attends to; and the Segment Anything Model (SAM ViT‐H) generates fine‐grained masks evaluated against manually annotated spoilage regions using ground‐truth‐derived box prompts. The final classification experiment uses a fixed stratified partition of 13,964 training, 2990 validation and 3002 test images. Each classifier is trained for 20 epochs with each of four seeds (42–45). Mean test accuracy sample standard deviation is for EfficientNetV2‐S, for ConvNeXt‐Base, for DeiT‐Small and for Swin‐V2‐Tiny. Exact two‐sided McNemar tests on paired seed‐42 predictions, with Holm correction across six comparisons, find no significant differences at (minimum adjusted ). The mean‐accuracy ranking is descriptive, not evidence of statistical superiority. Uncertainty describes training variation on a fixed split. A separate inference‐only SAM evaluation on 127 manually annotated rotten‐fruit images yields IoU and Dice (mean  SD across images). These results quantify pixel‐wise segmentation agreement conditional on correct spatial localisation, not end‐to‐end automatic spoilage detection. The pipeline runs in real time on controlled, single‐fruit webcam inputs; spatial object‐detection performance is neither claimed nor evaluated because tight ground‐truth bounding boxes are unavailable.