Heterogeneous SNN-ANN Multimodal Fusion Framework for Comprehensive Fruit Quality Assessment

Reliable fruit quality assessment is crucial for ensuring food safety and value in modern agriculture. However, many current approaches still rely heavily on visual cues, making it difficult to assess internal quality indicators such as sweetness or internal decay. To address this limitation, we propose HSAF-Net, a heterogeneous multimodal fusion framework integrating spiking neural networks (SNNs) and artificial neural networks (ANNs) for comprehensive, non-destructive fruit quality assessment. Specifically, the SNN encodes near-infrared (NIR) spectral signals to extract internal sugar-related features, whereas the ANN-based TH-YOLOv8 model detects external surface defects from high-resolution RGB images. A microsecond-level synchronous acquisition scheme is implemented to ensure precise alignment between the NIR and RGB modalities. To effectively combine heterogeneous features, we design a Heterogeneous Modality Attention (HMA) mechanism that dynamically fuses multi-source information based on task-specific relevance. Compared with image-only detection, the proposed framework explicitly separates internal biochemical sensing from external defect localization and then integrates their complementary decisions in a unified grading pipeline. Experimental results on 616 pear samples demonstrate that the HSAF-Net achieves 95.2% classification accuracy, 95.1% mAP95, and an internal defect miss rate as low as 7.5%, outperforming conventional single-modality and early-fusion baselines by a notable margin. The system maintains a real-time inference speed of 55 ms per sample on the Ascend Atlas 200DK A2 edge platform, validating its deployment potential. The current evaluation is based on crisp pear samples collected under controlled acquisition conditions; therefore, broader cross-variety and cross-season validation remains necessary before large-scale commercial deployment. This study presents an end-to-end multimodal SNN-ANN fusion architecture tailored for fruit grading and provides a scalable, high-precision solution for post-harvest quality assessment with broad applicability to other agricultural products.