A Hybrid ResNet U-Net++ Architecture with ASPP and SE for Fish Histological Image Segmentation

The histological segmentation of fish gill lesions is a crucial step in environmental biomarker analysis, as morphological alterations in bioindicator species, such as Sciades herzbergii, provide biologically meaningful evidence of exposure to aquatic contaminants. In this context, gill histology enables the assessment of biomarkers; however, manual lesion quantification remains time-consuming, observer-dependent, and challenging to scale for environmental monitoring programs. Moreover, this task remains challenging due to the presence of heterogeneous textures, fragmented lesion boundaries, low-contrast regions, and staining variability. To address these issues, this study proposes a deep learning framework for the semantic segmentation of epithelial lifting (EL) and hyperplasia (HY) in gill histological images. The proposed model combines a ResNet-50 encoder, an ASPP bottleneck for multiscale contextual aggregation, squeeze-and-excitation-based channel recalibration at the bridge, and a nested U-Net++ decoder with deep supervision. The GillHistDB dataset was also developed for this study, comprising 447 RGB histological images and 29,730 annotated lesions, including 16,855 EL and 12,875 HY instances. The proposed method achieved the best overall performance among the evaluated models in the main overlap-based metrics. At the class level, it obtained Dice values of (0.842 ± 0.055) for EL and (0.684 ± 0.190) for HY, with corresponding IoU values of (0.731 ± 0.080) and (0.548 ± 0.196), respectively. For EL, the method also achieved the highest recall (0.848 ± 0.074), while for HY it reached the highest precision (0.653 ± 0.205) and maintained a high recall (0.767 ± 0.139). These results indicate that the proposed architecture provides an effective and robust solution for gill histological lesion segmentation, while GillHistDB establishes a relevant benchmark to support future studies on environmental biomonitoring, histological biomarkers, and the assessment of aquatic pollution.