Per-Lesion Mask-Supervised Attention for Multi-Label Diabetic Retinopathy Lesion Detection

Background/Objectives: Deep learning models for diabetic retinopathy (DR) usually report a severity grade, and their attention maps explain that grade rather than individual lesions. We aimed to detect microaneurysms, hemorrhages, hard exudates and soft exudates (cotton-wool spots) separately, each with an attention map trained to fall on that lesion. Methods: An EfficientNet-B4 and Transformer encoder, pretrained on severity grades, feeds four lesion queries that each return a score and an attention map; a localization loss trains each map toward its lesion mask. We trained it on DDR and TJDR, tested it on their test sets and on two external sets (IDRiD and e-ophtha), and compared it with a U-Net. Results: The mean areas under the curve (AUCs) were 0.93–0.94 for hemorrhages, 0.90–0.96 for hard exudates and 0.81–0.90 for soft exudates; microaneurysms reached 0.91–0.96, except on DDR (0.58). We analyze this drop in detail. Attention ranked lesion pixels above the rest of the retina with an AUC-Judd of 0.948–0.996. Without the localization loss, detection was slightly lower, and attention moved to regions outside the lesions that still raised the prediction. The model detected lesions better than U-Net (mean AUC of 0.876 against 0.853) and, unlike the latter, also learned from images that have labels but no masks: with masks on half of the training images, detection and AUC-Judd were nearly unchanged. Conclusions: Supervising each lesion’s attention with its own mask gives a per-lesion score and a map that falls on that lesion, without reducing detection.