A Modular Diabetic Retinopathy Analysis Framework for Lesion Segmentation and Severity Grading: Cross-Dataset External Validation from APTOS to Messidor-2

Background/Objectives: High in-domain accuracy of diabetic retinopathy (DR) grading models does not guarantee a similar performance for a new population. We quantify how an APTOS-trained grader degrades on an independent cohort and examine why. Methods: A modular framework combines a ResNet-34 U-Net for lesion segmentation (IDRiD) with an EfficientNet-B0 severity grader (APTOS 2019), optionally fused with lesion-derived and frequency features. The grader was evaluated zero-shot on Messidor-2 (n=1744) with bootstrap confidence intervals, four training seeds, referable-DR analysis, a discrimination-versus-calibration decomposition, a paired preprocessing ablation, and a duplicate audit. Results: Quadratic weighted kappa (QWK) fell from 0.873 (95% CI 0.831–0.909) on the APTOS test set to 0.455 (0.400–0.507) on Messidor-2; across four seeds the external QWK was 0.410±0.057. Referable-DR sensitivity fell from 92.7% to 35.0% (specificity 93.9% and 96.6%) and the AUC from 0.984 to 0.795, indicating both a loss of discrimination and a calibration shift; recalibrating thresholds with target labels (cross-validated) raised QWK only to 0.583. Omitting CLAHE reduced QWK to 0.326 (paired difference 0.129, 0.089–0.171). Thirty-three APTOS test images duplicated in the training or validation data changed QWK marginally (0.866). Lesion-feature fusion did not significantly improve grading. The segmentation Dice on lesion-containing patches was 0.51–0.68, whereas the per-image Dice was 0.21–0.61. Conclusions: In-domain agreement is a poor proxy for external performance; multi-seed reporting, explicit calibration analysis, and preprocessing control are needed before clinical use.