Diagnostic Performance of Multimodal Artificial Intelligence in Differentiating Ocular Surface Squamous Neoplasia from Benign Lesions: A Retrospective Diagnostic Accuracy Study

Background/Objectives: Ocular surface squamous neoplasia (OSSN), comprising conjunctival intraepithelial neoplasia (CIN) and invasive squamous cell carcinoma (SCC), can be difficult to distinguish from benign ocular surface lesions. We evaluated the diagnostic performance of a multimodal vision-language model in differentiating OSSN from selected benign lesions using standardized clinical and imaging inputs. Methods: In this retrospective diagnostic accuracy study, 90 histopathologically confirmed lesions (20 CIN, 20 invasive SCC, 20 pterygia, 20 pseudopterygia, and 10 conjunctival papillomas) from a single tertiary center were evaluated with ChatGPT-4o (OpenAI) under three predefined conditions: expert-structured text, image-only, and combined multimodal input. The model was required to select a single diagnosis from predefined categories. Histopathology was the reference standard; agreement with expert clinical consensus was assessed separately. Scenario authors and consensus raters were blinded to histopathology. Results: In the multimodal condition, diagnostic accuracy was 78.9% (71/90; 95% CI, 69.4–86.0%), sensitivity 82.5% (95% CI, 68.1–91.3%), specificity 76.0% (95% CI, 62.6–85.7%), positive predictive value 73.3%, negative predictive value 84.4%, positive likelihood ratio 3.44 and negative likelihood ratio 0.23; false-negative and false-positive rates were 17.5% and 24.0%. Agreement with expert consensus was moderate (κ = 0.58). Accuracy was 60.0% (54/90; 95% CI, 49.7–69.5%) with expert-structured text alone and 70.0% (63/90; 95% CI, 59.9–78.5%) with images alone; because case-level paired predictions were not retained, between-condition differences were assessed descriptively only. Conclusions: In this single-center, diagnostically enriched cohort, a multimodal vision-language model showed measurable but imperfect diagnostic performance under standardized experimental conditions. These exploratory findings do not support autonomous clinical use; prospective external validation across broader lesion spectra and real-world clinical pathways is required.