Agreement and repeatability of general-purpose large language model recommendations for pediatric IOL power and target refraction
Lifang Bai, Xuanqiao Lin, Xingyao Wang, Xiaohuan Zhao, Yizhou Yang, Aijun Wang, Jin YangPurpose
To evaluate the agreement, repeatability, and decision patterns of general-purpose large language models (LLMs) for intraocular lens (IOL) planning in children with congenital cataract.
Methods
This retrospective comparative evaluation included 51 eyes from 37 children who underwent primary IOL implantation. ChatGPT 5.6 Sol, Gemini 3.0 Pro, DeepSeek V4, and Doubao Expert Mode were evaluated using standardized preoperative case information. The first independent response was used for primary performance analyses, and three independent runs were used to assess repeatability. Model-generated target refraction and recommended IOL power were compared with the documented institutional target refraction and implanted IOL power.
Results
Target-refraction and IOL-power recommendations differed significantly among models (both P<0.001). Mean absolute target error ranged from 0.63 to 0.93 D, with 25.5%–54.9% of recommendations within ±0.50 D of the institutional target. Mean absolute IOL-power error ranged from 2.53 to 4.92 D; 5.9%–31.4% of recommendations were within ±1.00 D of the implanted power. Exact repeatability ranged from 45.1%–98.0% for target refraction and 0%–33.3% for IOL power. Across all models, target refraction became less hyperopic with increasing age, and recommended IOL power decreased with increasing axial length.
Conclusions
General-purpose LLMs captured broad pediatric refractive-planning patterns but showed model-specific differences, variable agreement with institutional clinical planning, and lower stability in final IOL-power recommendations. In this unconstrained setting, current general-purpose LLMs are not sufficiently reliable for independent pediatric IOL planning and may be better positioned as adjunctive tools alongside validated calculation methods and specialist judgment.