P0556 A novel switching of artificial intelligence to generate simultaneously multimodal images to assess inflammation and predict outcomes in Ulcerative Colitis
M Iacucci, I Zammarchi, G Santacroce, B B Kolawole, U Chaudhari, R Del Amor, P Meseguer, V Naranjo, M Puga-Tejada, I Capobianco, I Ditonno, A Buda, B Hayes, R Crotty, R Bisschops, S Ghosh, E Grisan,Abstract
Background
Virtual Chromoendoscopy (VCE) is pivotal for assessing activity and predicting outcomes in Ulcerative Colitis (UC), though inter- and intra-observer variability and the need for expertise persist. Artificial intelligence (AI) has the potential to offer standardised VCE-based assessment. This study introduces a novel AI model to detect, generate and transition between various endoscopic modalities, enhancing AI-driven inflammation assessment and outcome prediction in UC.
Methods
Endoscopic videos in high-definition white-light (HD-WLE), iScan2, iScan3 and NBI modalities from UC patients of the international PICaSSO iScan and Narrow-Band Imaging (NBI) cohort (302 and 54 patients, respectively) were used to develop a neural network (NN) able to identify the acquisition modality of each frame and for inter-modality image switching. 2535 frames were switched to different endoscopic modalities and used to train a deep-learning model for inflammation assessment using single and multimodal inputs on 169 videos of the iScan cohort. Subsequently, the model was tested on a subset of the iScan and NBI cohort (72 and 51 videos, 1080 and 765 frames, respectively). The model performance in predicting endoscopic and histological activity and outcomes and the agreement with experts were evaluated.
Results
Table 1 details the diagnostic performance of the AI model for the prediction of endoscopic and histological remission. The AI model efficiently classified and converted images across modalities (92% NN classifier accuracy). It showed excellent performance in predicting endoscopic and histological remission, with the multimodal assessment outperforming the unimodal one in both iScan cohorts (accuracy 91.67 [95% CI 82.74-96.88] and 88.89 [80.4-97.73]; AUROC 0.96 and 0.90 by Ulcerative Colitis Endoscopic Index of Severity (UCEIS) and Paddington International Virtual Chromoendoscopy (PICaSSO) score, respectively) and NBI cohort (accuracy 84.62 [95% CI 65.13-95.64] and 88.46 [69.85-97.55]; AUROC 0.92 by UCEIS and PICaSSO score, respectively). Moreover, it showed a remarkable ability to predict clinical outcomes in the iScan and NBI cohort (HR 3.18 [0.98-10.35] and 1.7 [0.7-4.11] by endoscopy; 5.75 [1.77-18-71] and 3.9 [1.15-13.28] by histology, respectively). Finally, the agreement with the assessment performed by endoscopists and pathologists was good.
Conclusion
Our multimodal "AI-switching" model innovatively detects, generates, and transitions between different endoscopic enhancement modalities and platforms, refining inflammation assessment, outcome prediction, and precise UC management by integrating model-derived images.