A virtual chromoendoscopy artificial intelligence system to detect endoscopic and histologic activity/remission and predict clinical outcomes in ulcerative colitis

Abstract

Background Endoscopic and histological remission (ER, HR) are therapeutic targets in ulcerative colitis (UC). Virtual chromoendoscopy (VCE) improves endoscopic assessment and the prediction of histology; however, interobserver variability limits standardized endoscopic assessment. We aimed to develop an artificial intelligence (AI) tool to distinguish ER/activity, and predict histology and risk of flare from white-light endoscopy (WLE) and VCE videos.

Methods 1090 endoscopic videos (67 280 frames) from 283 patients were used to develop a convolutional neural network (CNN). UC endoscopic activity was graded by experts using the Ulcerative Colitis Endoscopic Index of Severity (UCEIS) and Paddington International virtual ChromoendoScopy ScOre (PICaSSO). The CNN was trained to distinguish ER/activity on endoscopy videos, and retrained to predict HR/activity, defined according to multiple indices, and predict outcome; CNN and human agreement was measured.

Results The AI system detected ER (UCEIS ≤ 1) in WLE videos with 72 % sensitivity, 87 % specificity, and an area under the receiver operating characteristic curve (AUROC) of 0.85; for detection of ER in VCE videos (PICaSSO ≤ 3), the sensitivity was 79 %, specificity 95 %, and the AUROC 0.94. The prediction of HR was similar between WLE and VCE videos (accuracies ranging from 80 % to 85 %). The model’s stratification of risk of flare was similar to that of physician-assessed endoscopy scores.

Conclusions Our system accurately distinguished ER/activity and predicted HR and clinical outcome from colonoscopy videos. This is the first computer model developed to detect inflammation/healing on VCE using the PICaSSO and the first computer tool to provide endoscopic, histologic, and clinical assessment.

^* Contributed equally to the manuscript.

Publication History

Received: 25 May 2022

Accepted after revision: 24 August 2022

Accepted Manuscript online:
13 October 2022

Article published online:
08 December 2022

• References

• 1 Ungaro R, Mehandru S, Allen PB. et al. Ulcerative colitis. Lancet 2017; 389: 1756-1770
• 2 Turner D, Ricciuto A, Lewis A. et al. STRIDE-II: an update on the selecting therapeutic targets in inflammatory bowel disease (STRIDE) initiative of the International Organization for the Study of IBD (IOIBD): determining therapeutic goals for treat-to-target strategies in IBD. Gastroenterology 2021; 160: 1570-1583
• 3 Yoon H, Jangi S, Dulai PS. et al. Incremental benefit of achieving endoscopic and histologic remission in patients with ulcerative colitis: a systematic review and meta-analysis. Gastroenterology 2020; 159: 1262-1275.e7
• 4 Schroeder KW, Tremaine WJ, Ilstrup DM. Coated oral 5-aminosalicylic acid therapy for mildly to moderately active ulcerative colitis. A randomized study. NEJM 1987; 317: 1625-1629
• 5 Travis SPL, Schnell D, Krzeski P. et al. Developing an instrument to assess the endoscopic severity of ulcerative colitis: the Ulcerative Colitis Endoscopic Index of Severity (UCEIS). Gut 2012; 61: 535-542
• 6 Bryant RV, Burger DC, Delo J. et al. Beyond endoscopic mucosal healing in UC: histological remission better predicts corticosteroid use and hospitalisation over 6 years of follow-up. Gut 2016; 65: 408-414
• 7 Iacucci M, Daperno M, Lazarev M. et al. Development and reliability of the new endoscopic virtual chromoendoscopy score: the PICaSSO (Paddington International Virtual ChromoendoScopy ScOre) in ulcerative colitis. Gastrointest Endosc 2017; 86: 1118-1127.e5
• 8 Trivedi PJ, Kiesslich R, Hodson J. et al. The Paddington International Virtual Chromoendoscopy Score in ulcerative colitis exhibits very good inter-rater agreement after computerized module training: a multicenter study across academic and community practice (with video). Gastrointest Endosc 2018; 88: 95-106.e2
• 9 Iacucci M, Smith SCL, Bazarova A. et al. An international multicenter real-life prospective study of electronic chromoendoscopy score PICaSSO in ulcerative colitis. Gastroenterology 2021; 160: 1558-1569.e8
• 10 Nardone OM, Cannatelli R, Zardo D. et al. Can advanced endoscopic techniques for assessment of mucosal inflammation and healing approximate histology in inflammatory bowel disease?. Therap Adv Gastroenterol 2019; 12: 1756284819863015
• 11 Fernandes SR, Pinto JSLD, Marques da Costa P. et al. Disagreement among gastroenterologists using the Mayo and Rutgeerts Endoscopic Scores. Inflamm Bowel Dis 2018; 24: 254-260
• 12 Gottlieb K, Requa J, Karnes W. et al. Central reading of ulcerative colitis clinical trial videos using neural networks. Gastroenterology 2021; 160: 710-719.e2
• 13 Gottlieb K, Daperno M, Usiskin K. et al. Endoscopy and central reading in inflammatory bowel disease clinical trials: achievements, challenges and future developments. Gut 2021; 70: 418-426
• 14 Takenaka K, Ohtsuka K, Fujii T. et al. Development and validation of a deep neural network for accurate evaluation of endoscopic images from patients with ulcerative colitis. Gastroenterology 2020; 158: 2150-2157
• 15 Takenaka K, Fujii T, Kawamoto A. et al. Deep neural network for video colonoscopy of ulcerative colitis: a cross-sectional study. Lancet Gastroenterol Hepatol 2022; 7: 230-237
• 16 Takenaka K, Ohtsuka K, Fujii T. et al. Deep neural network accurately predicts prognosis of ulcerative colitis using endoscopic images. Gastroenterology 2021; 160: 2175-2177.e3
• 17 Mosli MH, Feagan BG, Zou G. et al. Development and validation of a histological index for UC. Gut 2017; 66: 50-58
• 18 Marchal-Bressenot A, Salleron J, Boulagnon-Rombi C. et al. Development and validation of the Nancy histological index for UC. Gut 2017; 66: 43-49
• 19 Gui X, Bazarova A, Del Amor R. et al. PICaSSO Histologic Remission Index (PHRI) in ulcerative colitis: development of a novel simplified histological score for monitoring mucosal healing and predicting clinical outcomes and its applicability in an artificial intelligence system. Gut 2022; 71: 889-898
• 20 Clopper CJ, Pearson ES. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 1934; 26: 404-413
• 21 Mercaldo ND, Lau KF, Zhou XH. Confidence intervals for predictive values with an emphasis to case-control studies. Stat Med 2007; 26: 2170-2183
• 22 DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44: 837-845
• 23 Nardone OM, Bazarova A, Bhandari P. et al. PICaSSO virtual electronic chromendoscopy accurately reflects combined endoscopic and histological assessment for prediction of clinical outcomes in ulcerative colitis. United European Gastroenterol J 2022; 10: 147-159
• 24 Cannatelli R, Bazarova A, Furfaro F. et al. Reproducibility of the electronic chromoendoscopy PICaSSO score (Paddington International Virtual ChromoendoScopy ScOre) in ulcerative colitis using multiple endoscopic platforms: A prospective multicenter international study. Gastrointest Endosc 2022; 96: 73-83
• 25 Ozawa T, Ishihara S, Fujishiro M. et al. Novel computer-assisted diagnosis system for endoscopic disease activity in patients with ulcerative colitis. Gastrointest Endosc 2019; 89: 416-421.e1
• 26 Colombel J-F, Ordás I, Ullman T. et al. Agreement between rectosigmoidoscopy and colonoscopy analyses of disease activity and healing in patients with ulcerative colitis. Gastroenterology 2016; 150: 389-395.e3