CC BY-NC-ND 4.0 · Endoscopy 2023; 55(04): 332-341
DOI: 10.1055/a-1960-3645
Original article

A virtual chromoendoscopy artificial intelligence system to detect endoscopic and histologic activity/remission and predict clinical outcomes in ulcerative colitis

Marietta Iacucci
1   Institute of Immunology and Immunotherapy, NIHR Wellcome Trust Clinical Research Facilities, University of Birmingham, and University Hospitals Birmingham NHS Trust, Birmingham, UK
2   National Institute for Health Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
3   Division of Gastroenterology and Hepatology, University of Calgary, Calgary, Canada
,
Rosanna Cannatelli*
1   Institute of Immunology and Immunotherapy, NIHR Wellcome Trust Clinical Research Facilities, University of Birmingham, and University Hospitals Birmingham NHS Trust, Birmingham, UK
4   Gastroenterology and Digestive Endoscopy Unit, Department of Biochemical and Clinical Sciences “L. Sacco”, University of Milan, ASST Fatebenefratelli Sacco, Milan, Italy
,
Tommaso L. Parigi*
1   Institute of Immunology and Immunotherapy, NIHR Wellcome Trust Clinical Research Facilities, University of Birmingham, and University Hospitals Birmingham NHS Trust, Birmingham, UK
 5   Department of Biomedical Science, Humanitas University, Milan, Italy
,
Olga M. Nardone
1   Institute of Immunology and Immunotherapy, NIHR Wellcome Trust Clinical Research Facilities, University of Birmingham, and University Hospitals Birmingham NHS Trust, Birmingham, UK
 6   Gastroenterology, department of Public health, university of Naples Federico II, Naples, Italy
,
Gian Eugenio Tontini
 7   Division of Gastroenterology, Fondazione IRCCS Ca’ Granda Ospedale Maggiore Policlinico, Milan, Italy
 8   Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
,
 9   National Institute of Gastroenterology, IRCCS S. De Bellis Research Hospital, Castellana Grotte, Italy
,
Andrea Buda
10   Department of Gastrointestinal Oncological Surgery, Santa Maria del Prato Hospital, Feltre, Italy
,
Alessandro Rimondi
 8   Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
,
Alina Bazarova
1   Institute of Immunology and Immunotherapy, NIHR Wellcome Trust Clinical Research Facilities, University of Birmingham, and University Hospitals Birmingham NHS Trust, Birmingham, UK
11   Institute for Biological Physics, University of Cologne, Cologne, Germany
,
12   Division of Gastroenterology, University Hospitals Leuven, Leuven, Belgium
,
Rocio del Amor
13   Instituto de Investigación e Innovación en Bioingeniería, Universitat Politècnica de València, València, Spain
,
Pablo Meseguer
13   Instituto de Investigación e Innovación en Bioingeniería, Universitat Politècnica de València, València, Spain
,
Valery Naranjo
13   Instituto de Investigación e Innovación en Bioingeniería, Universitat Politècnica de València, València, Spain
,
Subrata Ghosh
1   Institute of Immunology and Immunotherapy, NIHR Wellcome Trust Clinical Research Facilities, University of Birmingham, and University Hospitals Birmingham NHS Trust, Birmingham, UK
2   National Institute for Health Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
3   Division of Gastroenterology and Hepatology, University of Calgary, Calgary, Canada
14   APC Microbiome Ireland, College of Medicine and Health, Cork, Ireland
,
Enrico Grisan
15   School of Engineering Computer Science and Informatics, London South Bank University, London, UK,
16   Department of Engineering, University of Padova, Padova, Italy
,
on behalf of the PICaSSO group
› Author Affiliations


Abstract

Background Endoscopic and histological remission (ER, HR) are therapeutic targets in ulcerative colitis (UC). Virtual chromoendoscopy (VCE) improves endoscopic assessment and the prediction of histology; however, interobserver variability limits standardized endoscopic assessment. We aimed to develop an artificial intelligence (AI) tool to distinguish ER/activity, and predict histology and risk of flare from white-light endoscopy (WLE) and VCE videos.

Methods 1090 endoscopic videos (67 280 frames) from 283 patients were used to develop a convolutional neural network (CNN). UC endoscopic activity was graded by experts using the Ulcerative Colitis Endoscopic Index of Severity (UCEIS) and Paddington International virtual ChromoendoScopy ScOre (PICaSSO). The CNN was trained to distinguish ER/activity on endoscopy videos, and retrained to predict HR/activity, defined according to multiple indices, and predict outcome; CNN and human agreement was measured.

Results The AI system detected ER (UCEIS ≤ 1) in WLE videos with 72 % sensitivity, 87 % specificity, and an area under the receiver operating characteristic curve (AUROC) of 0.85; for detection of ER in VCE videos (PICaSSO ≤ 3), the sensitivity was 79 %, specificity 95 %, and the AUROC 0.94. The prediction of HR was similar between WLE and VCE videos (accuracies ranging from 80 % to 85 %). The model’s stratification of risk of flare was similar to that of physician-assessed endoscopy scores.

Conclusions Our system accurately distinguished ER/activity and predicted HR and clinical outcome from colonoscopy videos. This is the first computer model developed to detect inflammation/healing on VCE using the PICaSSO and the first computer tool to provide endoscopic, histologic, and clinical assessment.

* Contributed equally to the manuscript.


Supplementary material



Publication History

Received: 25 May 2022

Accepted after revision: 24 August 2022

Accepted Manuscript online:
13 October 2022

Article published online:
08 December 2022

© 2022. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 
  • References

  • 1 Ungaro R, Mehandru S, Allen PB. et al. Ulcerative colitis. Lancet 2017; 389: 1756-1770
  • 2 Turner D, Ricciuto A, Lewis A. et al. STRIDE-II: an update on the selecting therapeutic targets in inflammatory bowel disease (STRIDE) initiative of the International Organization for the Study of IBD (IOIBD): determining therapeutic goals for treat-to-target strategies in IBD. Gastroenterology 2021; 160: 1570-1583
  • 3 Yoon H, Jangi S, Dulai PS. et al. Incremental benefit of achieving endoscopic and histologic remission in patients with ulcerative colitis: a systematic review and meta-analysis. Gastroenterology 2020; 159: 1262-1275.e7
  • 4 Schroeder KW, Tremaine WJ, Ilstrup DM. Coated oral 5-aminosalicylic acid therapy for mildly to moderately active ulcerative colitis. A randomized study. NEJM 1987; 317: 1625-1629
  • 5 Travis SPL, Schnell D, Krzeski P. et al. Developing an instrument to assess the endoscopic severity of ulcerative colitis: the Ulcerative Colitis Endoscopic Index of Severity (UCEIS). Gut 2012; 61: 535-542
  • 6 Bryant RV, Burger DC, Delo J. et al. Beyond endoscopic mucosal healing in UC: histological remission better predicts corticosteroid use and hospitalisation over 6 years of follow-up. Gut 2016; 65: 408-414
  • 7 Iacucci M, Daperno M, Lazarev M. et al. Development and reliability of the new endoscopic virtual chromoendoscopy score: the PICaSSO (Paddington International Virtual ChromoendoScopy ScOre) in ulcerative colitis. Gastrointest Endosc 2017; 86: 1118-1127.e5
  • 8 Trivedi PJ, Kiesslich R, Hodson J. et al. The Paddington International Virtual Chromoendoscopy Score in ulcerative colitis exhibits very good inter-rater agreement after computerized module training: a multicenter study across academic and community practice (with video). Gastrointest Endosc 2018; 88: 95-106.e2
  • 9 Iacucci M, Smith SCL, Bazarova A. et al. An international multicenter real-life prospective study of electronic chromoendoscopy score PICaSSO in ulcerative colitis. Gastroenterology 2021; 160: 1558-1569.e8
  • 10 Nardone OM, Cannatelli R, Zardo D. et al. Can advanced endoscopic techniques for assessment of mucosal inflammation and healing approximate histology in inflammatory bowel disease?. Therap Adv Gastroenterol 2019; 12: 1756284819863015
  • 11 Fernandes SR, Pinto JSLD, Marques da Costa P. et al. Disagreement among gastroenterologists using the Mayo and Rutgeerts Endoscopic Scores. Inflamm Bowel Dis 2018; 24: 254-260
  • 12 Gottlieb K, Requa J, Karnes W. et al. Central reading of ulcerative colitis clinical trial videos using neural networks. Gastroenterology 2021; 160: 710-719.e2
  • 13 Gottlieb K, Daperno M, Usiskin K. et al. Endoscopy and central reading in inflammatory bowel disease clinical trials: achievements, challenges and future developments. Gut 2021; 70: 418-426
  • 14 Takenaka K, Ohtsuka K, Fujii T. et al. Development and validation of a deep neural network for accurate evaluation of endoscopic images from patients with ulcerative colitis. Gastroenterology 2020; 158: 2150-2157
  • 15 Takenaka K, Fujii T, Kawamoto A. et al. Deep neural network for video colonoscopy of ulcerative colitis: a cross-sectional study. Lancet Gastroenterol Hepatol 2022; 7: 230-237
  • 16 Takenaka K, Ohtsuka K, Fujii T. et al. Deep neural network accurately predicts prognosis of ulcerative colitis using endoscopic images. Gastroenterology 2021; 160: 2175-2177.e3
  • 17 Mosli MH, Feagan BG, Zou G. et al. Development and validation of a histological index for UC. Gut 2017; 66: 50-58
  • 18 Marchal-Bressenot A, Salleron J, Boulagnon-Rombi C. et al. Development and validation of the Nancy histological index for UC. Gut 2017; 66: 43-49
  • 19 Gui X, Bazarova A, Del Amor R. et al. PICaSSO Histologic Remission Index (PHRI) in ulcerative colitis: development of a novel simplified histological score for monitoring mucosal healing and predicting clinical outcomes and its applicability in an artificial intelligence system. Gut 2022; 71: 889-898
  • 20 Clopper CJ, Pearson ES. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 1934; 26: 404-413
  • 21 Mercaldo ND, Lau KF, Zhou XH. Confidence intervals for predictive values with an emphasis to case-control studies. Stat Med 2007; 26: 2170-2183
  • 22 DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44: 837-845
  • 23 Nardone OM, Bazarova A, Bhandari P. et al. PICaSSO virtual electronic chromendoscopy accurately reflects combined endoscopic and histological assessment for prediction of clinical outcomes in ulcerative colitis. United European Gastroenterol J 2022; 10: 147-159
  • 24 Cannatelli R, Bazarova A, Furfaro F. et al. Reproducibility of the electronic chromoendoscopy PICaSSO score (Paddington International Virtual ChromoendoScopy ScOre) in ulcerative colitis using multiple endoscopic platforms: A prospective multicenter international study. Gastrointest Endosc 2022; 96: 73-83
  • 25 Ozawa T, Ishihara S, Fujishiro M. et al. Novel computer-assisted diagnosis system for endoscopic disease activity in patients with ulcerative colitis. Gastrointest Endosc 2019; 89: 416-421.e1
  • 26 Colombel J-F, Ordás I, Ullman T. et al. Agreement between rectosigmoidoscopy and colonoscopy analyses of disease activity and healing in patients with ulcerative colitis. Gastroenterology 2016; 150: 389-395.e3