Artificial Intelligence(AI) In Endoscopy-Deep Learning for Scoring of Ulcerative Colitis Disease Activity Under Multiple Scoring Systems

Aims Computer vision & deep learning(DL)to assess & help with tissue characterization of disease activity in Ulcerative Colitis(UC) through Mayo Endoscopic Subscore(MES)show good results in central reading for clinical trials.UCEIS(UC Endoscopic Index of Severity) may be more reflective of disease activity and more primed for artificial intelligence(AI). We set out to create a UC detection&scoring support tool for physicians,improving precision & reducing times between video collection & completed score.

Methods We leveraged >85,000 frames from endoscopy cases using Olympus(190&180series)scopes at 2 sites. Experienced endoscopists and 9 labellers reviewed~6000(7 %)images showing normal, disease state(Mayo,0-3 or UCEIS subscores) & non-scorable(blurry or presenting water,blood or stool) frames. Divided the total frames in 3:training (60 %/51,000), testing(20 %/17,000) & validation(18 %/17,000). Using Convolutional Neural Network(CNN)Inception V3 for detection at the frame level,with multiple separate units and dense layers that take features detected by the CNN as inputs & provide continuous scores for 5 separate outputs representing MES, the aggregate UCEIS & its individual components Vascular Pattern, Bleeding & Ulcers(fig.1). This enables the model to have parameters common for each score&ones that are specific to each unit.

Results We used Mean Absolute Error(MAE)and mean Bias, showing how far from truth the model is for each frame&whether the model tends to under or over predict the score. Our model performs as predicted, distributions are relatively close to the labelled(ground truth)ones & MAE&Bias for all frames are relatively low considering the magnitude of the scoring scale (table;fig2).

Conclusions We propose DL approach based on labelled images to predict MES and UCEIS scores. Although the investigation was carried on a limited dataset, it has shown relevant identification for the scoring of disease activity in UC patients, well-aligned with scoring guidelines & experts’ performance,demonstrating strong promise for generalization.By creating a regression output score,we can create a more precise level AI to score disease activity.We present work that builds a system for scoring disease activity at both the frame&video level, under both scoring modalities & that can accommodate other scoring systems like PiCaSSO.

Citation: Byrne MF, East JE, Iacucci M et al. OP139 ARTIFICIAL INTELLIGENCE(AI) IN ENDOSCOPY-DEEP LEARNING FOR SCORING OF ULCERATIVE COLITIS DISEASE ACTIVITY UNDER MULTIPLE SCORING SYSTEMS. Endoscopy 2021; 53: S57.

Publication History

Article published online:
19 March 2021