Download PDF
CC BY-NC-ND 4.0 · Endosc Int Open 2021; 09(09): E1361-E1370
DOI: 10.1055/a-1507-4980
Original article

A deep learning framework for autonomous detection and classification of Crohnʼs disease lesions in the small bowel and colon with capsule endoscopy

Tomáš Majtner
1   Applied Artificial Intelligence and Data Science, Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark
,
Jacob Broder Brodersen
2   Department of Internal Medicine, Section of Gastroenterology, Hospital of South West Jutland, Esbjerg, Denmark
,
Jürgen Herp
1   Applied Artificial Intelligence and Data Science, Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark
,
Jens Kjeldsen
3   Department of Medical Gastroenterology, Odense University Hospital, Odense, Denmark
,
Morten Lee Halling
2   Department of Internal Medicine, Section of Gastroenterology, Hospital of South West Jutland, Esbjerg, Denmark
,
Michael Dam Jensen
4   Department of Internal Medicine, Section of Gastroenterology, Lillebaelt Hospital, Vejle, Denmark
› Author Affiliations
› Further Information
Also available at
    Permissions and Reprints

Abstract

Background and study aims Small bowel ulcerations are efficiently detected with deep learning techniques, whereas the ability to diagnose Crohnʼs disease (CD) in the colon with it is unknown. This study examined the ability of a deep learning framework to detect CD lesions with pan-enteric capsule endoscopy (CE) and classify lesions of different severity.

Patients and methods CEs from patients with suspected or known CD were included in the analysis. Two experienced gastroenterologists classified anonymized images into normal mucosa, non-ulcerated inflammation, aphthous ulceration, ulcer, or fissure/extensive ulceration. An automated framework incorporating multiple ResNet-50 architectures was trained. To improve its robustness and ability to characterize lesions, image processing methods focused on texture enhancement were employed.

Results A total of 7744 images from 38 patients with CD were collected (small bowel 4972, colon 2772) of which 2748 contained at least one ulceration (small bowel 1857, colon 891). With a patient-dependent split of images for training, validation, and testing, ulcerations were diagnosed with a sensitivity, specificity, and diagnostic accuracy of 95.7 % (CI 93.4–97.4), 99.8 % (CI 99.2–100), and 98.4 % (CI 97.6–99.0), respectively. The diagnostic accuracy was 98.5 % (CI 97.5–99.2) for the small bowel and 98.1 % (CI 96.3–99.2) for the colon. Ulcerations of different severities were classified with substantial agreement (κ = 0.72).

Conclusions Our proposed framework is in excellent agreement with the clinical standard, and diagnostic accuracies are equally high for the small bowel and colon. Deep learning approaches have a great potential to help clinicians detect, localize, and determine the severity of CD with pan-enteric CE.



Publication History

Received: 20 January 2021

Accepted: 03 May 2021

Article published online:
16 August 2021

© 2021. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

Comment to this article:

Endoscopy International Open – recently publishedEndoscopy 2021; 53(10): 1093-1093
DOI: 10.1055/a-1526-9213

 

  • References

  • 1 Torres J, Mehandru S, Colombel JF. et al. Crohn's disease. Lancet 2017; 389: 1741-1755
    CrossrefPubMedGoogle Scholar
  • 2 Burisch J, Kiudelis G, Kupcinskas L. et al. Natural disease course of Crohn's disease during the first 5 years after diagnosis in a European population-based inception cohort: an Epi-IBD study. Gut 2019; 68: 423-433
    CrossrefPubMedGoogle Scholar
  • 3 Koulaouzidis A, Iakovidis DK, Karargyris A. et al. Optimizing lesion detection in small-bowel capsule endoscopy: from present problems to future solutions. Expert Rev Gastroenterol Hepatol 2015; 9: 217-235
    CrossrefPubMedGoogle Scholar
  • 4 Jensen MD, Nathan T, Rafaelsen SR. et al. Diagnostic accuracy of capsule endoscopy for small bowel Crohn's disease is superior to that of MR enterography or CT enterography. Clin Gastroenterol Hepatol 2011; 9: 124-129
    CrossrefPubMedGoogle Scholar
  • 5 Pennazio M, Spada C, Eliakim R. et al. Small-bowel capsule endoscopy and device-assisted enteroscopy for diagnosis and treatment of small-bowel disorders: European Society of Gastrointestinal Endoscopy (ESGE) Clinical Guideline. Endoscopy 2015; 47: 352-376
    Article in Thieme ConnectPubMedGoogle Scholar
  • 6 Maaser C, Sturm A, Vavricka SR. et al. ECCO-ESGAR Guideline for Diagnostic Assessment in IBD Part 1: Initial diagnosis, monitoring of known IBD, detection of complications. J Crohns Colitis 2019; 13: 144-164
    CrossrefPubMedGoogle Scholar
  • 7 Ahmad OF, Soares AS, Mazomenos E. et al. Artificial intelligence and computer-aided diagnosis in colonoscopy: current evidence and future directions. Lancet Gastroenterol Hepatol 2019; 4: 71-80
    CrossrefPubMedGoogle Scholar
  • 8 Soffer S, Klang E, Shimon O. et al. Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis. Gastrointest Endosc 2020; 92: 831-839 e838
    CrossrefPubMedGoogle Scholar
  • 9 Nadimi ES, Buijs MM, Herp J. et al. Application of deep learning for autonomous detection and localization of colorectal polyps in wireless colon capsule endoscopy. Comput Elect Eng 2020; 81.106531
    PubMedGoogle Scholar
  • 10 Gionchetti P, Dignass A, Danese S. et al. 3rd European Evidence-based Consensus on the Diagnosis and Management of Crohn's Disease 2016: Part 2: Surgical Management and Special Situations. J Crohns Colitis 2017; 11: 135-149
    CrossrefPubMedGoogle Scholar
  • 11 Spada C, Hassan C, Galmiche JP. et al. Colon capsule endoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Guideline. Endoscopy 2012; 44: 527-536
    Article in Thieme ConnectPubMedGoogle Scholar
  • 12 Daněk O, Matula P, Maška M. et al. Smooth Chan–Vese segmentation via graph cuts. Pattern Recognition Letters 2012; 33: 1405-1410
    CrossrefPubMedGoogle Scholar
  • 13 Zuiderveld K. Contrast Limited Adaptive Histogram Equalization. Graphics Gems IV Academic Press Professional 1994; 474-485
    CrossrefPubMedGoogle Scholar
  • 14 Yosinski J, Clune J, Bengio Y. et al. How transferable are features in deep neural networks?. Adv Neural Informat Proc Syst 2014; 3320-3328
    PubMedGoogle Scholar
  • 15 He K, Zhang X, Ren S. et al. Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition 2016. 770-778
    PubMedGoogle Scholar
  • 16 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-174
    CrossrefPubMedGoogle Scholar
  • 17 Annese V, Daperno M, Rutter MD. et al. European evidence based consensus for endoscopy in inflammatory bowel disease. J Crohns Colitis 2013; 7: 982-1018
    CrossrefPubMedGoogle Scholar
  • 18 Hansel SL, McCurdy JD, Barlow JM. et al. Clinical Benefit of capsule endoscopy in crohn's disease: impact on patient management and prevalence of proximal small bowel involvement. Inflamm Bowel Dis 2018; 24: 1582-1588
    CrossrefPubMedGoogle Scholar
  • 19 Kopylov U, Nemeth A, Koulaouzidis A. et al. Small bowel capsule endoscopy in the management of established Crohn's disease: clinical impact, safety, and correlation with inflammatory biomarkers. Inflamm Bowel Dis 2015; 21: 93-100
    CrossrefPubMedGoogle Scholar
  • 20 Rondonotti E, Spada C, Adler S. et al. Small-bowel capsule endoscopy and device-assisted enteroscopy for diagnosis and treatment of small-bowel disorders: European Society of Gastrointestinal Endoscopy (ESGE) Technical Review. Endoscopy 2018; 50: 423-446
    Article in Thieme ConnectPubMedGoogle Scholar
  • 21 Jensen MD, Brodersen JB, Kjeldsen J. Capsule endoscopy for the diagnosis and follow up of Crohn's disease: a comprehensive review of current status. Ann Gastroenterol 2017; 30: 168-178
    PubMedGoogle Scholar
  • 22 Aoki T, Yamada A, Aoyama K. et al. Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 2019; 89: 357-363 e352 DOI: 10.1016/j.gie.2018.10.027.
    CrossrefPubMedGoogle Scholar
  • 23 Klang E, Barash Y, Margalit RY. et al. Deep learning algorithms for automated detection of Crohn's disease ulcers by video capsule endoscopy. Gastrointest Endosc 2020; 91: 606-613 e602
    CrossrefPubMedGoogle Scholar
  • 24 Ding Z, Shi H, Zhang H. et al. Gastroenterologist-level identification of small-bowel diseases and normal variants by capsule endoscopy using a deep-learning model. Gastroenterology 2019; 157: 1044-1054 e1045
    CrossrefPubMedGoogle Scholar
  • 25 Barash Y, Azaria L, Soffer S. et al. Ulcer severity grading in video capsule images of patients with Crohn's disease: an ordinal neural network solution. Gastrointest Endosc 2021; 93: 187-192
    CrossrefPubMedGoogle Scholar