Download PDF
Endoscopy 2021; 53(05): 491-498
DOI: 10.1055/a-1244-5698
Original article

Intelligent difficulty scoring and assistance system for endoscopic extraction of common bile duct stones based on deep learning: multicenter study

Li Huang‡   *
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Xiaoyan Lu‡   *
4   State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan, China
,
Xu Huang
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Xiaoping Zou
5   Department of Gastroenterology, Nanjing Drum Tower Hospital of Nanjing University Medical School, Nanjing, China
,
Lianlian Wu
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Zhongyin Zhou
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Deqing Wu
6   Department of Gastroenterology, Tenth People Hospital of Tongji University, Shanghai, China
,
Dehua Tang
5   Department of Gastroenterology, Nanjing Drum Tower Hospital of Nanjing University Medical School, Nanjing, China
,
Dingyuan Chen
4   State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan, China
,
Xinyue Wan
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Zhongchao Zhu
7   Department of Pancreatic Surgery, Renmin Hospital of Wuhan University, Wuhan, China
,
Tao Deng
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Lei Shen
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Jun Liu
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Yijie Zhu
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Dexin Gong
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Di Chen
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
,
Yanfei Zhong
4   State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan, China
,
Feng Liu
6   Department of Gastroenterology, Tenth People Hospital of Tongji University, Shanghai, China
,
Honggang Yu
1   Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China
2   Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China
3   Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
› Author Affiliations
› Further Information
Also available at
    Permissions and Reprints

Referred to by:

Author commentary on Li Huang et al.Endoscopy 2021; 53(05): v21-v21
DOI: 10.1055/a-1289-5529

Referred to by:

May the force be with you: will artificial intelligence take over traditional endoscopy?Endoscopy 2021; 53(05): 499-500
DOI: 10.1055/a-1313-7499

Abstract

Background The study aimed to construct an intelligent difficulty scoring and assistance system (DSAS) for endoscopic retrograde cholangiopancreatography (ERCP) treatment of common bile duct (CBD) stones.

Methods 1954 cholangiograms were collected from three hospitals for training and testing the DSAS. The D-LinkNet34 and U-Net were adopted to segment the CBD, stones, and duodenoscope. Based on the segmentation results, the stone size, distal CBD diameter, distal CBD arm, and distal CBD angulation were estimated. The performance of segmentation and estimation was assessed by mean intersection over union (mIoU) and average relative error. A technical difficulty scoring scale, which was used for assessing the technical difficulty of CBD stone removal, was developed and validated. We also analyzed the relationship between scores evaluated by the DSAS and clinical indicators including stone clearance rate and need for endoscopic papillary large-balloon dilation (EPLBD) and lithotripsy.

Results The mIoU values of the stone, CBD, and duodenoscope segmentation were 68.35 %, 86.42 %, and 95.85 %, respectively. The estimation performance of the DSAS was superior to nonexpert endoscopists. In addition, the technical difficulty scoring performance of the DSAS was more consistent with expert endoscopists than two nonexpert endoscopists. A DSAS assessment score ≥ 2 was correlated with lower stone clearance rates and more frequent EPLBD.

Conclusions An intelligent DSAS based on deep learning was developed. The DSAS could assist endoscopists by automatically scoring the technical difficulty of CBD stone extraction, and guiding the choice of therapeutic approach and appropriate accessories during ERCP.

* These authors contributed equally to this work.


Supplementary material



Publication History

Received: 14 March 2020

Accepted: 21 August 2020

Accepted Manuscript online:
21 August 2020

Article published online:
09 November 2020

© 2020. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Tazuma S. Gallstone disease: epidemiology, pathogenesis, and classification of biliary stones (common bile duct and intrahepatic). Best Pract Res Clin Gastroenterol 2006; 20: 1075-1083
    CrossrefPubMedGoogle Scholar
  • 2 Galeazzi M, Mazzola P, Valcarcel B. et al. Endoscopic retrograde cholangiopancreatography in the elderly: results of a retrospective study and a geriatricians’ point of view. BMC Gastroenterol 2018; 18: 38
    CrossrefPubMedGoogle Scholar
  • 3 Evans N, Buxbaum JL. Endoscopic treatment of ERCP-related duodenal perforation. Tech Gastrointest Endosc 2019; 21: 83-90
    CrossrefPubMedGoogle Scholar
  • 4 Buxbaum JL, Abbas FehmiSM. ASGE Standards of Practice Committee. et al. ASGE guideline on the role of endoscopy in the evaluation and management of choledocholithiasis. Gastrointest Endosc 2019; 89: 1075-1105
    CrossrefPubMedGoogle Scholar
  • 5 Manes G, Paspatis G, Aabakken L. Endoscopic management of common bile duct stones: European Society of Gastrointestinal Endoscopy (ESGE) guideline. Endoscopy 2019; 51: 472-491
    Google Scholar
  • 6 Williams E, Beckingham I, El Sayed G. et al. Updated guideline on the management of common bile duct stones (CBDS). Gut 2017; 66: 765-782
    CrossrefPubMedGoogle Scholar
  • 7 Park SJ, Kim JH, Hwang JC. et al. Factors predictive of adverse events following endoscopic papillary large balloon dilation: results from a multicenter series. Dig Dis Sci 2013; 58: 1100-1109
    CrossrefPubMedGoogle Scholar
  • 8 Stefanidis G, Viazis N, Pleskow D. et al. Large balloon dilation vs. mechanical lithotripsy for the management of large bile duct stones: a prospective randomized study. Am J Gastroenterol 2011; 106: 278-285
    CrossrefPubMedGoogle Scholar
  • 9 Paspatis GA, Konstantinidis K, Tribonias G. et al. Sixty- versus thirty-seconds papillary balloon dilation after sphincterotomy for the treatment of large bile duct stones: a randomized controlled trial. Dig Liver Dis 2013; 45: 301-304
    CrossrefPubMedGoogle Scholar
  • 10 Teoh AY, Cheung FK, Hu B. et al. Randomized trial of endoscopic sphincterotomy with balloon dilation versus endoscopic sphincterotomy alone for removal of bile duct stones. Gastroenterology 2013; 144: 341-345.e1
    Google Scholar
  • 11 Kim HJ, Choi HS, Park JH. et al. Factors influencing the technical difficulty of endoscopic clearance of bile duct stones. Gastrointest Endosc 2007; 66: 1154-1160
    Google Scholar
  • 12 Lecun Y, Bengio Y, Hinton GE. Deep learning. Nature 2015; 521: 436-444
    CrossrefPubMedGoogle Scholar
  • 13 Torkamani A, Andersen KG, Steinhubl SR. et al. High-definition medicine. Cell 2017; 170: 828-843
    CrossrefPubMedGoogle Scholar
  • 14 Wu L, Zhang J, Zhou W. et al. Randomised controlled trial of WISENSE, a real-time quality improving system for monitoring blind spots during esophagogastroduodenoscopy. Gut 2019; 68: 2161-2169
    CrossrefPubMedGoogle Scholar
  • 15 Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 2017; 39: 640-651
    CrossrefPubMedGoogle Scholar
  • 16 Ouhmich F, Agnus V, Noblet V. et al. Liver tissue segmentation in multiphase CT scans using cascaded convolutional neural networks. Int J Comput Assist Radiol Surg 2019; 14: 1275-1284
    CrossrefPubMedGoogle Scholar
  • 17 Brandao P, Mazomenos EB, Ciuti G. et al. Fully convolutional neural networks for polyp segmentation in colonoscopy. Proc SPIE Int Soc Opt Eng 2017; 10134 DOI: 10.1117/12.2254361.
    CrossrefPubMedGoogle Scholar
  • 18 Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. Navab N, Hornegger J, Wells W, Frangi A. Medical image computing and computer-assisted intervention – MICCAI 2015. Lecture Notes in Computer Science. Springer; DOI: 10.1007/978-3-319-24574-4_28
    CrossrefGoogle Scholar
  • 19 Kim HJ, Choi HS, Park JH. et al. Factors influencing the technical difficulty of endoscopic clearance of bile duct stones. Gastrointest Endosc 2007; 66: 1154-1160
    Google Scholar
  • 20 Ragunath K, Thomas L, Cheung W. Objective evaluation of ERCP procedures: a simple grading scale for evaluating technical difficulty. Postgrad Med J 2002; 79: 467-470
    CrossrefPubMedGoogle Scholar
  • 21 Scheker LR. What predicts failed cannulation and therapy at ERCP? Results of a large-scale multicenter analysis. . Endoscopy 2012; 44: 674-683
    Article in Thieme ConnectPubMedGoogle Scholar
  • 22 Schutz SM, Abbott RM. Grading ERCPs by degree of difficulty: a new concept to produce more meaningful outcome data. Gastrointest Endosc 2000; 51: 535-539
    CrossrefPubMedGoogle Scholar
  • 23 Haseeb A, Freeman ML. Endoscopic papillary large balloon dilation versus endoscopic sphincterotomy for treatment of bile duct stones. Curr Treat Options Gastroenterol 2019; 17: 221-230
    CrossrefPubMedGoogle Scholar
  • 24 Rutter M, Senore C, Bisschops R. et al. The European Society of Gastrointestinal Endoscopy Quality Improvement Initiative: developing performance measures. Endoscopy 2015; 48: 81-89
    Article in Thieme ConnectPubMedGoogle Scholar
  • 25 Jiang H, Learned-Miller E. Face detection with the Faster R-CNN. arXiv.org; 2016
    PubMedGoogle Scholar
  • 26 Mchenry L, Lehman GA. Difficult bile duct stones. Curr Treat Options Gastroenterol 2006; 9: 123-132
    CrossrefPubMedGoogle Scholar
  • 27 Manes G, Paspatis G, Aabakken L. et al. Endoscopic management of common bile duct stones: European Society of Gastrointestinal Endoscopy (ESGE) guideline. Endoscopy 2019; 51: 472-491
    Google Scholar
  • 28 Heo JH, Kang DH, Jung HJ. et al. Endoscopic sphincterotomy plus large-balloon dilation versus endoscopic sphincterotomy for removal of bile-duct stones. Gastrointest Endosc 2007; 66: 720-726
    CrossrefPubMedGoogle Scholar
  • 29 Stefanidis G, Viazis N, Pleskow D. et al. Large balloon dilation vs. mechanical lithotripsy for the management of large bile duct stones: a prospective randomized study. Am J Gastroenterol 2011; 106: 278-285
    PubMedGoogle Scholar