A real-time deep learning-based system for colorectal polyp size estimation by white-light endoscopy: development and multicenter prospective validation

Abstract

Background The choice of polypectomy device and surveillance intervals for colorectal polyps are primarily decided by polyp size. We developed a deep learning-based system (ENDOANGEL-CPS) to estimate colorectal polyp size in real time.

Methods ENDOANGEL-CPS calculates polyp size by estimating the distance from the endoscope lens to the polyp using the parameters of the lens. The depth estimator network was developed on 7297 images from five virtually produced colon videos and tested on 730 images from seven virtual colon videos. The performance of the system was first evaluated in nine videos of a simulated colon with polyps attached, then tested in 157 real-world prospective videos from three hospitals, with the outcomes compared with that of nine endoscopists over 69 videos. Inappropriate surveillance recommendations caused by incorrect estimation of polyp size were also analyzed.

Results The relative error of depth estimation was 11.3% (SD 6.0%) in successive virtual colon images. The concordance correlation coefficients (CCCs) between system estimation and ground truth were 0.89 and 0.93 in images of a simulated colon and multicenter videos of 157 polyps. The mean CCC of ENDOANGEL-CPS surpassed all endoscopists (0.89 vs. 0.41 [SD 0.29]; P<0.001). The relative accuracy of ENDOANGEL-CPS was significantly higher than that of endoscopists (89.9% vs. 54.7%; P<0.001). Regarding inappropriate surveillance recommendations, the system's error rate is also lower than that of endoscopists (1.5% vs. 16.6%; P<0.001).

Conclusions ENDOANGEL-CPS could potentially improve the accuracy of colorectal polyp size measurements and size-based surveillance intervals.

Publication History

Received: 27 May 2023

Accepted after revision: 11 October 2023

Accepted Manuscript online:
12 October 2023

Article published online:
05 December 2023

• References

• 1 Rutter MD, Jover R. Personalizing polypectomy techniques based on polyp characteristics. Clin Gastroenterol Hepatol 2020; 18: 2859-2867 DOI: 10.1016/j.cgh.2019.09.025. (PMID: 31563558)
• 2 Ferlitsch M, Moss A, Hassan C. et al. Colorectal polypectomy and endoscopic mucosal resection (EMR): European Society of Gastrointestinal Endoscopy (ESGE) Clinical Guideline. Endoscopy 2017; 49: 270-297 DOI: 10.1055/s-0043-102569. (PMID: 28212588)
• 3 Rutter MD, East J, Rees CJ. et al. British Society of Gastroenterology/Association of Coloproctology of Great Britain and Ireland/Public Health England post-polypectomy and post-colorectal cancer resection surveillance guidelines. Gut 2020; 69: 201-223 DOI: 10.1136/gutjnl-2019-319858. (PMID: 31776230)
• 4 Tanaka S, Saitoh Y, Matsuda T. et al. Evidence-based clinical practice guidelines for management of colorectal polyps. J Gastroenterol 2021; 56: 323-335 DOI: 10.1007/s00535-021-01776-1. (PMID: 33710392)
• 5 Gupta S, Lieberman D, Anderson JC. et al. Recommendations for follow-up after colonoscopy and polypectomy: a consensus update by the US Multi-Society Task Force on Colorectal Cancer. Gastroenterology 2020; 158: 1131-1153.e1135
• 6 Smith RA, Andrews KS, Brooks D. et al. Cancer screening in the United States, 2019: A review of current American Cancer Society guidelines and current issues in cancer screening. CA Cancer J Clin 2019; 69: 184-210 DOI: 10.3322/caac.21557. (PMID: 30875085)
• 7 Ralaidovy AH, Gopalappa C, Ilbawi A. et al. Cost-effective interventions for breast cancer, cervical cancer, and colorectal cancer: new results from WHO-CHOICE. Cost Eff Resour Alloc 2018; 16: 38
• 8 Sakata S, McIvor F, Klein K. et al. Measurement of polyp size at colonoscopy: a proof-of-concept simulation study to address technology bias. Gut 2018; 67: 206-208 DOI: 10.1136/gutjnl-2016-312915. (PMID: 27697826)
• 9 Utsumi T, Horimatsu T, Nishikawa Y. et al. Short educational video to improve the accuracy of colorectal polyp size estimation: Multicenter prospective study. Digestive Endoscopy 2020; 32: 1074-1081 DOI: 10.1111/den.13638. (PMID: 31994222)
• 10 Elwir S, Shaukat A, Shaw M. et al. Variability in, and factors associated with, sizing of polyps by endoscopists at a large community practice. Endosc Int Open 2017; 5: E742-E745
• 11 Sakata S, Klein K, Stevenson ARL. et al. Measurement bias of polyp size at colonoscopy. Dis Colon Rectum 2017; 60: 987-991 DOI: 10.1097/DCR.0000000000000859. (PMID: 28796738)
• 12 Plumb AA, Nickerson C, Wooldrage K. et al. Terminal digit preference biases polyp size measurements at endoscopy, computed tomographic colonography, and histopathology. Endoscopy 2016; 48: 899-908 DOI: 10.1055/s-0042-108727. (PMID: 27441685)
• 13 Rex DK, Rabinovitz R. Variable interpretation of polyp size by using open forceps by experienced colonoscopists. Gastrointest Endosc 2014; 79: 402-407 DOI: 10.1016/j.gie.2013.08.030. (PMID: 24119506)
• 14 Itoh H, Roth HR, Mori Y. et al. Polyp-size classification with RGB-D features for colonoscopy. In Medical Imaging 2019: Computer-Aided Diagnosis (Vol. 10950, pp. 282-288). SPIE.
• 15 Itoh H, Roth HR, Lu L. et al. Towards automated colonoscopy diagnosis: binary polyp size estimation via unsupervised depth learning. In: Frangi AF, Schnabel JA, Davatzikos C. Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Proceedings of the 21st International Conference; 16–20 September 2018; Granada, Spain; Part II. Cham, Switzerland: Springer; 611-619
• 16 Ling T, Wu L, Fu Y. et al. A deep learning-based system for identifying differentiation status and delineating the margins of early gastric cancer in magnifying narrow-band imaging endoscopy. Endoscopy 2021; 53: 469-477 DOI: 10.1055/a-1229-0920. (PMID: 32725617)
• 17 The Paris endoscopic classification of superficial neoplastic lesions: esophagus, stomach, and colon: November 30 to December 1, 2002. Gastrointest Endosc 2003; 58: S3-S43 DOI: 10.1016/s0016-5107(03)02159-x. (PMID: 14652541)
• 18 Lai EJ, Calderwood AH, Doros G. et al. The Boston bowel preparation scale: a valid and reliable instrument for colonoscopy-oriented research. Gastrointest Endosc 2009; 69: 620-625 DOI: 10.1016/j.gie.2008.05.057. (PMID: 19136102)
• 19 Hassan C, Antonelli G, Dumonceau JM. et al. Post-polypectomy colonoscopy surveillance: European Society of Gastrointestinal Endoscopy (ESGE) Guideline - Update 2020. Endoscopy 2020; 52: 687-700 DOI: 10.1055/a-1185-3109. (PMID: 32572858)
• 20 Lawrence I, Lin K. A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989; 45: 255-268
• 21 Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res 2005; 19: 231-240 DOI: 10.1519/15184.1. (PMID: 15705040)
• 22 Kaz AM, Anwar A, O'Neill DR. et al. Use of a novel polyp "ruler snare" improves estimation of colon polyp size. Gastrointest Endosc 2016; 83: 812-816 DOI: 10.1016/j.gie.2015.08.082. (PMID: 26382052)
• 23 Shaukat A, Shamsi N, Menk J. et al. Polyp sizing poster improves polyp measurement but not adenoma detection rates by endoscopists in a large community practice. Clin Gastroenterol Hepatol 2019; 17: 2034-2041 DOI: 10.1016/j.cgh.2018.10.008. (PMID: 30312788)
• 24 Kwak MS, Cha JM, Jeon JW. et al. Artificial intelligence-based measurement outperforms current methods for colorectal polyp size measurement. Dig Endosc 2022; 34: 1188-1195 DOI: 10.1111/den.14318. (PMID: 35385184)
• 25 von Renteln D, Djinbachian R, Zarandi-Nowroozi M. et al. Measuring size of smaller colorectal polyps using a virtual scale function during endoscopies. Gut 2023; 72: 417
• 26 Djinbachian R, Taghiakbari M, Haumesser C. et al. Comparing size measurement of colorectal polyps using a novel virtual scale endoscope, endoscopic ruler or forceps: A preclinical randomized trial. Endosc Int Open 2023; 11: E128-E135 DOI: 10.1055/a-2005-7548. (PMID: 36726860)
• 27 Yao S, Zhang J, Hu Z. et al. Autonomous driving vehicle test technology based on virtual reality. J Eng 2018; 2018: 1768-1771
• 28 Anderson BW, Smyrk TC, Anderson KS. et al. Endoscopic overestimation of colorectal polyp size. Gastrointest Endosc 2016; 83: 201-208 DOI: 10.1016/j.gie.2015.06.058. (PMID: 26318830)
• 29 Chaptini L, Chaaya A, Depalma F. et al. Variation in polyp size estimation among endoscopists and impact on surveillance intervals. Gastrointest Endosc 2014; 80: 652-659
• 30 Kessler WR, Imperiale TF, Klein RW. et al. A quantitative assessment of the risks and cost savings of forgoing histologic examination of diminutive polyps. Endoscopy 2011; 43: 683-691 DOI: 10.1055/s-0030-1256381. (PMID: 21623556)
• 31 Eichenseer PJ, Dhanekula R, Jakate S. et al. Endoscopic mis-sizing of polyps changes colorectal cancer surveillance recommendations. Dis Colon Rectum 2013; 56: 315-321 DOI: 10.1097/DCR.0b013e31826dd138. (PMID: 23392145)