The development and ex vivo evaluation of a computer-aided quality control system for Barrett’s esophagus endoscopy

Abstract

Background Timely detection of neoplasia in Barrett’s esophagus (BE) remains challenging. While computer-aided detection (CADe) systems have been developed to assist endoscopists, their effectiveness depends heavily on the quality of the endoscopic procedure. This study introduces a novel computer-aided quality (CAQ) system for BE, evaluating its stand-alone performance and integration with a CADe system.

Method The CAQ system was developed using 7,463 images from 359 BE patients. It assesses objective quality parameters (e. g., blurriness, illumination) and subjective parameters (mucosal cleanliness, esophageal expansion) and can exclude low-quality images when integrated with a CADe system.

To evaluate CAQ stand-alone performance, the Endoscopic Image Quality test set, consisting of 647 images from 51 BE patients across 8 hospitals, was labeled for objective and subjective quality. To assess the benefit of the CAQ system as a preprocessing filter of a CADe system, the Barrett CADe test set was developed. It consisted of 956 video frames from 62 neoplastic patients and 557 frames from 35 non-dysplastic patients, in 12 Barrett referral centers.

Results As stand-alone tool, the CAQ system achieved Cohen’s Kappa scores of 0.73, 0.91, and 0.89 for objective quality, mucosal cleanliness, and esophageal expansion, comparable to inter-annotator scores of 0.73, 0.93, and 0.83. As preprocessing filter, the CAQ system improved CADe sensitivity from 82 % to 90 % and AUC from 87 % to 91 %, while maintaining specificity at 75 %.

Conclusion This study presents the first CAQ system for automated quality control in BE. The system effectively distinguishes poorly from well-visualized mucosa and enhances neoplasia detection when integrated with CADe.

^* All members and collaborators of the Barrett’s Oesophagus Imaging for Artificial Intelligence (BONS-AI) Consortium are listed in the Supplementary materials.

Publication History

Received: 29 November 2024

Accepted after revision: 10 February 2025

Accepted Manuscript online:
11 February 2025

Article published online:
06 March 2025

• References

• 1 Schölvinck DW, van der Meulen K, Bergman J. et al. Detection of lesions in dysplastic Barrett’s esophagus by community and expert endoscopists. Endoscopy 2017; 49: 113-120
• 2 Sawas T, Majzoub AM, Haddad J. et al. Magnitude and time-trend analysis of postendoscopy esophageal adenocarcinoma: a systematic review and meta-analysis. Clin Gastroenterol Hepatol 2022; 20: e31-e50
• 3 Bergman J, de Groof AJ, Pech O. et al. An interactive web-based educational tool improves detection and delineation of Barrett’s esophagus-related neoplasia. Gastroenterology 2019; 156: 1299-308.e3
• 4 Meinikheim M, Mendel R, Palm C. et al. Influence of artificial intelligence on the diagnostic performance of endoscopists in the assessment of Barrett’s esophagus: a tandem randomized and video trial. Endoscopy 2024; 56: 641-649
• 5 Abdelrahim M, Saiko M, Maeda N. et al. Development and validation of artificial neural networks model for detection of Barrett’s neoplasia: a multicenter pragmatic nonrandomized trial (with video). Gastrointest Endosc 2023; 97: 422-434
• 6 Hussein M, González-Bueno JPuyal, Lines D. et al. A new artificial intelligence system successfully detects and localises early neoplasia in Barrett’s esophagus by using convolutional neural networks. United European Gastroenterol J 2022; 10: 528-537
• 7 Fockens KN, Jukema JB, Boers T. et al. Towards a robust and compact deep learning system for primary detection of early Barrett’s neoplasia: initial image-based results of training on a multi-center retrospectively collected data set. United European Gastroenterol J 2023; 11: 324-336
• 8 Fockens N, Jong MR, Jukema JB. et al. A deep learning system for detection of early Barrett’s neoplasia: a model development and validation study. Lancet Digit Health 2023; 5: 905-921
• 9 Jong MR, Jaspers TJ, Kusters KC. et al. Tu2011 Image quality challenges in AI: improving robustness of a computer aided detection system for Barrett’s neoplasia. Gastroenterology 2024; 166 (Suppl. 05) S-1491
• 10 Weusten BLAM, Bisschops R, Dinis-Ribeiro M. et al. Diagnosis and management of Barrett esophagus: European Society of Gastrointestinal Endoscopy (ESGE) Guideline. Endoscopy 2023; 55: 1124-1146
• 11 Wu L, Zhang J, Zhou W. et al. Randomised controlled trial of WISENSE, a real-time quality improving system for monitoring blind spots during esophagogastroduodenoscopy. Gut 2019; 68: 2161-2169
• 12 Wu L, He X, Liu M. et al. Evaluation of the effects of an artificial intelligence system on endoscopy quality and preliminary testing of its performance in detecting early gastric cancer: a randomized controlled trial. Endoscopy 2021; 53: 1199-1207
• 13 Barua I, Misawa M, Glissen Brown JR. et al. Speedometer for withdrawal time monitoring during colonoscopy: a clinical implementation trial. Scand J Gastroenterol 2023; 58: 664-670
• 14 Zhang L, Lu Z, Yao L. et al. Effect of a deep learning-based automatic upper GI endoscopic reporting system: a randomized crossover study (with video). Gastrointest Endosc 2023; 98: 181-190.e10
• 15 Yao L, Zhang L, Liu J. et al. Effect of an artificial intelligence-based quality improvement system on efficacy of a computer-aided detection system in colonoscopy: a four-group parallel study. Endoscopy 2022; 54: 757-768
• 16 Li YD, Zhu SW, Yu JP. et al. Intelligent detection endoscopic assistant: an artificial intelligence-based system for monitoring blind spots during esophagogastroduodenoscopy in real-time. Dig Liver Dis 2021; 53: 216-223
• 17 Zhou R, Liu J, Zhang C. et al. Efficacy of a real-time intelligent quality-control system for the detection of early upper gastrointestinal neoplasms: a multicentre, single-blinded, randomised controlled trial. EClinicalMedicine 2024; 75: 102803
• 18 Boers TGW, Fockens KN, van der Putten JA. et al. Foundation models in gastrointestinal endoscopic AI: impact of architecture, pre-training approach and data efficiency. Med Image Anal 2024; 98: 103298
• 19 Jelmer J, Fockens K, Boers T. et al. Domain-specific pretraining of deep learning systems in gastrointestinal endoscopy improves performance over current state-of-the-art pretraining methods. Endoscopy 2023; 55 (Suppl. 02) S83-S84
• 20 Esposito G, Dilaghi E, Santos C. et al. The Gastroscopy RAte of Cleanliness Evaluation (GRACE) Scale: an international reliability and validation study. Endoscopy 2024;
• 21 Gong D, Wu L, Zhang J. et al. Detection of colorectal adenomas with a real-time computer-aided system (ENDOANGEL): a randomised controlled study. Lancet Gastroenterol Hepatol 2020; 5: 352-361