Large language model for interpreting the Paris classification of colorectal polyps

Abstract

Background and study aims

Reporting of colorectal polyp morphology using the Paris classification is often inaccurate. Multimodal large language models (M-LLMs) may support morphological assessment. This study aimed to evaluate the accuracy of an M-LLM (GPT-4o) in classifying colorectal polyp morphology compared with expert and non-expert endoscopists.

Patients and methods

We used the SUN dataset of colonoscopy videos from 100 unique colorectal polyps, each labeled with the validated Paris classification. An M-LLM (GPT-4o) classified five representative frames per lesion. Three expert and three non-expert endoscopists, blinded to one another, performed the same task. The primary outcome was accuracy in differentiating non-polypoid (IIa/IIc) from polypoid (Is/Ip/Isp) lesions. The secondary outcome was accuracy in differentiating sessile (Is) from pedunculated (Ip/Isp) lesions. Given the exploratory design, no multiplicity correction was applied; point estimates are presented with 95% confidence intervals (CIs), and P values are interpreted descriptively.

Results

M-LLM accuracy for differentiating non-polypoid from polypoid lesions was 73% (95% CI 63%-81%), comparable to experts (75%, 65%-83%; P = 0.84) and non-experts (77%, 68%-85%; P = 0.52), with similar sensitivity and specificity. Accuracy for differentiating sessile from pedunculated lesions was 55% (95% CI 42%-67%), lower than experts (76%; P = 0.02) and non-experts (77%; P = 0.01), primarily due to poor specificity (12% vs. experts 82% and non-experts 88%; P < 0.01 for both comparisons).

Conclusions

M-LLMs performed comparably to endoscopists in distinguishing non-polypoid from polypoid lesions but failed to reliably identify pedunculated morphology.

Keywords

Endoscopy Lower GI Tract - Polyps / adenomas / ... - Colorectal cancer - Tissue diagnosis - CRC screening - Diagnosis and imaging (inc chromoendoscopy, NBI, iSCAN, FICE, CLE...)

• References

• 1 Participants in the Paris Workshop. The Paris endoscopic classification of superficial neoplastic lesions: esophagus, stomach, and colon. Gastrointestinal Endoscopy 2003; 58: S3-S43
• 2 van Doorn SC, Hazewinkel Y, East JE. et al. Polyp morphology: an interobserver evaluation for the Paris classification among international experts. Am J Gastroenterol 2015; 110: 180-187
• 3 Hassan C, Spadaccini M, Mori Y. et al. Real-time computer-aided detection of colorectal neoplasia during colonoscopy: A systematic review and meta-analysis. Ann Intern Med 2023; 176: 1209-1220
• 4 Hassan C, Misawa M, Rizkala T. et al. Computer-aided diagnosis for leaving colorectal polyps in situ: A systematic review and meta-analysis. Ann Intern Med 2024; 77: 919-928
• 5 Hassan C, Rizkala T, Mori Y. et al. Computer-aided diagnosis for the resect-and-discard strategy for colorectal polyps: a systematic review and meta-analysis. Lancet Gastroenterol Hepatol 2024; 9: 1010-1019
• 6 Krenzer A, Heil S, Fitting D. et al. Automated classification of polyps using deep learning architectures and few-shot learning. BMC Med Imaging 2023; 23: 59
• 7 Zhang Y, Pan Y, Zhong T. et al. Potential of multimodal large language models for data mining of medical images and free-text reports. arXiv 2024;
• 8 Carlini L, Massimi D, Mori Y. et al. Large language models detecting colorectal polyps on endoscopic images. Gut 2025;
• 9 Misawa M, Kudo SE, Mori Y. et al. Development of a computer-aided detection system for colonoscopy and a publicly accessible large colonoscopy video database (with video). Gastrointest Endosc 2021; 93: 960-967.e3
• 10 ChatGPT version 4o. https://chatgpt.com