Interobserver agreement of the Paris and simplified classifications of superficial colonic lesions: a Western study

Abstract

Background and study aims The Paris classification of superficial colonic lesions has been widely adopted, but a simplified description that subgroups the shape into pedunculated, sessile/flat and depressed lesions has been proposed recently. The aim of this study was to evaluate the accuracy and inter-rater agreement among 13 Western endoscopists for the two classification systems.

Methods Seventy video clips of superficial colonic lesions were classified according to the two classifications, and their size estimated. The interobserver agreement for each classification was assessed using both Cohen k and AC1 statistics. Accuracy was taken as the concordance between the standard morphology definition and that made by participants. Sensitivity analyses investigated agreement between trainees (T) and staff members (SM), simple or mixed lesions, distinct lesion phenotypes, and for laterally spreading tumors (LSTs).

Results Overall, the interobserver agreement for the Paris classification was substantial (κ = 0.61; AC1 = 0.66), with 79.3 % accuracy. Between SM and T, the values were superimposable. For size estimation, the agreement was 0.48 by the κ-value, and 0.50 by AC1. For single or mixed lesions, κ-values were 0.60 and 0.43, respectively; corresponding AC1 values were 0.68 and 0.57. Evaluating the several different polyp subtypes separately, agreement differed significantly when analyzed by the k-statistics (0.08–0.12) or the AC1 statistics (0.59–0.71). Analyses of LSTs provided a κ-value of 0.50 and an AC1 score of 0.62, with 77.6 % accuracy. The simplified classification outperformed the Paris classification: κ = 0.68, AC1 = 0.82, accuracy = 91.6 %.

Conclusions Agreement is often measured with Cohen’s κ, but we documented higher levels of agreement when analyzed with the AC1 statistic. The level of agreement was substantial for the Paris classification, and almost perfect for the simplified system.

Publication History

Received: 17 August 2020

Accepted: 09 December 2020

Article published online:
19 February 2021

• References

• 1 Kudo S, Lambert R, Allen JI. et al. Nonpolypoid neoplastic lesions of the colorectal mucosa. Gastrointest Endosc 2008; 68: S3-S47
• 2 The Paris endoscopic classification of superficial neoplastic lesions. esophagus, stomach, and colon. November 30 to December 1, 2002. Gastrointest Endosc 2003; 58: S3-S43
• 3 Endoscopic Classification Review Group. Update on the Paris classification of superficial neoplastic lesions in the digestive tract. Endoscopy 2005; 37: 570-578
• 4 Ferlitsch M, Moss A, Hassan C. et al. Colorectal polypectomy and endoscopic mucosal resection (EMR): European Society of Gastrointestinal Endoscopy (ESGE) Clinical Guideline. Endoscopy 2017; 49: 270-297
• 5 Kaltenbach T, Anderson JC, Burke CA. et al. Endoscopic removal of colorectal lesions – Recommendations by the US multi-society task force on colorectal cancer. Gastrointest Endosc 2020; 91: 486-519
• 6 Japanese Society for Cancer of the Colon and Rectum. Japanese classification of colorectal, appendiceal, and anal carcinoma: the 3rd English version. J Anus Rectum Colon 2019; 3: 175-195
• 7 Rao AK, Soetikno R, Raju GS. et al. Large sessile serrated polyps can be safely and effectively removed by endoscopic mucosal resection. Clin Gastroenterol Hepatol 2016; 14: 568-574
• 8 Van Doorn SC, Hazewinkel Y, East EJ. et al. Polyp Morphology: an interobserver evaluation for the Paris Classification among international experts. Am J Gastroenterol 2015; 110: 180-187
• 9 Kim JH, Nam KS, Kwon HJ. et al. Assessment of colon polyp morphology: is education effective?. World J Gastroenterol 2017; 23: 6281-6286
• 10 Aziz Aadam A, Wani S, Kahi CH. et al. Physician assessment and management of complex colon polyps: a multicenter video-based survey study. Am J Gastroenterol 2014; 109: 1312-1317
• 11 Lee YJ, Kim ES, Park KS. et al. Inter-observer agreement in the endoscopic classification of colorectal laterally spreading tumors: a multicenter study between experts and trainees. Dig Dis Sci 2014; 59: 2550-2556
• 12 Bogie RMM, Veldman MHJ, Snijders LARS. et al. Endoscopic subtypes of colorectal laterally spreading tumors (LSTs) and the risk of submucosal invasion: a meta-analysis. Endoscopy 2018; 50: 263-282
• 13 Kottner J, Audigé L, Brorson S. et al. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol 2011; 64: 96-106
• 14 Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 1990; 43: 543-549
• 15 Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol 1990; 43: 551-558
• 16 Di Eugenio B, Glass M. The kappa statistic: a second look. Comput Linguist 2004; 30: 95-101
• 17 Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol 2008; 61: 29-48
• 18 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-174
• 19 Vleugels JLA, Hazewinkel Y, Dekker E. Morphological classifications of gastrointestinal lesions. Best Pract Res Clin Gastroenterol 2017; 31: 359-367
• 20 Matsuda T, Fujii T, Saito Y. et al. Efficacy of the invasive/non-invasive pattern by magnifying chromoendoscopy to estimate the depth of invasion of early colorectal neoplasms. Am J Gastroenterol 2008; 103: 2700-2706
• 21 Iwatate M, Ikumoto T, Hattori S. et al. NBI and NBI combined with magnifying colonoscopy. Diagn Ther Endosc 2012; 2012: 173269
• 22 Rex DK, Hassan C, Bourke MJ. The colonscopist’s guide to the vocabulary of colorectal neoplasia: histology, morphology, and management. Gastrointest Endosc 2017; 86: 253-263
• 23 Bianco MA, Cipolletta L, Rotondano G. et al. Prevalence of nonpolypoid colorectal neoplasia: an Italian multicenter observational study [published correction appears in Endoscopy. 2010 Jul; 42(7): 563]. Endoscopy 2010; 42: 279-285
• 24 Soetikno RM, Kaltenbach T, Rouse RV. et al. Prevalence of nonpolypoid (flat and depressed) colorectal neoplasms in asymptomatic and symptomatic adults. JAMA 2008; 299: 1027-1035
• 25 Sanduleanu S, Rondagh EJ, Masclee AA. Development of expertise in the detection and classification of non-polypoid colorectal neoplasia: Experience-based data at an academic GI unit. Gastrointest Endosc Clin N Am 2010; 20: 449-460
• 26 Kim BC, Chang HJ, Han KS. et al. Clinicopathological differences of laterally spreading tumors of the colorectum according to gross appearance. Endoscopy 2011; 43: 100-107
• 27 Rembacken BJ, Fujii T, Cairns A. et al. Flat and depressed colonic neoplasms: a prospective study of 1000 colonoscopies in the UK. Lancet 2000; 355: 1211-1214
• 28 Puig I, Mármol C, Bustamante M. Endoscopic imaging techniques for detecting early colorectal cancer. Curr Opin Gastroenterol 2019; 35: 432-439
• 29 Bisschops R, East JE, Hassan C. et al. Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) Guideline - Update 2019. Endoscopy 2019; 51: 1155-1179