Download PDF
Endoscopy 2023; 55(02): 176-185
DOI: 10.1055/a-1929-1318
Systematic review

Accuracy of self-assessment in gastrointestinal endoscopy: a systematic review and meta-analysis

Michael A. Scaffidi  ‡   *
1   Division of Gastroenterology, St. Michael’s Hospital, University of Toronto, Toronto, Canada
2   Faculty of Health Sciences, School of Medicine, Queen’s University, Toronto, Canada
,
Juana Li‡   *
1   Division of Gastroenterology, St. Michael’s Hospital, University of Toronto, Toronto, Canada
,
Shai Genis 
1   Division of Gastroenterology, St. Michael’s Hospital, University of Toronto, Toronto, Canada
,
Elizabeth Tipton
3   Department of Statistics and Institute for Policy Research, Northwestern University, Evanston, Illinois, USA
,
Rishad Khan
1   Division of Gastroenterology, St. Michael’s Hospital, University of Toronto, Toronto, Canada
,
Chandni Pattni
1   Division of Gastroenterology, St. Michael’s Hospital, University of Toronto, Toronto, Canada
,
Nikko Gimpaya
1   Division of Gastroenterology, St. Michael’s Hospital, University of Toronto, Toronto, Canada
,
Glyneva Bradley-Ridout
4   Gerstein Science Information Centre, University of Toronto, Toronto, Canada
,
Catharine M. Walsh 
5   Department of Paediatrics, University of Toronto, Toronto, Canada
6   Department of Medicine, University of Toronto, Toronto, Canada
7   Division of Gastroenterology, Hepatology and Nutrition, Hospital for Sick Children, University of Toronto, Toronto, Canada
,
Samir C. Grover 
1   Division of Gastroenterology, St. Michael’s Hospital, University of Toronto, Toronto, Canada
8   Li Ka Shing Knowledge Institute, Toronto, Canada
› Author Affiliations
› Further Information
Also available at
    Permissions and Reprints


Abstract

Background Assessment is necessary to ensure both attainment and maintenance of competency in gastrointestinal (GI) endoscopy, and this can be accomplished through self-assessment. We conducted a systematic review with meta-analysis to evaluate the accuracy of self-assessment among GI endoscopists.

Methods This was an individual participant data meta-analysis of studies that investigated self-assessment of endoscopic competency. We performed a systematic search of the following databases: Ovid MEDLINE, Ovid EMBASE, Wiley Cochrane CENTRAL, and ProQuest Education Resources Information Center. We included studies if they were primary investigations of self-assessment accuracy in GI endoscopy that used statistical analyses to determine accuracy. We conducted a meta-analysis of studies using a limits of agreement (LoA) approach to meta-analysis of Bland–Altman studies.

Results After removing duplicate entries, we screened 7138 records. After full-text review, we included 16 studies for qualitative analysis and three for meta-analysis. In the meta-analysis, we found that the LoA were wide (−41.0 % to 34.0 %) and beyond the clinically acceptable difference. Subgroup analyses found that both novice and intermediate endoscopists had wide LoA (−45.0 % to 35.1 % and −54.7 % to 46.5 %, respectively) and expert endoscopists had narrow LoA (−14.2 % to 21.4 %).

Conclusions GI endoscopists are inaccurate in self-assessment of their endoscopic competency. Subgroup analyses demonstrated that novice and intermediate endoscopists were inaccurate, while expert endoscopists have accurate self-assessment. While we advise against the sole use of self-assessment among novice and intermediate endoscopists, expert endoscopists may wish to integrate it into their practice.

* Equal first authors


Supplementary material



Publication History

Received: 06 November 2021

Accepted after revision: 20 June 2022

Article published online:
26 September 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Scaffidi MA, Khan R, Grover SC. et al. Self-assessment of competence in endoscopy: challenges and insights. J Can Assoc Gastroenterol 2020; 4: 151-157
    CrossrefPubMedGoogle Scholar
  • 2 Panadero E. A review of self-regulated learning: Six models and four directions for research. Front Psychol 2017; 8: 1-28
    CrossrefPubMedGoogle Scholar
  • 3 Panadero E, Jonsson A, Botella J. Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses. Educ Res Rev 2017; 22: 74-98
    CrossrefPubMedGoogle Scholar
  • 4 Davis DA, Mazmanian PE, Fordis M. et al. Accuracy of physician self-assessment compared a systematic review. JAMA 2006; 296: 1094-1102
    CrossrefPubMedGoogle Scholar
  • 5 Blanch-Hartigan D. Medical students’ self-assessment of performance: Results from three meta-analyses. Patient Educ Couns 2011; 84: 3-9
    CrossrefPubMedGoogle Scholar
  • 6 Nayar SK, Musto L, Baruah G. et al. Self-assessment of surgical skills: a systematic review. J Surg Educ 2020; 77: 348-361
    CrossrefPubMedGoogle Scholar
  • 7 Shamseer L, Moher D, Clarke M. et al. Preferred reporting items for systematic review and meta-analysis protocols (PRIMSA-P) 2015: Elaboration and explanation. BMJ 2015; 349: 1-25
    CrossrefPubMedGoogle Scholar
  • 8 McGowan J, Sampson M, Salzwedel DM. et al. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol 2016; 75: 40-46
    CrossrefPubMedGoogle Scholar
  • 9 Bramer WM, Giustini D, De Jonge GB . et al. De-duplication of database search results for systematic reviews in EndNote. J Med Libr Assoc 2016; 104: 240-243
    CrossrefPubMedGoogle Scholar
  • 10 National Institute for Health Research. PROSPERO International prospective register of systematic reviews [Internet]. Accessed: 17 August 2020. Available from: www.crd.york.ac.uk/PROSPERO
    PubMedGoogle Scholar
  • 11 World Health Organization. International Clinical Trials Registry Platform [Internet]. Accessed: 17 August 2020. Available from: apps.who.int/trialsearch
    PubMedGoogle Scholar
  • 12 Walsh CM, Ling SC, Khanna N. et al. Gastrointestinal endoscopy competency assessment tool: Reliability and validity evidence. Gastrointest Endosc 2015; 81: 1417-1424
    Google Scholar
  • 13 Khan R, Zheng E, Wani S. et al. Colonoscopy competence assessment tools: A systematic review of validity evidence. Endoscopy 2021; 53: 1235-1245
    Article in Thieme ConnectPubMedGoogle Scholar
  • 14 Whiting PF, Rutjes AWS, Westwood ME. et al. QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011; 155: 529-536
    CrossrefPubMedGoogle Scholar
  • 15 Tipton E, Shuster J. A framework for the meta-analysis of Bland-Altman studies based on a limits of agreement approach. Stat Med 2017; 176: 139-148
    PubMedGoogle Scholar
  • 16 Walsh CM, Ling SC, Mamula P. et al. The gastrointestinal endoscopy competency assessment tool for pediatric colonoscopy. J Pediatr Gastroenterol Nutr 2015; 60: 474-480
    CrossrefPubMedGoogle Scholar
  • 17 Al-Mansour MR, Tyler KM, Kutayli ZN. et al. Concurrent performance in virtual reality and clinical endoscopy. SAGES conference 2012. Accessed: 11 July 2022. Available from: https://www.sages.org/meetings/annual-meeting/abstracts-archive/?search=P171&meeting=2012
    PubMedGoogle Scholar
  • 18 Amadio JM, Walsh CM, Scaffidi MA. et al. Su1553: Can novice endoscopists accurately self-assess performance during their initial clinical colonoscopies? a prospective, cross-sectional study. . Gastrointest Endosc 2015; 81: AB325
    CrossrefPubMedGoogle Scholar
  • 19 Ansell J, Hurley JJ, Horwood J. et al. Can endoscopists accurately self-assess performance during simulated colonoscopic polypectomy? A prospective, cross-sectional study. . Am J Surg 2014; 207: 32-38
    Google Scholar
  • 20 Figueroa Barojas Barojas, Sobrino-Cossio SR, Hernandez-Guerrero A. et al. M1412: Endoscopic inanimate biological simulators for training in endoscopic mucosal dissection. Gastrointest Endosc 2010; 71: AB214
    CrossrefPubMedGoogle Scholar
  • 21 Cookson TA, Beilman CL, Kohansal AR. et al. A231: Endoscopy trainees are able to accurately self-assess procedural skills. J Can Assoc Gastroenterol 2019; 2 (Suppl. 02) 451-452
    CrossrefPubMedGoogle Scholar
  • 22 Lee RX, Smith M, Samuels S. et al. WP38: Accuracy of clinician-reported adenoma detection rate in colonoscopy. Colorectal Dis 2019; 21 (Suppl. 03) 19-32
    CrossrefPubMedGoogle Scholar
  • 23 Moritz V, Holme O, Leblanc M. et al. An explorative study from the Norwegian Quality Register Gastronet comparing self-estimated versus registered quality in colonoscopy performance. Endosc Int Open 2016; 04: E326-E332
    Article in Thieme ConnectPubMedGoogle Scholar
  • 24 Muething L, Kaltenbach TR, Nguyen-vu T. et al. Fr051: Moderate concordance between trainee self-assessment and expert assessment of cold snare polypectomy competence: results from a randomized controlled study. Gastroenterology 2020; 160: 197
    CrossrefPubMedGoogle Scholar
  • 25 Neumann M, Hahn C, Horbach T. et al. Score card endoscopy: A multicenter study to evalaute learning curves in 1-week courses using the Elangen Endo-Trainer. Endoscopy 2003; 35: 515-520
    Article in Thieme ConnectPubMedGoogle Scholar
  • 26 Sedlack RE, Kolars JC, Alexander JA. Computer simulation training enhances patient comfort during endoscopy. Clin Gastroenterol Hepatol 2004; 2: 348-352
    Google Scholar
  • 27 Vassiliou MC, Kaneva PA, Poulose BK. et al. Global Assessment of Gastrointestinal Endoscopic Skills (GAGES): a valid measurement tool for technical skills in flexible endoscopy. Surg Endosc 2010; 24: 1834-1841
    Google Scholar
  • 28 Vyasa P, Willis RE, Dunkin BJ. et al. Are general surgery residents accurate assessors of their own flexible endoscopy skills?. J Surg Educ 2016; 74: 23-29
    CrossrefPubMedGoogle Scholar
  • 29 Wani S, Muthusamy VR, Wang AY. et al. Su1541: Interobserver agreement between trainers and trainees: results from a multicenter study evaluating learning curves and competency in ERCP. Gastrointest Endosc 2014; 79: AB314-AB315
    CrossrefPubMedGoogle Scholar
  • 30 Scaffidi MA, Grover SC, Carnahan H. et al. Impact of experience on self-assessment accuracy of clinical colonoscopy competence. Gastrointest Endosc 2018; 87: 688-694.e2
    CrossrefPubMedGoogle Scholar
  • 31 Scaffidi MA, Khan R, Carnahan H. et al. Can pediatric endoscopists accurately assess their clinical competency? A comparison across skill levels. . J Pediatr Gastroenterol Nutr 2019; 68: 311-317
    Google Scholar
  • 32 Scaffidi M, Walsh C, Khan R. et al. Influence of video-based feedback on self-assessment accuracy of endoscopic skills: a randomized controlled trial. Endosc Int Open 2019; 07: E678-E684
    Article in Thieme ConnectPubMedGoogle Scholar
  • 33 Dunning D, Johnson K, Ehrlinger J. et al. Why people fail to recognize their own incompetence. Curr Dir Psychol Sci 2003; 12: 83-87
    CrossrefPubMedGoogle Scholar
  • 34 Zaki R, Bulgiba A, Ismail R. et al. Statistical methods used to test for agreement of medical instruments measuring continuous variables in method comparison studies: A systematic review. PLoS One 2012; 7: 1-7
    CrossrefPubMedGoogle Scholar
  • 35 Jeyalingam T, Walsh CM. Characterizing the development of colonoscopy competence using assessment data. Am J Gastroenterol 2020; 115: 214-215
    CrossrefPubMedGoogle Scholar