Endoscopy 2014; 46(09): 735-744
DOI: 10.1055/s-0034-1365463
Original article
© Georg Thieme Verlag KG Stuttgart · New York

Development and initial validation of an endoscopic part-task training box

Christopher C. Thompson
1   Division of Gastroenterology, Brigham and Women’s Hospital, Boston, Massachusetts, USA
2   Harvard Medical School, Boston, Massachusetts, USA
,
Pichamol Jirapinyo
1   Division of Gastroenterology, Brigham and Women’s Hospital, Boston, Massachusetts, USA
2   Harvard Medical School, Boston, Massachusetts, USA
,
Nitin Kumar
1   Division of Gastroenterology, Brigham and Women’s Hospital, Boston, Massachusetts, USA
,
Amy Ou
1   Division of Gastroenterology, Brigham and Women’s Hospital, Boston, Massachusetts, USA
,
Andrew Camacho
1   Division of Gastroenterology, Brigham and Women’s Hospital, Boston, Massachusetts, USA
,
Balazs Lengyel
3   Department of Radiology, Brigham and Women’s Hospital, Boston, Massachusetts, USA
,
Michele B. Ryan
1   Division of Gastroenterology, Brigham and Women’s Hospital, Boston, Massachusetts, USA
› Author Affiliations
Further Information

Publication History

submitted 03 September 2013

accepted after revision 11 March 2014

Publication Date:
25 April 2014 (online)

Background and study aims: There is currently no objective and validated methodology available to assess the progress of endoscopy trainees or to determine when technical competence has been achieved. The aims of the current study were to develop an endoscopic part-task simulator and to assess scoring system validity.

Methods: Fundamental endoscopic skills were determined via kinematic analysis, literature review, and expert interviews. Simulator prototypes and scoring systems were developed to reflect these skills. Validity evidence for content, internal structure, and response process was evaluated.

Results: The final training box consisted of five modules (knob control, torque, retroflexion, polypectomy, and navigation and loop reduction). A total of 5 minutes were permitted per module with extra points for early completion. Content validity index (CVI)-realism was 0.88, CVI-relevance was 1.00, and CVI-representativeness was 0.88, giving a composite CVI of 0.92. Overall, 82 % of participants considered the simulator to be capable of differentiating between ability levels, and 93 % thought the simulator should be used to assess ability prior to performing procedures in patients. Inter-item assessment revealed correlations from 0.67 to 0.93, suggesting that tasks were sufficiently correlated to assess the same underlying construct, with each task remaining independent. Each module represented 16.0 % – 26.1 % of the total score, suggesting that no module contributed disproportionately to the composite score. Average box scores were 272.6 and 284.4 (P = 0.94) when performed sequentially, and average score for all participants with proctor 1 was 297.6 and 308.1 with proctor 2 (P = 0.94), suggesting reproducibility and minimal error associated with test administration.

Conclusion: A part-task training box and scoring system were developed to assess fundamental endoscopic skills, and validity evidence regarding content, internal structure, and response process was demonstrated.

 
  • References

  • 1 National Institutes of Health, U.S. Department of Health and Human Services. Opportunities and Challenges in Digestive Diseases Research: Recommendations of the National Commission on Digestive Diseases. NIH Publication No. 08-6514, March 2009
  • 2 Cherry DK, Hing E, Woodwell DA, Rechsteiner EA. National Ambulatory Medical Survey: 2006 Summary. 3. Hyattsville: National Center for Health Statistics; 2008
  • 3 Leffler DA, Kheraj R, Garud S et al. The incidence and cost of unexpected hospital use after scheduled outpatient endoscopy. Arch Intern Med 2010; 170: 1752-1757
  • 4 Mallery JS, Baron TH, Dominitz JA et al. Standards of Practice Committee. American Society for Gastrointestinal Endoscopy. Complications of ERCP. Gastrointest Endosc 2003; 57: 633-638
  • 5 Freeman ML, Nelson DB, Sherman S et al. Complications of endoscopic biliary sphincterotomy. N Engl J Med 1996; 335: 909-918
  • 6 Sedlack RE, Shami VM, Adler DG et al. Colonoscopy core curriculum. Gastrointest Endosc 2012; 76: 482-490
  • 7 Cohen J, Cohen SA, Vora KC et al. Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc 2006; 64: 361-368
  • 8 Sedlack RE, Kolars JC. Computer simulator training enhances the competence of gastroenterology fellows at colonoscopy results of a pilot study. Am J Gastroenterol 2004; 99: 33-37
  • 9 Gerson LB. Evidence-based assessment of endoscopic simulators for training. Gastrointest Endosc Clin N Am 2006; 16: 489-509 , vii-viii
  • 10 Haycock AV, Youd P, Bassett P et al. Simulator training improves practical skills in therapeutic GI endoscopy: results from a randomized, blinded, controlled study. Gastrointest Endosc 2009; 70: 835-845
  • 11 Shirai Y, Yoshida T, Shiraishi R et al. Prospective randomized study on the use of a computer-based endoscopic simulator for training in esophagogastroduodenoscopy. J Gastroenterol Hepatol 2008; 23: 1046-1050
  • 12 Joint Committee on Standards for Educational and Psychological Testing APA, American Educational Research Association and National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: AERA Publications; 1985
  • 13 Cureton EE. Validity. In: Lindquist EF, ed. Educational measurement. Washington, DC: American Council on Education; 1951: 621-694
  • 14 Joint Committee on Standards for educational and Psychological testing APA, American Educational Research Association and National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: 1999
  • 15 Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ 2003; 37: 830-837
  • 16 Grant JS, Davis LT. Selection and use of content experts in instrument development. Res Nurs Health 1997; 20: 269-274
  • 17 Spofford IS, Kumar N, Obstein KL et al. Deconstructing the colonoscopic examination: preliminary results comparing expert and novice kinematic profiles in screening colonoscopy. Gastrointest Endosc 2011; 73: AB415-416
  • 18 Obstein KL, Patil VD, Jayender J et al. Evaluation of colonoscopy technical skill levels by use of an objective kinematic-based system. Gastrointest Endosc 2011; 73: 315-321
  • 19 Sedlack RE. The Mayo Colonoscopy Skills Assessment Tool: validation of a unique instrument to assess colonoscopy skills in trainees. Gastrointest Endosc 2010; 72: 1125-1133
  • 20 Rex DK, Petrini JL, Baron TH et al. Quality indicators for colonoscopy. Gastrointest Endosc 2006; 63: 16-28
  • 21 Rex DK, Bond JH, Winawer S et al. Quality in the technical performance of colonoscopy and the continuous quality improvement process for colonoscopy: recommendations of the U.S. Multi-Society Task Force on Colorectal Cancer. Am J Gastroenterol 2002; 97: 1296-1308
  • 22 ASGE Committee on Training. Colonoscopy Core Curriculum, March 2001. Available from: http://www.asge.org/assets/0/71328/71340/4beb70579b0546c281fd2feb21d3b144.pdf Accessed: 20 July 2010
  • 23 Cohen J, Bosworth BP, Chak A et al. Preservation and incorporation of valuable endoscopic innovations (PIVI) on the use of endoscopy simulators for training and assessing skill. Gastrointest Endosc 2012; 76: 471-475
  • 24 Spier BJ, Benson M, Pfau PR et al. Colonoscopy training in gastroenterology fellowships: determining competence. Gastrointest Endosc 2010; 71: 319-324
  • 25 Sedlack RE. Training to competency in colonoscopy: assessing and defining competency standards. Gastrointest Endosc 2011; 74: 355-366
  • 26 Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006; 119: 166.e7-166.e16
  • 27 Cohen J, Thompson CC. The next generation of endoscopic simulation. Am J Gastroenterol 2013; 108: 1036-1039
  • 28 Downing SM, Haladyna TM. Validity threats: overcoming interference with proposed interpretations of assessment data. Med Educ 2004; 38: 327-333