Background and study aims: Simulators have potential value in providing objective evidence of technical skill
for procedures within medicine. The aim of this study was to determine face and construct
validity for the Olympus colonoscopy simulator and to establish which assessment measures
map to clinical benchmarks of expertise.
Patients and methods: Thirty-four participants were recruited: 10 novices with no prior colonoscopy experience,
13 intermediate (trainee) endoscopists with fewer than 1000 previous colonoscopies,
and 11 experienced endoscopists with more than 1000 previous colonoscopies. All participants
completed three standardized cases on the simulator and experts gave feedback regarding
the realism of the simulator. Forty metrics recorded automatically by the simulator
were analyzed for their ability to distinguish between the groups.
Results: The simulator discriminated participants by experience level for 22 different parameters.
Completion rates were lower for novices than for trainees and experts (37 % vs. 79 %
and 88 % respectively, P < 0.001) and both novices and trainees took significantly longer to reach all major
landmarks than the experts. Several technical aspects of competency were discriminatory;
pushing with an embedded tip (P = 0.03), correct use of the variable stiffness function (P = 0.004), number of sigmoid N-loops (P = 0.02); size of sigmoid N-loops (P = 0.01), and time to remove alpha loops (P = 0.004). Out of 10, experts rated the realism of movement at 6.4, force feedback
at 6.6, looping at 6.6, and loop resolution at 6.8.
Conclusions: The Olympus colonoscopy simulator has good face validity and excellent construct
validity. It provides an objective assessment of colonoscopic skill on multiple measures
and benchmarks have been set to allow its use as both a formative and a summative
assessment tool.
References
- 1
Marshall J B.
Technical proficiency of trainees performing colonoscopy: a learning curve.
Gastrointest Endosc.
1995;
42
287-291
- 2
Wigton R S, Nicolas J A, Blank L L.
Procedural skills of the general internist. A survey of 2500 physicians.
Ann Intern Med.
1989;
111
1023-1034
- 3
Tassios P S, Ladas S D, Grammenos I. et al .
Acquisition of competence in colonoscopy: the learning curve of trainees.
Endoscopy.
1999;
31
702-706
- 4
Wexner S D, Eisen G M, Simmang C.
Principles of privileging and credentialing for endoscopy and colonoscopy.
Surg Endosc.
2002;
16
367-369
- 5
Cass O, Freeman M L, Cohen J. et al .
Acquisition of competency in endoscopic skills (ACES) during training; a multicenter
study.
Gastrointest Endosc.
1996;
43
308
- 6
Cass O W.
Objective evaluation of competence: technical skills in gastrointestinal endoscopy.
Endoscopy.
1995;
27
86-89
- 7
Parry B R, Williams S M.
Competency and the colonoscopist: a learning curve.
Aust N Z J Surg.
1991;
61
419-422
- 8
Church J, Oakley J, Milsom J. et al .
Colonoscopy training: the need for patience (patients).
Aust N Z J Surg.
2002;
72
89-91
- 9
Bowles C J, Leicester R, Romaya C. et al .
A prospective study of colonoscopy practice in the UK today: are we adequately prepared
for national colorectal cancer screening tomorrow?.
Gut.
2004;
53
277-283
- 10
Sarker S K, Albrani T, Zaman A, Patel B.
Procedural performance in gastrointestinal endoscopy: an assessment and self-appraisal
tool.
Am J Surg.
2008;
196
450-455
- 11
Park J, MacRae H, Musselman L J. et al .
Randomized controlled trial of virtual reality simulator training: transfer to live
patients.
Am J Surg.
2007;
194
205-211
- 12
Grantcharov T P, Carstensen L, Schulze S.
Objective assessment of gastrointestinal endoscopy skills using a virtual reality
simulator.
JSLS.
2005;
9
130-133
- 13
Felsher J J, Olesevich M, Farres H. et al .
Validation of a flexible endoscopy simulator.
Am J Surg.
2005;
189
497-500
- 14
Sedlack R E, Kolars J C.
Validation of a computer-based colonoscopy simulator.
Gastrointest Endosc.
2003;
57
214-218
- 15
Mahmood T, Darzi A.
A study to validate the colonoscopy simulator.
Surg Endosc.
2003;
17
1583-1589
- 16
MacDonald J, Ketchum J, Williams R G, Rogers L Q.
A lay person versus a trained endoscopist: can the PreOp endoscopy simulator detect
a difference?.
Surg Endosc.
2003;
17
896-898
- 17
Datta V, Mandalia M, Mackay S, Darzi A.
The PreOp flexible sigmoidoscopy trainer. Validation and early evaluation of a virtual
reality based system.
Surg Endosc.
2002;
16
1459-1463
- 18
Moorthy K, Munz Y, Orchard T R. et al .
An innovative method for the assessment of skills in lower gastrointestinal endoscopy.
Surg Endosc.
2004;
18
1613-1619
- 19
Koch A D, Buzink S N, Heemskerk J. et al .
Expert and construct validity of the Simbionix GI Mentor II endoscopy simulator for
colonoscopy.
Surg Endosc.
2007;
22
158-162
- 20
Mahmood T, Darzi A.
The incidence of bowel perforations is positively related to the intensity of pain
experienced during simulated colonoscopy.
Gastrointest Endosc.
2004;
59
P125, S1554
- 21
Aabakken L, Adamsen S, Kruse A.
Performance of a colonoscopy simulator: experience from a hands-on endoscopy course.
Endoscopy.
2000;
32
911-913
- 22
Fitzgerald T N, Duffy A J, Bell R L. et al .
Computer-based endoscopy simulation: emerging roles in teaching and professional skills
assessment.
J Surg Educ.
2008;
65
229-235
- 23
Gerson L B.
Evidence-based assessment of endoscopic simulators for training.
Gastrointest Endosc Clin N Am.
2006;
16
489-509, vii-viii
- 24
Williams C B, Thomas-Gibson S.
Rational colonoscopy, realistic simulation, and accelerated teaching.
Gastrointest Endosc Clin N Am.
2006;
16
457-470
- 25
Koch A D, Haringsma J, Schoon E J. et al .
A second-generation virtual reality simulator for colonoscopy: validation and initial
experience.
Endoscopy.
2008;
40
735-738
- 26
Shah S G, Saunders B P, Brooker J C, Williams C B.
Magnetic imaging of colonoscopy: an audit of looping, accuracy and ancillary maneuvers.
Gastrointest Endosc.
2000;
52
1-8
- 27 NHS Modernisation Agency .Improving endoscopy services. Meeting the challenges. Leicester,
UK; NHS Modernisation Agency 2004
- 28 General Medical Council .Principles of good medical education and training. 2004
- 29 Joint Advisory Group for Gastrointestinal Endoscopy .Guidelines for the training,
appraisal and assessment of trainees in gastrointestinal endoscopy. British Society
of Gastroenterology, London; 2004. Available from: http://www.bsg.org.uk/pdf_word_docs/jag_recommendations_2004.pdf
- 30
Michelson J D, Manning L.
Competency assessment in simulation-based procedural education.
Am J Surg.
2008;
196
609-615
- 31
Sedlack R E, Baron T H, Downing S M, Schwartz A J.
Validation of a colonoscopy simulation model for skills assessment.
Am J Gastroenterol.
2007;
102
64-74
- 32
Shah S G, Brooker J C, Williams C B. et al .
Effect of magnetic endoscope imaging on colonoscopy performance: a randomised controlled
trial.
Lancet.
2000;
356
1718-1722
- 33
Hoff G, Bretthauer M, Dahler S. et al .
Improvement in caecal intubation rate and pain reduction by using 3-dimensional magnetic
imaging for unsedated colonoscopy: a randomized trial of patients referred for colonoscopy.
Scand J Gastroenterol.
2007;
42
885-889
- 34
Wehrmann K, Fruhmorgen P.
Evaluation of a new three-dimensional magnetic imaging system for use during colonoscopy.
Endoscopy.
2002;
34
905-908
- 35
Cheung H Y, Chung C C, Kwok S Y. et al .
Improvement in colonoscopy performance with adjunctive magnetic endoscope imaging:
a randomized controlled trial.
Endoscopy.
2006;
38
214-217
- 36
Shah S G, Pearson H J, Moss S. et al .
Magnetic endoscope imaging: a new technique for localizing colonic lesions.
Endoscopy.
2002;
34
900-904
- 37
Shah S G, Brooker J C, Thapar C. et al .
Effect of magnetic endoscope imaging on patient tolerance and sedation requirements
during colonoscopy: a randomized controlled trial.
Gastrointest Endosc.
2002;
55
832-837
A. V. HaycockMRCP
Wolfson Unit for Endoscopy
St. Mark’s Hospital
Imperial College London
London
UK
Fax: +44-20-84233588
Email: ahaycock@imperial.ac.uk
