Endoscopy 2018; 50(08): 770-778
DOI: 10.1055/a-0576-6667
Original article
© Georg Thieme Verlag KG Stuttgart · New York

Changes in scoring of Direct Observation of Procedural Skills (DOPS) forms and the impact on competence assessment

Keith Siau
 1  Department of Gastroenterology, Dudley Group Hospitals NHS Foundation Trust, Dudley, United Kingdom
 2  Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, United Kingdom
,
Paul Dunckley
 2  Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, United Kingdom
 3  Department of Gastroenterology, Gloucestershire Hospitals NHS Foundation Trust, Gloucester, United Kingdom
,
Roland Valori
 2  Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, United Kingdom
 3  Department of Gastroenterology, Gloucestershire Hospitals NHS Foundation Trust, Gloucester, United Kingdom
,
Mark Feeney
 2  Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, United Kingdom
 4  Department of Gastroenterology, Torbay and South Devon NHS Foundation Trust, Torquay, United Kingdom
,
Neil D. Hawkes
 2  Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, United Kingdom
 5  Department of Gastroenterology, Cwm Taf University Health Board, Llantrisant, United Kingdom
,
John T. Anderson
 2  Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, United Kingdom
 3  Department of Gastroenterology, Gloucestershire Hospitals NHS Foundation Trust, Gloucester, United Kingdom
,
Ian L. P. Beales
 2  Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, United Kingdom
 6  Department of Gastroenterology, Norfolk and Norwich University Hospital, Norwich, United Kingdom
,
Christopher Wells
 2  Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, United Kingdom
 7  Department of Gastroenterology, North Tees and Hartlepool NHS Foundation Trust, United Kingdom
,
Siwan Thomas-Gibson
 2  Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, United Kingdom
 8  Wolfson Unit for Endoscopy, St Mark’s Hospital, London, United Kingdom
 9  Imperial College London, London, United Kingdom
,
Gavin Johnson
 2  Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, United Kingdom
10  Department of Gastroenterology, University College London Hospitals NHS Foundation Trust, London, United Kingdom
,
on behalf of the Joint Advisory Group on Gastrointestinal Endoscopy (JAG)› Author Affiliations
Further Information

Publication History

submitted 06 November 2017

accepted after revision 23 January 2018

Publication Date:
03 April 2018 (eFirst)

Abstract

Background Direct Observation of Procedural Skills (DOPS) is an established competence assessment tool in endoscopy. In July 2016, the DOPS scoring format changed from a performance-based scale to a supervision-based scale. We aimed to evaluate the impact of changes to the DOPS scale format on the distribution of scores in novice trainees and on competence assessment.

Methods We performed a prospective, multicenter (n = 276), observational study of formative DOPS assessments in endoscopy trainees with ≤ 100 lifetime procedures. DOPS were submitted in the 6-months before July 2016 (old scale) and after (new scale) for gastroscopy (n = 2998), sigmoidoscopy (n = 1310), colonoscopy (n = 3280), and polypectomy (n = 631). Scores for old and new DOPS were aligned to a 4-point scale and compared.

Results 8219 DOPS (43 % new and 57 % old) submitted for 1300 trainees were analyzed. Compared with old DOPS, the use of the new DOPS was associated with greater utilization of the lowest score (2.4 % vs. 0.9 %; P < 0.001), broader range of scores, and a reduction in competent scores (60.8 % vs. 86.9 %; P < 0.001). The reduction in competent scores was evident on subgroup analysis across all procedure types (P < 0.001) and for each quartile of endoscopy experience. The new DOPS was superior in characterizing the endoscopy learning curve by demonstrating progression of competent scores across quartiles of procedural experience.

Conclusions Endoscopy assessors applied a greater range of scores using the new DOPS scale based on degree of supervision in two cohorts of trainees matched for experience. Our study provides construct validity evidence in support of the new scale format.

Supplementary Tables e1 – e4