Development of a Video Assessment Scoring Method to Determine the Accuracy of Endoscopist Performance at Screening Flexible Sigmoidoscopy

Background and Aims: Variation in the adenoma detection rate (ADR) at flexible sigmoidoscopy screening has been shown to be due to variation in endoscopist performance. There are no objective methods for scoring an endoscopist’s performance reliably, and the aim of this study was to develop a valid and reliable objective scoring method using video footage of screening flexible sigmoidoscopies.
Methods: In a series of five experiments, experienced endoscopists (the scorers) independently scored a sample (n = 43) of the 40 000 flexible sigmoidoscopy extubations recorded as part of the United Kingdom Flexible Sigmoidoscopy Screening Trial (UK FSST). The scoring system, the parameters scored, and their definitions evolved over the course of the five experiments. The initial visual analogue score (range 0 - 100) used in the first two experiments evolved into a five-point score that ranged from 1 (E, poor) to 5 (A, excellent) in the last three experiments. The final parameters scored were: time spent viewing the mucosa, re-examination of poorly viewed areas, suctioning of fluid pools, distension of the lumen, lower rectal examination, and overall quality of the examination. The first four experiments scored one individual case per endoscopist; in experiment 5, an overall score was awarded for five cases performed by each endoscopist being assessed.
Results: Scoring five cases examined by an individual endoscopist using the A - E grading system was the most reliable method (interclass correlation coefficient 0.89). Cluster analysis demonstrated that the endoscopists in the high-scoring ADR group (ADR 14.7 - 15.9 %) could be differentiated from those in the intermediate- and low-scoring ADR groups (ADR 8.6 - 12.6 %).
Conclusions: An objective scoring system for assessing the accuracy of performance at screening flexible sigmoidoscopy, based on video footage, is described. Endoscopists who might benefit from further training can be identified using this method.

References

• 1 Atkin W S, Cuzick J, Northover J M, Whynes D K. Prevention of colorectal cancer by once-only sigmoidoscopy.  Lancet. 1993;  341 736-740
• 2 UK Flexible Sigmoidoscopy Screening Trial Investigators . Single flexible sigmoidoscopy screening to prevent colorectal cancer: baseline findings of a UK multicentre randomised trial.  Lancet. 2002;  359 1291-1300
• 3 Atkin W, Rogers P, Cardwell C. et al . Wide variation in adenoma detection rates at screening flexible sigmoidoscopy.  Gastroenterology. 2004;  126 1247-1256
• 4 Bretthauer M, Skovlund E, Grotmol T. et al . Inter-endoscopist variation in polyp and neoplasia pick-up rates in flexible sigmoidoscopy screening for colorectal cancer.  Scand J Gastroenterol. 2003;  38 1268-1274
• 5 Winawer S J, Zauber A G, Ho M N. et al . Prevention of colorectal cancer by colonoscopic polypectomy.  The National Polyp Study Workgroup N Engl J Med. 1993;  329 1977-1981
• 6 Rex D K, Bond J H, Winawer S. et al . Quality in the technical performance of colonoscopy and the continuous quality improvement process for colonoscopy: recommendations of the U.S. Multi-Society Task Force on Colorectal Cancer.  Am J Gastroenterol. 2002;  97 1296-1308
• 7 Bowles C J, Leicester R, Romaya C. et al . A prospective study of colonoscopy practice in the UK today: are we adequately prepared for national colorectal cancer screening tomorrow?.  Gut. 2004;  53 277-283
• 8 Joint Advisory Group on Gastrointestinal Endoscopy .Guidelines for the training, appraisal and assessment of trainees in gastrointestinal endoscopy. 2004 [accessed 2005 Jul 5]. Available from: URL: http//www.thejag.org.uk/JAG_2004.pdf. 
• 9 Rex D K. Colonoscopic withdrawal technique is associated with adenoma miss rates.  Gastrointest Endosc. 2000;  51 33-36
• 10 Ramakrishnan K, Scheid D C. Predictors of incomplete flexible sigmoidoscopy.  J Am Board Fam Pract. 2003;  16 478-484
• 11 Walter L C, de Garmo P, Covinsky K E. Association of older age and female sex with inadequate reach of screening flexible sigmoidoscopy.  Am J Med. 2004;  116 174-178
• 12 Eloubeidi M A, Wallace M B, Desmond R, Farraye F A. Female gender and other factors predictive of a limited screening flexible sigmoidoscopy examination for colorectal cancer.  Am J Gastroenterol. 2003;  98 1634-1639
• 13 Armitage P, Berry G, Matthews J NS. Statistical methods in medical research. 4th edn. Oxford; Blackwell Science 2002
• 14 Hawes R, Lehman G A, Hast J. et al . Training resident physicians in fiberoptic sigmoidoscopy: how many supervised examinations are required to achieve competence?.  Am J Med. 1986;  80 465-470
• 15 Cass O W. Training to competence in gastrointestinal endoscopy: a plea for continuous measuring of objective end points.  Endoscopy. 1999;  31 751-754
• 16 Lal S K, Barrison A, Heeren T, Schroy P C III. A national survey of flexible sigmoidoscopy training in primary care graduate and postgraduate education programs.  Am J Gastroenterol. 2004;  99 830-836
• 17 Weissman G S, Winawer S J, Baldwin M P. et al . Multicenter evaluation of training of non-endoscopists in 30-cm flexible sigmoidoscopy.  CA Cancer J Clin. 1987;  37 26-30
• 18 Health and Public Policy Committee, American College of Physicians . Clinical competence in colonoscopy.  Ann Intern Med. 1987;  107 772-774
• 19 Saad J A, Pirie P, Sprafka J M. Relationship between flexible sigmoidoscopy training during residency and subsequent sigmoidoscopy performance in practice.  Fam Med. 1994;  26 250-253
• 20 ASGE . Principles of training in gastrointestinal endoscopy. From the ASGE. American Society for Gastrointestinal Endoscopy.  Gastrointest Endosc. 1999;  49 845-853
• 21 Maule W F. Screening for colorectal cancer by nurse endoscopists.  N Engl J Med. 1994;  330 183-187
• 22 Ashley O S, Nadel M, Ransohoff D F. Achieving quality in flexible sigmoidoscopy screening for colorectal cancer.  Am J Med. 2001;  111 643-653
• 23 Proctor D D, Price J, Dunn K A. et al . Prospective evaluation of a teaching model to determine competency in performing flexible sigmoidoscopies.  Am J Gastroenterol. 1998;  93 1217-1221
• 24 Holman J R, Marshall R C, Jordan B, Vogelman L. Technical competency in flexible sigmoidoscopy.  J Am Board Fam Pract. 2001;  14 424-429
• 25 Painter J, Saunders D B, Bell G D. et al . Depth of insertion at flexible sigmoidoscopy: implications for colorectal cancer screening and instrument design.  Endoscopy. 1999;  31 227-231
• 26 Cutler A F, Pop A. Fifteen years later: colonoscopic retroflexion revisited.  Am J Gastroenterol. 1999;  94 1537-1538
• 27 Shinya H, Cwern M, Karlstadt R. Colonoscopy: technique and training methods.  Surg Clin North Am. 1982;  62 869-876
• 28 Cotton P, Williams C B. Practical gastrointestinal endoscopy. 5th edn. Oxford; Blackwell Scientific 2003
• 29 Stranc M F, McDiarmid J G, Stranc L C. Video assessment of surgical technique.  Br J Plast Surg. 1991;  44 65-68
• 30 Scott D J, Rege R V, Bergen P C. et al . Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation.  J Laparoendosc Adv Surg Tech A. 2000;  10 183-190
• 31 Eubanks T R, Clements R H, Pohl D. et al . An objective scoring system for laparoscopic cholecystectomy.  J Am Coll Surg. 1999;  189 566-574
• 32 Miller G E. The assessment of clinical skills/competence/performance.  Acad Med. 1990;  65 (9 Suppl) S63-S67