Download PDF
Clin Colon Rectal Surg 2023; 36(04): 233-239
DOI: 10.1055/s-0043-1761421
Review Article

Measuring Quality at the Surgeon Level

Elias A. Chamely
1   Department of Surgery, University of Texas Health Science Center at Houston, McGovern Medical School, Houston, Texas
,
Jonah J. Stulberg
1   Department of Surgery, University of Texas Health Science Center at Houston, McGovern Medical School, Houston, Texas
› Author Affiliations
› Further Information
    Permissions and Reprints

Abstract

Patients expect high-quality surgical care and increasingly are looking for ways to assess the quality of the surgeon they are seeing, but quality measurement is often more complicated than one might expect. Measurement of individual surgeon quality in a manner that allows for comparison among surgeons is particularly difficult. While the concept of measuring individual surgeon quality has a long history, technology now allows for new and innovative ways to measure and achieve surgical excellence. However, some recent efforts to make surgeon-level quality data publicly available have highlighted the challenges of this work. Through this chapter, the reader will be introduced to a brief history of surgical quality measurement, learn about the current state of quality measurement, and get a glimpse into what the future holds.

Keywords

surgical quality - technical skills - video-based assessment - surgical quality


Publication History

Article published online:
24 March 2023

© 2023. Thieme. All rights reserved.

Thieme Medical Publishers, Inc.
333 Seventh Avenue, 18th Floor, New York, NY 10001, USA

 

  • References

  • 1 Chun J, Bafford AC. History and background of quality measurement. Clin Colon Rectal Surg 2014; 27 (01) 5-9
    Article in Thieme ConnectPubMedGoogle Scholar
  • 2 Roberts JS, Coale JG, Redman RR. A history of the Joint Commission on Accreditation of Hospitals. [published correction appears in JAMA 1987 Nov 20;258(19):2698] JAMA 1987; 258 (07) 936-940
    CrossrefPubMedGoogle Scholar
  • 3 Donabedian A. Evaluating the quality of medical care. 1966. Milbank Q 2005; 83 (04) 691-729
    CrossrefPubMedGoogle Scholar
  • 4 Hannan EL, Cozzens K, King III SB, Walford G, Shah NR. The New York State cardiac registries: history, contributions, limitations, and lessons for future efforts to assess and publicly report healthcare outcomes. J Am Coll Cardiol 2012; 59 (25) 2309-2316
    CrossrefPubMedGoogle Scholar
  • 5 Ferraris VA, Ferraris SP, Wehner PS, Setser ER. The dangers of gathering data: surgeon-specific outcomes revisited. Int J Angiol 2011; 20 (04) 223-228
    Article in Thieme ConnectPubMedGoogle Scholar
  • 6 Shih T, Cole AI, Al-Attar PM. et al. Reliability of surgeon-specific reporting of complications after colectomy. Ann Surg 2015; 261 (05) 920-925
    CrossrefPubMedGoogle Scholar
  • 7 Hall BL, Huffman KM, Hamilton BH. et al. Profiling individual surgeon performance using information from a high-quality clinical registry: opportunities and limitations. J Am Coll Surg 2015; 221 (05) 901-913
    CrossrefPubMedGoogle Scholar
  • 8 Quinn CM, Bilimoria KY, Chung JW, Ko CY, Cohen ME, Stulberg JJ. Creating individual surgeon performance assessments in a statewide hospital surgical quality improvement collaborative. J Am Coll Surg 2018; 227 (03) 303-312.e3
    CrossrefPubMedGoogle Scholar
  • 9 Khuri SF, Daley J, Henderson W. et al. Relation of surgical volume to outcome in eight common operations: results from the VA National Surgical Quality Improvement Program. Ann Surg 1999; 230 (03) 414-429 , discussion 429–432
    CrossrefPubMedGoogle Scholar
  • 10 Birkmeyer JD, Stukel TA, Siewers AE, Goodney PP, Wennberg DE, Lucas FL. Surgeon volume and operative mortality in the United States. N Engl J Med 2003; 349 (22) 2117-2127
    CrossrefPubMedGoogle Scholar
  • 11 Porter GA, Soskolne CL, Yakimets WW, Newman SC. Surgeon-related factors and outcome in rectal cancer. Ann Surg 1998; 227 (02) 157-167
    CrossrefPubMedGoogle Scholar
  • 12 Parry JM, Collins S, Mathers J, Scott NA, Woodman CB. Influence of volume of work on the outcome of treatment for patients with colorectal cancer. Br J Surg 1999; 86 (04) 475-481
    CrossrefPubMedGoogle Scholar
  • 13 Engelhardt KE, Bilimoria KY, Stulberg JJ. Surgeon scorecards: accurate or not?. Adv Surg 2018; 52 (01) 1-14
    CrossrefPubMedGoogle Scholar
  • 14 Friedberg MW, Pronovost PJ, Shahian DM. et al. A methodological critique of the ProPublica surgeon scorecard. Rand Health Q 2016; 5 (04) 1
    PubMedGoogle Scholar
  • 15 Ban KA, Cohen ME, Ko CY. et al. Evaluation of the ProPublica surgeon scorecard “adjusted complication rate” measure specifications. Ann Surg 2016; 264 (04) 566-574
    CrossrefPubMedGoogle Scholar
  • 16 Jaffe TA, Hasday SJ, Dimick JB. Power outage-inadequate surgeon performance measures leave patients in the dark. JAMA Surg 2016; 151 (07) 599-600
    CrossrefPubMedGoogle Scholar
  • 17 Regenbogen SE, Read TE, Roberts PL, Marcello PW, Schoetz DJ, Ricciardi R. Urinary tract infection after colon and rectal resections: more common than predicted by risk-adjustment models. J Am Coll Surg 2011; 213 (06) 784-792
    CrossrefPubMedGoogle Scholar
  • 18 Burack JH, Impellizzeri P, Homel P, Cunningham Jr JN. Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons. Ann Thorac Surg 1999; 68 (04) 1195-1200 , discussion 1201–1202
    CrossrefPubMedGoogle Scholar
  • 19 Burns EM, Pettengell C, Athanasiou T, Darzi A. Understanding the strengths and weaknesses of public reporting of surgeon-specific outcome data. Health Aff (Millwood) 2016; 35 (03) 415-421
    CrossrefPubMedGoogle Scholar
  • 20 Birkmeyer JD, Finks JF, O'Reilly A. et al; Michigan Bariatric Surgery Collaborative. Surgical skill and complication rates after bariatric surgery. N Engl J Med 2013; 369 (15) 1434-1442
    CrossrefPubMedGoogle Scholar
  • 21 Stulberg JJ, Huang R, Kreutzer L. et al. Association between surgeon technical skills and patient outcomes. [published correction appears in JAMA Surg. 2020 Oct 1;155(10):1002] [published correction appears in JAMA Surg. 2021 Jul 1;156(7):694] JAMA Surg 2020; 155 (10) 960-968
    CrossrefPubMedGoogle Scholar
  • 22 Mackenzie H, Ni M, Miskovic D. et al. Clinical validity of consultant technical skills assessment in the English National Training Programme for laparoscopic colorectal surgery. Br J Surg 2015; 102 (08) 991-997
    CrossrefPubMedGoogle Scholar
  • 23 Bilgic E, Valanci-Aroesty S, Fried GM. Video assessment of surgeons and surgery. Adv Surg 2020; 54: 205-214
    CrossrefPubMedGoogle Scholar
  • 24 Merkow RP, Massarweh NN. Looking beyond perioperative morbidity and mortality as measures of surgical quality. Ann Surg 2022; 275 (02) e281-e283
    CrossrefPubMedGoogle Scholar
  • 25 Fecso AB, Szasz P, Kerezov G, Grantcharov TP. The effect of technical performance on patient outcomes in surgery: a systematic review. Ann Surg 2017; 265 (03) 492-501
    CrossrefPubMedGoogle Scholar
  • 26 Martin JA, Regehr G, Reznick R. et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997; 84 (02) 273-278
    CrossrefPubMedGoogle Scholar
  • 27 Reznick RK, MacRae H. Teaching surgical skills—changes in the wind. N Engl J Med 2006; 355 (25) 2664-2669
    CrossrefPubMedGoogle Scholar
  • 28 Brajcich BC, Stulberg JJ, Palis BE. et al. Association between surgical technical skill and long-term survival for colon cancer. JAMA Oncol 2021; 7 (01) 127-129
    CrossrefPubMedGoogle Scholar
  • 29 Wood TC, Raison N, Haldar S. et al. Training tools for nontechnical skills for surgeons—a systematic review. J Surg Educ 2017; 74 (04) 548-578
    CrossrefPubMedGoogle Scholar
  • 30 Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hospitals. Surgery 2003; 133 (06) 614-621
    CrossrefPubMedGoogle Scholar
  • 31 Baker DP, Beaubien JM, Holtzman AK. DoD Medical Team Training Programs: An Independent Case Study Analysis. Washington, DC: American Institutes for Research; 2003
    Google Scholar
  • 32 Sinyard RD, Rentas CM, Gunn EG. et al. Managing a team in the operating room: The science of teamwork and non-technical skills for surgeons. Curr Probl Surg 2022; 59 (07) 101172
    CrossrefPubMedGoogle Scholar
  • 33 Noonan M, Olaussen A, Mathew J, Mitra B, Smit V, Fitzgerald M. What is the clinical evidence supporting trauma team training (TTT): a systematic review and meta-analysis. Medicina (Kaunas) 2019; 55 (09) 551
    CrossrefPubMedGoogle Scholar
  • 34 Tan SB, Pena G, Altree M, Maddern GJ. Multidisciplinary team simulation for the operating theatre: a review of the literature. ANZ J Surg 2014; 84 (7-8): 515-522
    CrossrefPubMedGoogle Scholar
  • 35 Forse RA, Bramble JD, McQuillan R. Team training can improve operating room performance. Surgery 2011; 150 (04) 771-778
    CrossrefPubMedGoogle Scholar
  • 36 Chheang V, Fischer V, Buggenhagen H. et al. Toward interprofessional team training for surgeons and anesthesiologists using virtual reality. Int J CARS 2020; 15 (12) 2109-2118
    CrossrefPubMedGoogle Scholar
  • 37 Yule S, Flin R, Paterson-Brown S, Maran N, Rowley D. Development of a rating system for surgeons' non-technical skills. Med Educ 2006; 40 (11) 1098-1104
    CrossrefPubMedGoogle Scholar
  • 38 Pradarelli JC, Gupta A, Lipsitz S. et al. Assessment of the Non-Technical Skills for Surgeons (NOTSS) framework in the USA. Br J Surg 2020; 107 (09) 1137-1144
    CrossrefPubMedGoogle Scholar
  • 39 Neily J, Mills PD, Young-Xu Y. et al. Association between implementation of a medical team training program and surgical mortality. JAMA 2010; 304 (15) 1693-1700
    CrossrefPubMedGoogle Scholar
  • 40 Abahuje E, Johnson J, Halverson A, Stulberg JJ. Intraoperative Assessment of Non-Technical Skills for Surgeons (NOTSS) and qualitative description of their effects on intraoperative performance. J Surg Educ 2022; 79 (05) 1237-1245
    CrossrefPubMedGoogle Scholar
  • 41 Louridas M, Sachdeva AK, Yuen A, Blair P, MacRae H. Coaching in surgical education: a systematic review. Ann Surg 2022; 275 (01) 80-84
    CrossrefPubMedGoogle Scholar
  • 42 Pradarelli JC, Hu YY, Dimick JB, Greenberg CC. The value of surgical coaching beyond training. Adv Surg 2020; 54: 31-47
    CrossrefPubMedGoogle Scholar
  • 43 Kreutzer L, Hu YY, Stulberg J, Greenberg CC, Bilimoria KY, Johnson JK. Formative evaluation of a peer video-based coaching initiative. J Surg Res 2021; 257: 169-177
    CrossrefPubMedGoogle Scholar
  • 44 Rivard SJ, Varlamos C, Hibbard CE. et al. A national qualitative study of surgical coaching: opportunities and barriers for colorectal surgeons. Surgery 2022; 172 (02) 546-551
    CrossrefPubMedGoogle Scholar