Controversies Surrounding Quality Measurement in Colon and Rectal Surgery

Abstract

Quality improvement in health care has become a major topic of discussion among health care providers, patients, insurance companies, and the government. National Surgical Quality Improvement Project (NSQIP), along with a multitude of other programs, exists in an attempt to create objective data that can be used to compare hospitals and providers against a national average. Studies have shown that despite good patient care and proper surgical technique, patients who undergo procedures such as colectomy have a higher incidence of surgical site infection (SSI) and other morbidities. Therefore, hospitals with a large volume of colon and rectal surgery cases are routinely identified as “high outliers” in these quality improvement programs. Programs, such as NSQIP, may not be the best way to measure quality in specific subspecialties such as colon and rectal surgery.

Keywords

NSQIP - quality improvement - colon and rectal surgery

CME Objectives: On completion of this article, the reader should be able to (1) identify the surgical subspecialty with the highest postoperative morbidity and (2) understand what the term “high outlier” means.

Numerous national entities and programs have emerged in recent years focusing upon quality measurement in health care. In general, these programs seek to improve health care quality through standardized, reproducible measurement of numerous variables involved in health care, with an eye toward controlling and reducing the costs of this care. The purpose of this article is to describe the major quality improvement initiatives currently being used in hospitals nationwide, their strengths and weaknesses, and the relevance and impact of these programs to the practicing colon and rectal surgeon. Additionally, resistance to implementation by hospitals and surgeons, as well as other controversies surrounding quality measurement in colon and rectal surgery, will be reviewed.

Background

The impetus to measure the quality of health care provided by a hospital can be traced to 1986 when Congress passed a law requiring all Veterans Affairs (VA) hospitals to report their risk-adjusted surgical outcomes and compare them to the national averages. This was done in response to reports of high morbidity and mortality associated with surgical care in the VA system. As no database existed at the national level or within the VA system that would allow for analysis of these averages, a measuring stick needed to be created. Thus, surgeons within the VA system created the National VA Surgical Risk Study (NVASRS). Specialized clinical nurse reviewers collected data points from the preoperative, intraoperative, and postoperative course. From these data, risk-adjusted morbidity and mortality could be calculated and the VA hospitals then ranked accordingly. As more VA hospitals began to contribute to NVASRS, the National Surgical Quality Improvement Project (NSQIP) was developed, which included 123 VA hospitals from which 88 surgical clinical nurse reviewers collected over 90 data points.[1] Armed with this new information, the VA hospitals saw a dramatic decrease in their morbidity and mortality related to surgery—up to 47% in some cases.[2] This significant improvement led to interest in NSQIP from many hospitals in the private sector. In 2001, the American College of Surgeons (ACS) started a pilot program known as ACS-NSQIP. Since 2004, ACS-NSQIP enrolled hospitals from both the private and public sectors. Currently, there are approximately 500 hospitals enrolled nationwide.

Today ACS-NSQIP is considered the standard for the measurement of quality in surgical care. Its outcomes are based on more than 130 variables, and they have been validated at a national level. While this is considered one of the strengths of this program, some argue that it is tedious to collect the data and can be even more tedious to analyze it. Since it was introduced on a national level, there have been multiple attempts to replicate the results with fewer variables, while taking into account some of the higher risk associated with certain types of surgeries. In recent years, ACS-NSQIP has been joined by multiple other programs and entities emphasizing quality and quality measurement in health care ([Fig. 1]).

Fig. 1 Multiple various entities and programs focusing on quality measurement in health care.

Overview of Quality Improvement Programs and Databases

Quality improvement in surgical care has become an increasingly important topic among health care professionals and consumers of health care, as well as those paying for health care, to include insurance providers, Medicare, hospital administrators, and politicians. Below are some examples of the additional quality measurement programs that are available to surgeons today.

The Surgical Care Improvement Project (SCIP) is a joint effort between a multitude of professional organizations that was founded on July 1, 2006, with the goal of reducing surgical complications.[3] Performance measures specific to SCIP include (1) dosing of antibiotics with 1 hour of incision, (2) appropriate antibiotic use, and (3) discontinuation of antibiotics within 24 hours of surgery. Hospital data are collected and published in the Joint Commission Annual Report.[4]

The National Association for Healthcare Quality (NAHQ) was founded in 1976. It is led by a board of eight health care professionals and has more than 40 state affiliates. The goal of the NAHQ is to promote patient safety through education, leadership development, and development of safe products for health care professionals. As their major contribution to quality improvement, NAHQ offers a certification test called the Certified Professional in Healthcare Quality to all levels of health care providers. It is meant to signify achievement in health care quality management. Their focus is on education and training.[5]

Members from the American Medical Association and various other health care organizations formed the National Patient Safety Foundation (NPSF) in 1997. The goal of the foundation is to decrease medical errors and thus improve patient safety. The NPSF strives to attain this goal through various educational opportunities, sponsoring Patient Safety Week each year, publication of resources for medical professionals, and research grants. They also focus on education and training.[6]

The National Quality Forum (NQF) was created in 1999 under the guidance of the President's Advisory Commission on Consumer Protection and Quality in the Health Care Industry. The NQF works with both public and private sectors on a national, state, regional, and local level to enhance health care and patient safety. In 2002, they published a list of Serious Reportable Events (aka “Never Events”), which are now tracked and documented in an effort to reduce serious medical error. The NQF has also recently published the National Standards for Hospital Care for Outcomes and Efficiency, which examined hospital readmission rates, surgical outcomes, improvement in quality of life, transition of care, and palliative care.[7]

Consumers Advancing Patient Safety (CAPS) was formed in the early 2000s as a nonprofit organization made up of patients, patient families, and health care providers. The goal of CAPS is to prevent bad outcomes through reporting of near misses/bad outcomes, education of patients and health care providers, and implementation of policy.[8]

The Agency for Healthcare Research and Quality (AHRQ) is one of twelve agencies within the Department of Health and Human Services and is one of the three organizational focuses of the department along with the National Institutes of Health and the Center for Disease Control. Initiatives of AHRQ include medical liability reform, Team STEPPS, and publication of the National Healthcare Quality Report and clinical guidelines for safe practice.[9]

The University HealthSystem Consortium (UHC) was formed in 1984. It is an alliance of 118 academic medical centers and their affiliated hospitals. The UHC offers performance improvement through databases used to collect information on patients, procedure, operational management, and financial management.[10]

This list is by no means complete, and it is beyond the scope of this article to review each and every different quality measurement program. What should be noted is that each program is attempting to find a novel, less time-consuming, and more cost-effective way to measure quality improvement as compared with NSQIP. Similar to procedures relating to pilonidal cystectomies, the multitude of novel ways to quantify quality improvement means that a consensus has yet to be reached. Creating an effective way to measure quality improvement that adequately reflects the risks factors of both patients and procedures has proven difficult. A 65-year-old diabetic male with hypertension and chronic kidney disease undergoing a laparoscopic cholecystectomy for symptomatic gallstones has a much different risk profile than the patient undergoing a low anterior resection for rectal cancer. The patient risk factors have not changed, but the risk profiles between the two procedures are very different.

Controversies Specific to Colon and Rectal Surgery

Programs such as NSQIP may misrepresent certain subspecialties, such as colon and rectal Surgery. Data collected by these programs are showing a trend of higher morbidity in some surgical subspecialties. This may be due to the nature of the patient's disease and the procedure they undergo. Studies have shown postoperative infection rates to be between 9.4 and 18%.[11] Another common problem is a higher rate of postoperative urinary tract infections (UTIs) in patients who have undergone pelvic dissections or other large, combined resections, as they have prolonged Foley catheter drainage.[12] Unfortunately, these higher rates of surgical site infections (SSIs) and UTIs are inherent risks that are largely nonmodifiable. Closer analysis of the NSQIP data collected for patients undergoing general surgery cases has revealed some interesting trends. When the current procedural terminology (CPT) codes for general surgery procedures were grouped into 36 clinically recognizable procedures and ranked according to their proportion of adverse events, 10 procedures accounted for 62% of the adverse events (see [Table 1]). Colectomy was number one on the list with a 24.3% complication rate and proctectomy, with or without colectomy, was number eight with a 2.9% complication rate. Together, they make up over a quarter of the total complications.[13]

Table 1 Top 10 procedures with adverse events

Procedure	Adverse events (in percentile)
Colectomy	24.3
Small intestine resection	7.7
Cholecystectomy	5.7
Ventral hernia repair	4.9
Pancreatectomy	4.4
Appendectomy	4.3
Bariatric procedures	3.4
Proctectomy	2.9
Lysis of adhesions	2
Liver resection	1.9

Source: Shilling PL, Dimick JB, Birkmeyer JD. Prioritizing quality improvement in general surgery. J Am Coll Surg. 2008;207(5):698–704.

With the advent and implementation of the Affordable Health Care Act, there has been an increasing push toward a pay-for-performance program for reimbursement of health care costs. The aim of this system is to improve the quality of health care throughout the nation by rewarding physicians and hospitals for meeting quality/performance standards and penalizing those that do not. The quality measures are divided into four broad categories: process, outcome, patient experience, and structure. Examples are provided in [Table 2].

Table 2 Quality measures in pay for performance

Quality measure	Example
Process	Did the patient receive appropriate DVT prophylaxis for colon cancer resection?
Outcome	Did the patient have to be readmitted for a postoperative infection?
Patient experience	Did the patients feel the treatment team communicated the daily plan to them in an effective manner?
Structure	Does this hospital use an electronic medical record?

Under the Affordable Health Care Act, hospitals will be rewarded for their performance on these quality measures and how well they improve over time. Currently, there are more than 40 pay-for-performance programs in the private sector, and the new laws regarding health care are instituting similar efforts for publically funded hospitals and Medicare reimbursement.[14]

With the implementation of pay for performance, the insurance companies or the government, in the case of Medicare and Medicaid, will need to have some way to measure quality to determine reimbursement. Colon and rectal surgery practices and hospitals with a high volume of colon and rectal surgery, such as university centers or tertiary referral centers, are at increased risk of being labeled “high outliers” by ACS-NSQIP, which could lead to decreased compensation. In 2012, Wick et al demonstrated a 14.3% SSI rate for colon and rectal surgery cases performed at the Cleveland Clinic, compared with 9.4% for other general surgery cases performed at the same location. They also found that many of the risk factors were nonmodifiable, such as body mass index and age.[15] It has been suggested that the current quality improvement programs do not accurately reflect risks associated with colon and rectal surgery. The current programs tend to focus on risk factors associated with the patient's comorbidities. There has been less focus on variables within the surgical procedure itself. A novel approach to calculating risk-adjusted quality assessments using relative value units (RVUs) and clinically meaningful CPTs was found to have better outcomes compared with the models ACS-NSQIP uses to predict morbidity and mortalities—specifically in colon and rectal surgery.[16]

It is evident that there is not an adequate program or system in place to measure quality in colon and rectal surgery. As the current law goes into effect, this will have to be taken into account when deciding on reimbursement for these types of procedures and their subsequent complications. As will be shown, even in the measurement of other surgical outcomes, the current programs and systems in place may not be providing valid data.

Outcomes of Quality Improvement Programs

All of these different quality improvement programs are attempting to reach the same goal: lower the rate of morbidity and mortality in surgical patients. A quick literature search of NSQIP will return many papers that illustrate the effectiveness of the program in both private and public hospitals. An early critique of the program was that the patient population in the VA system (largely older males with multiple comorbidities) was not concordant with the population of a hospital in the private sector. With the help of the ACS, the transition of the program from the VA system to the private sector has yielded reliable results. There have been large reductions in morbidity and mortality, especially in SSI and renal complications.[2] [17] These large reductions are impressive considering the diversity of patients between the VA and the private sector and speak highly of its effectiveness.

The SCIP program has had mixed results. Despite high compliance rates with SCIP-1 and SCIP-2 in one study, one study was unable to demonstrate a significant benefit related to SSI.[18] In 2009, Stulberg et al from Case Western Reserve University in Ohio were able to show an overall, but not significant, decrease in their postoperative infection rates over a 2-year period.[19] In 2013, Tillman et al from the Texas A&M University System compared outcomes 1 year prior and 1 year postimplementation of SCIP in their surgical population. SSI rates across the surgical subspecialty board did not significantly drop except for the colorectal subgroup.[20]

The validity of these national databases has been studied, as well. For a majority of programs, either a third party or a hired nurse collects the data with a wide variety in CPT codes for procedures and admissions. Discrepancies have been found between internal reviews and what is being reported in these large databases. In 2013, Hechenbleikner et al[21] from Johns Hopkins compared readmission rates of colorectal surgery patients using three different databases: NSQIP, UHC, and chart review. The chart review was used as the control to compare NSQIP to UHC. Interestingly, NSQIP identified less readmissions than UHC due to either a broader definition of readmission or the reviewers had missed them. However, the NSQIP database was able to identify readmissions at other hospitals that UHC could not “see” because UHC is used for internal purposes.[21] In 2011, Lawson et al[22] from University of California, Los Angeles, compared mortality rates as reported by NSQIP to the self-reported ACS Case Log System. They found significant differences in risk factors and outcomes data between the two systems, which were likely due to the method of data collection—third party versus self-reporting.[22] In 2008, Steinberg et al[23] from the Ohio State University compared 121 consecutive patient records in NSQIP that were matched to their UHC submissions. Overall, NSQIP underreported patient comorbidities and overreported complications related to surgery.[23] Finally, in 2013, Reinke et al[24] from the University of Pennsylvania compared anastomotic leak rates in NSQIP and UHC. NSQIP does not have a code for anastomotic leak, so the study used organ space surgical site infection. Using this method, they found that NSQIP had a higher specificity but lower sensitivity than UHC.[24] It also pointed out an important observation—the complexity of medicine does not lend itself to a simple data collection system that will be able to generate valid, useful data.

Are these measures working? The literature would argue yes. Hospitals that join NSQIP and other quality improvement programs are now forced to look at their complications and make adjustments to improve patient care. But are they valid? Can the results be trusted? It would seem that there are still large discrepancies between internal data collection and the national data collection. This is likely due to how the data are assimilated—one by the physicians and/or hospital staff and another by a hired nurse or a third party. One could also argue that it is due to the variation in CPT coding. While CPT codes are meant to be objective data points, they are largely subjective and based on the person who is reviewing the chart and entering the codes. This adds another wrinkle to an already complex problem, but it is one that will need to be dealt with, as quality improvement programs become more and more integrated into health care systems.

Resistance to Quality Improvement Programs

Enrollment into these programs is not yet a mandatory requirement for hospitals, but there are some barriers that need to be overcome, notably cost, staffing, and buy-in from the surgeons. The average cost of NSQIP implementation is around $35,000.[25] To a university hospital or a large private-practice hospital in an urban setting, this may not be a tremendous sum of money. However, to a smaller, rural hospital with limited resources, this can be a significant hurdle.

Buy-in from practitioners is a significant barrier. In 2005, Audet et al[26] looked at barriers to implementation of quality improvement programs by surveying practicing physicians in America concerning access to patient data, access to quality of care data, and involvement in quality improvement. They found that physicians in larger practices (i.e., more than 50 providers), salaried physicians, and those involved in patient care for more than 20 hours a week tended to have better access and be involved in quality improvement programs (either internally or externally).[26] This would argue that private practice physicians tend to be more resistant to quality improvement programs. However, an article from the New York Times in 2008 showed that the national trend is for physicians to join larger practices for a salaried position.[27] As this trend continues, it may be that more physicians have access to quality care data and improvement measures, and participation will increase. Yet, taking the next step and allowing full transparency of both hospital and individual surgeon outcomes to the public is a completely different issue.

Conclusion

The multitude of quality improvement programs and scoring systems is likely a reflection of the complexity, not only of the patients themselves but also of the surgical procedure and the health care system. Medicine, particularly surgery, is an art, and its complexities may occasionally lead to apparently random outcomes. While each program has its own individual strengths, there is yet to be an all-encompassing program that can balance the different, and often unpredictable, factors that go into the care of a surgical patient, and more specifically, the care of a colorectal surgical patient. Programs will likely continue to be proposed until there is one that not only the medical industry can agree on but individual subspecialties find fair and balanced.

• References

• 1 Russell MM. The National Surgical Quality Improvement Program: background and methodology. Semin Colon Rectal Surg 2012; 23 (4) 141-145
• 2 Rowell KS, Turrentine FE, Hutter MM, Khuri SF, Henderson WG. Use of national surgical quality improvement program data as a catalyst for quality improvement. J Am Coll Surg 2007; 204 (6) 1293-1300
• 3 Surgical Care Improvement Project website: http://www.jointcommission.org/surgical_care_improvement_project/ . Accessed June 20, 2013
• 4 Delaney CP. Quality improvement and efficiency of care: are acronyms really the way of the future?. Semin Colon Rectal Surg 2011; 22 (4) 192-196
• 5 National Association for Healthcare Quality website. http://www.nahq.org/ . Accessed June 30, 2013
• 6 National Patient Safety Foundation website. http://www.npsf.org/ . Access June 30, 2013
• 7 National Quality Forum website: http://www.qualityforum.org/Home.aspx . Access June 27, 2013
• 8 Consumers Advancing Patient Safety website: http://www.patientsafety.org/ . Accessed June 30, 2013
• 9 Agency for Healthcare Research and Quality website: http://www.ahrq.gov/index.html . Accessed June 30, 2013
• 10 University HealthSystems Consortium website: https://www.uhc.edu/1244.htm . Access June 29, 2013
• 11 Konishi T, Watanabe T, Kishimoto J, Nagawa H. Elective colon and rectal surgery differ in risk factors for wound infection: results of prospective surveillance. Ann Surg 2006; 244 (5) 758-763
• 12 Regenbogen SE, Read TE, Roberts PL, Marcello PW, Schoetz DJ, Ricciardi R. Urinary tract infection after colon and rectal resections: more common than predicted by risk-adjustment models. J Am Coll Surg 2011; 213 (6) 784-792
• 13 Schilling PL, Dimick JB, Birkmeyer JD. Prioritizing quality improvement in general surgery. J Am Coll Surg 2008; 207 (5) 698-704
• 14 James J. Health policy brief: pay-for-performance. Health Aff 2012; (October): 1-6
• 15 Wick EC, Vogel JD, Church JM, Remzi F, Fazio VW. Surgical site infections in a “high outlier” institution: are colorectal surgeons to blame?. Dis Colon Rectum 2009; 52 (3) 374-379
• 16 Raval MV, Cohen ME, Ingraham AM , et al. Improving American College of Surgeons National Surgical Quality Improvement Program risk adjustment: incorporation of a novel procedure risk score. J Am Coll Surg 2010; 211 (6) 715-723
• 17 Khuri SF, Henderson WG, Daley J , et al; Principal Investigators of the Patient Safety in Surgery Study. Successful implementation of the Department of Veterans Affairs' National Surgical Quality Improvement Program in the private sector: the Patient Safety in Surgery study. Ann Surg 2008; 248 (2) 329-336
• 18 Ingraham AM, Cohen ME, Bilimoria KY , et al. Association of surgical care improvement project infection-related process measure compliance with risk-adjusted outcomes: implications for quality measurement. J Am Coll Surg 2010; 211 (6) 705-714
• 19 Stulberg JJ, Delaney CP, Neuhauser DV, Aron DC, Fu P, Koroukian SM. Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA 2010; 303 (24) 2479-2485
• 20 Tillman M, Wehbe-Janek H, Hodges B, Smythe WR, Papaconstantinou HT. Surgical care improvement project and surgical site infections: can integration in the surgical safety checklist improve quality performance and clinical outcomes?. J Surg Res 2013; 184 (1) 150-156
• 21 Hechenbleikner EM, Makary MA, Samarov DV , et al. Hospital readmission by method of data collection. J Am Coll Surg 2013; 216 (6) 1150-1158
• 22 Lawson EH, Wang X, Cohen ME, Hall BL, Tanzman H, Ko CY. Morbidity and mortality after colorectal procedures: comparison of data from the American College of Surgeons case log system and the ACS NSQIP. J Am Coll Surg 2011; 212 (6) 1077-1085
• 23 Steinberg SM, Popa MR, Michalek JA, Bethel MJ, Ellison EC. Comparison of risk adjustment methodologies in surgical quality improvement. Surgery 2008; 144 (4) 662-667 , discussion 662–667
• 24 Reinke CE, Showalter S, Mahmoud NN, Kelz RR. Comparison of anastomotic leak rate after colorectal surgery using different databases. Dis Colon Rectum 2013; 56 (5) 638-644
• 25 Anderson J. ACS strives to help rural areas adopt NSQIP. ACS Surgery News 2012; 8 (5) 1 , 8
• 26 Audet AM, Doty MM, Shamasdin J, Schoenbaum SC. Measure, learn and improve: physicians' involvement in quality improvement. Health Aff 2005; 24 (3) 843-853
• 27 Harris G. More Doctors Giving Up Private Practices. New York Times. March 26, 2010:B1