Keywords
SCIP - NSQIP - SCOAP
CME Objectives: On completion of this article, the reader should be able to summarize the current
status of quality measurement protocols and practices in colon and rectal surgery.
Since the publication of the Institute of Medicine report To Err is Human: Building a Safer Health System in 1999 and Crossing the Quality Chasm: A New Health System for the 21st Century in 2001, detailing the rate of medical errors and their human costs, there has been
increasing scrutiny of the quality of medical care by the government, insurance payors,
patients, and health care providers themselves. There is growing demand for transparency
in the evaluation and reporting of health care quality in the public media. This quality
movement has also led to efforts to incentivize providers and reward good quality
care through “pay for performance” programs. Significant resources are being devoted
to the measurement and improvement of quality of care. But what defines good quality
care? How do we measure it? What should be measured? How is this information being
used? This article will discuss several national programs that attempt to define and
measure quality of care benchmarks to improve colorectal surgery outcomes for our
patients. The process and outcomes measures being evaluated by these programs (the
Surgical Care Improvement Project [SCIP], the National Surgical Quality Improvement
Program [NSQIP], and the Surgical Care and Outcomes Assessment Program [SCOAP]) and
their impact on patients, providers, and our current medical system will be reviewed.
Conceptual Model of Health Care Quality
Conceptual Model of Health Care Quality
The Donabedian model conceptualizes health care quality into three interrelated components
of structure, process, and outcome.[1] “Structure” refers to the context in which care is provided, and includes metrics
such as hospital and surgeon volume, nursing ratios, and availability of an electronic
medical record. These measures are often indicators of the amount of resources within
a system, and while they can impact the overall quality of care, they are often more
static and nonmutable. Examples of structural measures specific to colon and rectal
surgery include hospital and surgeon volume and surgeon board certification. “Process”
refers to the activities of care provision. Process measures assess whether a specific
intervention was performed for a defined patient population. “Outcomes” are the results
of providing care. Examples include mortality and morbidity, surgical site infections
(SSIs), disease-free survival, quality of life, and patient satisfaction. The underlying
idea is that the structure of the system impacts the process of providing care, which
then affects patient outcomes. Any one of these components can be modified to improve
overall quality of care.
The Surgical Care Improvement Project
The Surgical Care Improvement Project
An increased awareness of the frequency and cost of medical adverse events led to
the development of several national initiatives aimed at standardizing treatment protocols
to improve patient outcomes in the United States. The first such initiative was aimed
at reducing the incidence of SSIs and was called the Surgical Infection Prevention
Project (SIP). SIP was initiated by the Centers for Medicare and Medicaid Services
(CMS) and the Centers for Disease Control and Prevention (CDC) in 2002 and focused
on the improvement of process measures aimed at decreasing the incidence of SSIs, the third most commonly reported nosocomial
infection, and a major source of surgical morbidity and health care costs.[2] SSIs were targeted because they were believed to be largely preventable and the
goal of SIP was to reduce SSI rates for major, high-volume procedures. These included
five areas of care: elective colorectal resection, total hip/knee replacement, open
heart surgery, peripheral vascular procedures, and abdominal/vaginal hysterectomy.
To reduce the incidence of SSIs, SIP focused on process measures designed to promote
the proper use of antibiotics, which were being inappropriately administered in a
large percentage of patients. The proposed interventions relied on well-established,
evidence-based infection prevention measures that were shown to reduce SSIs.
In January 2003, the Medicare National Surgical Infection Prevention Project hosted
the SIP guideline writer's workgroup meeting, which consisted of surgical experts
as well as representatives from various surgical infection societies to review the
published data and establish guidelines for surgical antimicrobial prophylaxis.[3] Three process measures, which served as quality metrics, were identified. These
included (1) antibiotic administration within a 60-minute window before skin incision,
(2) proper choice of antibiotic, and (3) discontinuation of antibiotics within 24
hours of completion of the operation. All were based on strong medical evidence.
Compliance with these three measures was tested at the initiation of SIP using a national
sample of Medicare patients who underwent one of these five high-volume surgical procedures
in 2001. The study showed that compliance with these three process measures was low
(55.7% received antibiotics within 60 minutes, 92.6% were given the appropriate antibiotic,
and 59.3% received antibiotics beyond 24 hours) and confirmed that there was considerable
room for improvement.[4]
To test the impact of these measures on the prevention of SSIs, a national collaborative
was sponsored by CMS that included 56 hospitals and 43 Medicare Quality Improvement
organizations. Following the implementation of SIP, hospitals that participated in
this 1-year collaborative effort identified statistically significant reductions in
postoperative infection rates, reporting a mean 27% reduction in SSI rates.[5]
Given its early success, in 2005 SIP was expanded to become the SCIP. SCIP differed
from SIP by expanding on existing process measures. Similar to SIP, SCIP focused on
perioperative adverse events that were common, potentially preventable, and associated
with a high medical cost.
As with SIP, evidence-based practice guidelines were created to define best practices
aimed at improving specific perioperative outcomes. These guidelines were based on
the medical literature and are still reviewed and updated regularly. Process measures
related to SSI reduction were expanded from three to six criteria and included (1)
proper glucose control in cardiac surgery, (2) hair removal at the surgical site,
and (3) normothermia in patients undergoing colorectal resection. Performance measures
were expanded to also address cardiovascular (the use of perioperative β-blockers
to reduce the risk of cardiac events), venous thromboembolic (deep venous thrombosis
prophylaxis), and respiratory complications (reducing postoperative pneumonia). The
goal of SCIP was to reduce preventable surgical morbidity and mortality by 25% by
2010.
CMS published SCIP in 2006, and it was subsequently adopted by multiple organizations
including the Joint Commission on the Accreditation of Hospital Organizations, the
Agency for Healthcare Research and Quality, the Centers for Disease Control and Prevention,
the Veteran Affairs, the Institution for Healthcare Improvement, and the American
College of Surgeons (ACS). SCIP performance measures are now collected by more than
3,700 hospitals that submit abstracted data.
The Joint Commission also recommended that hospitals voluntarily report compliance
with these SCIP measures. In 2011, this became a mandatory component of Joint Commission
accreditation. Compliance with SCIP is now linked to pay for performance measures
designed to incentivize better quality and cost containment. Since 2011, SCIP compliance
has affected Medicare and Medicaid reimbursement rates.
National Surgical Quality Improvement Program
National Surgical Quality Improvement Program
NSQIP is a program that was formed in parallel with SCIP, and was originally developed
out of the Department of Veterans Affairs (VA) health system. In response to public
scrutiny over its operative mortality rates, and under mandate by Public Law 99–166
passed by the Congress in 1986, the VA conducted the National Veterans Affairs Surgical
Risk Study (NVASRS) to measure and report risk-adjusted surgical outcomes, and to compare VA outcomes to national averages. NVASRS was a
prospective cohort study conducted in 44 VA medical centers. The study was designed
to collect reliable and valid data on patient risk factors and postoperative outcomes.
Patients undergoing major noncardiac operations in eight surgical subspecialties under
general, spinal, and epidural anesthesia between 1991 and 1993 were included. Dedicated,
trained nurses were used to collect preoperative, intraoperative, and postoperative
data. Outcomes measured included 30-day all-cause mortality and 21 major morbidities.[6] Using these data, multivariable logistic regression risk adjustment models were
then created to compare outcomes across VA facilities. The risk-adjusted outcomes
were expressed as observed-to-expected (O:E) ratios.[7]
[8] The “expected” outcome is based on the preoperative risk profile of the patient
population. A ratio of less than 1 means that the institution is performing better
than expected, and vice versa for a ratio greater than 1.
Subsequently in 1994, the Surgical Risk Study was then rolled out to 123 VA facilities
as the VA NSQIP.[9] The VA NSQIP program provided individual facilities with site-specific outcome reports,
self-assessment tools, site visits, and best practices. Using the information derived
from this program, individual facilities were then able to institute targeted quality
improvement projects to yield measurable improvements in patient care.[10] By 2002, the VA NSQIP program was able to demonstrate a 27% decrease in 30-day postoperative
mortality, and 45% decrease in 30-day postoperative morbidity.[11] In 1999, a pilot study was then conducted in three academic institutions that demonstrated
the feasibility of applying NSQIP methodology in non-VA hospitals.[12] In 2001, the ACS received an Agency for Healthcare Research and Quality grant to
expand the NSQIP program to 18 private sector hospitals.
Currently, more than 400 hospitals in the United States participate in the ACS NSQIP.
Each hospital is required to pay a participation fee to the program, and provide a
surgeon champion and funding for a trained surgical clinical reviewer (SCR) for data
collection. Validity and reliability of the data are ensured by standardized training
of the SCRs by the ACS, site visits to ensure inter-rater reliability, and continued
training for data collectors. In large volume institutions, there is random sampling
of cases for data collection, and in smaller volume institutions all cases are included.
Patients aged 18 or older receiving all types of anesthesia in both inpatient and
outpatient settings are included. All data are reported to NSQIP via a web-based data
entry program. The validity of the 30-day outcomes is increased by direct communication
with surgical patients by phone or letter, and by public record death searches to
complete the 30-day follow-up. The program collects more than 130 preoperative, intraoperative,
and postoperative data variables, and reports on more than 20 risk-adjusted outcomes.
Outcomes analyzed include 30-day mortality, cardiac arrest requiring cardiopulmonary
resuscitation, myocardial infarction, deep vein thrombosis, pulmonary embolism, sepsis,
SSI (superficial, deep, and organ space), wound disruption, unplanned reintubation,
ventilator dependence > 48 hours, pneumonia, renal failure, and unplanned reoperation
and urinary tract infection, among other morbidities. The outcomes are reported to
individual participating hospitals as an “observed versus expected ratio (O:E ratio),”
and surgeon-level outcomes are also available.
NSQIP provides participating institutions with semi-annual outcomes reports to compare
risk-adjusted outcomes to other participating hospitals. There are site-specific reports
for preoperative risk factors, patient statistics, 30-day mortality, and morbidity,
and blinded surgeon-level outcomes are also reported. Participants also have access
to best practices guidelines, risk calculators to help inform patients about operative
risk, and a participant use data file for research and quality improvement programs.
For example, there is a morbidity and mortality risk calculator specifically for colorectal
surgery.[13] The colorectal risk calculator uses data from the pooled NSQIP database to help
inform patients' individual risk for various outcomes based on their age, gender,
and other preoperative factors. The data provided by NSQIP can be used to aid in quality
improvement activities by identifying areas in which an individual hospital is performing
below what is based on the risk-adjusted outcomes.
NSQIP is a validated nationwide outcomes-based database that only examines 30-day
postoperative patient outcomes. It is not intended to examine long-term outcomes,
and does not necessarily collect data on all outcomes of interest to the surgeon.
And although hospital-based processes can be targeted to improve outcomes in deficient
areas, it may be limited when it comes to targeting individual-based processes, such
as the technical aspects of surgical care.
Surgical Care and Outcomes Assessment Program
Surgical Care and Outcomes Assessment Program
“SCOAP is a developing Washington State initiative that was designed and implemented
by practicing surgeons, the leadership of the statewide ACS chapter, the Washington
State Hospital Association, and [Quality Improvement] organizations across the state
to track and reduce variability in abdominal surgical practice and outcomes.”[14] It is a physician and hospital collaborative that includes more than 50 hospitals
in Washington state. Although it has data variables in common with NSQIP and SCIP,
unlike NSQIP and SCIP, which focus on outcomes and process measures, respectively,
SCOAP includes both process and outcomes measures. Also unlike other quality measures
such as SCIP or other Joint Commission Core Measures, which are mandated and publicly
reported, SCOAP's nature as a quality improvement project allows the information gathered
under SCOAP to be protected by Washington State Statute.
SCOAP is a quality improvement project that aims to improve quality by reducing variation
in the provision of surgical care. The specific procedures that were initially included
are colon and rectal resection, appendectomy, and bariatric surgery. Because of the
surgeon driven and procedure-focused nature of the project, the data variables are
more nuanced and procedure specific. For example, the data collection form for colon
procedures includes information about prior colon or pelvic surgery, the specific
type of anastomosis performed (e.g., the use of Seamguard or other sealing devices
during stapled anastomosis), whether a leak test was performed and via what method
(scope, methylene blue, air insufflation, palpation/inspection, etc.), use of bowel
prep and oral antibiotics, information on neoadjuvant treatment, lymph node harvest,
resection margins, whether a total mesorectal excision was performed, and many others.[15]
As SCOAP includes both process and outcome measures, it allows performance improvement
investigators to query the impact of clinical processes on patient outcomes. For example,
Kwon et al looked at the impact of routine leak testing of left-sided colorectal anastomoses
on postoperative adverse events, and found a significant reduction in adverse events
in hospitals where leak testing is routinely preformed.[16] Based on this type of data, there is potential to reduce the variation in the use
of routine leak testing to decrease anastomotic complications and improve patient
outcomes.
What Has Been the Impact of These Programs?
What Has Been the Impact of These Programs?
SCIP, NSQIP, and SCOAP all provide infrastructure for quality improvement measures,
along with real data to assess the actual impact of these measures on patient outcomes.
These programs have the potential to improve the quality of medical care that is provided
at a lower cost, which is beneficial to patients, hospitals, and providers of health
care at all levels.
Despite their potential advantages, support of such programs requires considerable
resources to collect, report, and analyze the data appropriately. The accuracy of
the program relies on detailed and labor-intensive surveillance and reporting.
A distinction can be made between what we know to be high-quality care, what can be measured, what is currently routinely measured, and what should be measured to assess quality of care. For example, though each SCIP measure is based
on the best available medical evidence, there has been a lack of consistent data supporting
a strong correlation between SCIP compliance and improvement in the associated outcome
measures,[17] though the correlation is stronger in outlier institutions.[18] In fact, from 2001 to 2006, post-op infection rates increased in colon surgery,[19] as well as from 2006 to 2008.[18] Because the focus of SIP and SCIP has been on perioperative processes that can be
reliably gathered and analyzed, the selected process measures only partially impact
the surgical outcome being assessed.
While the validity of these programs remains debatable, the value lies in their focus
on quality improvement, self-reporting, and self-assessment. In an era of transparency
and accountability, SCOAP, SCIP, and NSQIP are the first steps in a national effort
to improve surgical outcomes and improve the quality of care.
Conclusion
To conclude, SCIP, NSQIP, and SCOAP have had a significant impact on surgical practices
nationwide. Providers now must comply with evidence-based practices that standardize
specific, evidence-based perioperative processes. Hospitals have adopted this quality
improvement culture and have dedicated considerable resources to insure SCIP compliance,
such as the hiring of SCIP nurses for data collection and review, and to track risk-adjusted
patient outcomes. Ultimately, these efforts should improve the overall quality of
medical care that we provide for our patients in the United States.
