Keywords
clinical decision support - community health centers - cervical cancer screening -
mixed methods
Background and Significance
Background and Significance
Widespread implementation of routine Papanicolaou (Pap) testing yielded decreases
in cervical cancer (CC) incidence and mortality.[1]
[2] Yet despite the proven benefits of CC screening, in 2016 only two-thirds of 30 to
65-year-old women in the United States. were up-to-date on such screening.[3] Furthermore, 13% of CC deaths are attributed to inadequate follow-up on positive
screening results,[4] but 47% of patients with a CC diagnosis had a >6 month interval between the test
and receipt of indicated follow-up care.[4] Patients served by community health centers, many of whom experience socioeconomic
barriers to acting on care plans (e.g., following up on recommended care), are more
likely than those in other settings to experience such delays.[5]
The use of clinical decision support (CDS) tools in electronic health records (EHRs)
might help improve rates of such follow-up care. Such tools have been shown to support
clinical teams' adherence to care guidelines in some settings,[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17] including the provision of CC follow-up care.[9]
[13]
[18] None of these studies evaluating CDS for CC follow-up assessed tool adoption, but
instead estimated or demonstrated the benefit of these tools in improving CC follow-up
care.[9]
[13]
[18]
Despite these potential benefits, CDS adoption in primary care is suboptimal.[19] Prior research found that when users participate in the development and testing
of new/updated CDS tools they are more likely to adopt the tool in practice. However,
such engagement is rare. Far more common is that CDS tools are simply activated in
EHRs without users receiving information about them.[20] Research is needed about how such widely disseminated tools are adopted and the
barriers to their use, especially in care settings serving socioeconomically disadvantaged
patient populations.[21]
One CDS tool (see [Supplementary Appendix 1], available in the online version) available in the Epic EHR (Epic Systems, Verona,
Wisconsin, United States) is designed to support documenting patients' CC screening
results and tracking their receipt of appropriate follow-up care, and thus to increase
receipt of such care. This CC cytology SmartForm (referred to hereafter as the “CC-tool”)
facilitates documenting screening results received via FAX/PDF, and ordered follow-up
care subsequent to an abnormal Pap result, and panel management (generating lists
of patients due for screening or follow-up care).
Table 1
Top five provider type users
|
Provider types
|
Tool uses
|
Providers[a]
|
% all tool uses
|
Monthly use rate[b]
|
|
Registered nurse
|
3,177
|
28
|
29%
|
6.86
|
|
Nurse practitioner
|
2,239
|
115
|
21%
|
1.32
|
|
Physician
|
1,954
|
138
|
18%
|
1.02
|
|
Physician assistant
|
1,326
|
28
|
12%
|
2.98
|
|
Medical assistant
|
628
|
22
|
6%
|
2.37
|
Note: Total number of tool touches in the study period = 10,896.
a Number of providers who used the tool by type.
b Average monthly use over the study period, calculated as the sum of tool uses by
provider type divided by the sum of months providers had made use of the tool—from
their first use to the end of the study period.
OCHIN, Inc., is a health information technology provider that modifies Epic to meet
community health centers' needs. In October 2018, OCHIN (not an acronym) modified
the CC-tool described above to reflect 2012 American Society of Colposcopy and Cervical
Pathology guidelines. These guidelines provide recommendations on CC screening frequency
and follow-up test for abnormal CC results. It was also modified to let users enter
and update related data until follow-up care for a given CC screening result was completed,
and to ensure that all added information automatically informed related “Health Maintenance”
alerts. While Health Maintenance tools are standard in the Epic EHR, they are only
as accurate as the data that feed their algorithms; these CC-tool changes were meant
to improve the accuracy of the CC results used by the Health Maintenance tools, and
are not standard in the Epic EHR. The modified version of the CC-tool was implemented
in OCHIN community health centers in November 2018, and information about the tool
including links to a web-based self-directed learning system for details about its
use was emailed to all EHR users. In 2020, Healthy People 2030 goals for CC screening
were released changing the target for CC screening prevalence from 80.5 to 84.3%.
Considering that the average prevalence for CC screening in community health centers
is 50%,[22] this target will be challenging to reach for these clinics. Moreover, the new American
Society of Colposcopy and Cervical Pathology guidelines were released in 2020 changing
the recommendations for screening prevalence and abnormal results follow-up. In response,
the 2018 CC-tool was further refined both to reflect these updated guidelines and
to integrate input on the tool obtained through the user-centered design process described
below.
Objective
Little is known about how CDS tools are adopted in community clinics and the barriers
to their use in this setting. This study describes the frequency of and factors associated
with use of the 2018 CC-tool and barriers and facilitators to its use.
Methods
Setting
This study uses a sequential mixed method design. The sample included community health
centers within the OCHIN practice-based research network, which share an Epic EHR
hosted by OCHIN, and provide care regardless of patients' ability to pay. OCHIN EHR
data were extracted from the Accelerating Data Value Across a National Community Health
Center Network (ADVANCE) Clinical Research Network, a member of PCORnet, and supplemented
with CC-tool use data. Analyses included 480 clinics, located across 18 states, at
which ≥1 Pap test was ordered after February 1, 2020. This criterion ensured that
included clinics were active after March 2020, performing cancer screenings, and able
to participate in interviews and the tool refinement process.
Qualitative Analysis
Eight community health centers participated in user-centered design[23]
[24] sessions and semi-structured interviews in 4 months. Community health centers recruited
to ensure variation[25] in CC-tool utilization among clinic staff involved in CC screening. Participants
were from community health centers in diverse geographical locations (i.e., rural, urban, suburban), and had varying care team roles (e.g., medical doctor,
midwife, medical assistant, nurse practitioner, panel manager). In qualitative data
collection activities in April to August 2021, we observed user-centered design sessions
with nine clinic staff representing six clinics; observers took detailed field notes
at these sessions. Participants with diverse experience using the CC-tool volunteered
to participate in the user-centered design process, by providing feedback on how to
improve the tool's support of CC screening and follow-up of abnormal results; this
was one source of data on EHR users' knowledge and perceptions of this tool. In addition,
we conducted semi-structured interviews with 8 staff from 2 community health centers.
Qualitative data were analyzed thematically, informed by constructs from the Consolidated
Framework for Implementation Research.[26]
[27]
[28] Interviews were audio-recorded, professionally transcribed, and uploaded with the
field notes to NVivo (QSR International Pty Ltd. (2020) NVivo (Release 1.0) for analysis.
Quantitative Analysis
Adoption of the 2018 version of the CC-tool from November 1, 2018 to December 31,
2020 is described. To differentiate those who used the CC-tool rarely versus regularly,
we assessed both any instance of CC-tool use as a binary variable (any vs. never used
for entry of screening results or follow-up plan) and the number of tool uses per
month during the study period, at the clinic level. Patient panel demographics (race/ethnicity,
age, federal poverty level [FPL], rural/urban location) and insurance status were
obtained from ADVANCE data. CC screening dates and abnormal Pap results were extracted
from order and laboratory results and health maintenance fields in the EHR.
First, CC-tool use was summarized using descriptive statistics. Logistic regression
was then used to model the binary outcome of any CC-tool use (use vs. never) and negative
binomial regression to model the outcome of monthly tool use rates. Each model considered
two sets of covariates: (1) models with practice-level characteristics only and (2)
models with practice-level characteristics plus rates of CC screening and rates of
abnormal Pap results as covariates.
Results
During the study period, 41% of eligible clinics used the CC-tool at least once. It
was used most frequently by registered nurses, nurse practitioners, and medical doctors
([Table 1]). There were few differences in the characteristics of clinics where the CC-tool
was ever versus never used ([Table 2]). Notably, 47% of clinics with CC-tool use were in the top tertile of CC screening
rates versus 22% of clinics where the tool was never used; 77% of clinics with any
tool use were in the highest and middle tertiles for abnormal Pap rates compared to
59% among nonusers. Both logistic and negative binomial models showed that clinics
where the CC-tool was used more often had significantly more encounters, a greater
proportion of patients of with FPL ≥138%, fewer pediatric patients, and higher rates
of up-to-date CC screening and of abnormal results ([Table 3]).
Table 2
Clinic characteristics of cervical cancer screening tool users and nonusers
|
User (n = 214)
|
Nonuser (n = 266)
|
|
Panel mix
|
|
Ambulatory encounters, mean (SD)
|
32,131 (37,895)
|
11,065 (14,754)
|
|
Percent White, mean (SD)
|
62.1 (26.9)
|
61.3 (26.5)
|
|
Percent pediatric patient, mean (SD)
|
12.6 (12.9)
|
18.0 (26.6)
|
|
Percent Hispanic, mean (SD)
|
31.4 (26.8)
|
25.0 (25.6)
|
|
Percent Medicaid, mean (SD)
|
51.9 (19.0)
|
50.2 (20.8)
|
|
Percent uninsured, mean (SD)
|
14.2 (14.0)
|
17.1 (15.6)
|
|
Percent FPL <138%, mean (SD)
|
58.7 (27.0)
|
63.2 (27.8)
|
|
Percent rural, mean (SD)
|
7.0 (3.3)
|
19.0 (7.1)
|
|
Implementation impact
|
|
Rates of CC screening, n (%)
|
|
Highest tertile (>57.3%)
|
100 (46.7)
|
60 (22.6)
|
|
Middle tertile (39.7–57.3%)
|
81 (37.9)
|
79 (29.7)
|
|
Lowest tertile (<39.7%)
|
33 (15.4)
|
127 (47.7)
|
|
Rates of abnormal Pap results, n (%)
|
|
Highest tertile (>17.8%)
|
79 (36.9)
|
81 (30.5)
|
|
Middle tertile (12.6–17.8%)
|
85 (39.7)
|
75 (28.2)
|
|
Lowest tertile (<12.6)
|
50 (23.4)
|
110 (41.4)
|
Abbreviations: CC, cervical cancer screening; FPL, federal poverty level; SD, standard
deviation.
Note: Users are clinics with clinic staff members who used the CC-tool at least once
in the study period.
Table 3
Clinic characteristics associated with the odds and rates of the cervical cancer screening
tool use (95% confidence intervals)
|
Relative odds of tool use
|
Relative rates of tool use
|
|
Model A
|
Model B
|
Model A
|
Model B
|
|
Panel mix
|
|
Ambulatory encounters
|
1.042 (1.030–1.056)
|
1.037 (1.025–1.050)
|
1.053 (1.047–1.060)
|
1.050 (1.041–1.059)
|
|
Percent White
|
0.993 (0.984–1.001)
|
0.993 (0.984–1.002)
|
0.991 (0.979–1.002)
|
0.988 (0.977–1.000)
|
|
Percent pediatric patient
|
0.985 (0.972–0.996)
|
0.990 (0.975–1.003)
|
0.959 (0.950–0.967)
|
0.968 (0.953–0.984)
|
|
Percent Hispanic
|
1.011 (1.002–1.020)
|
1.005 (0.995–1.015)
|
1.011 (0.998–1.024)
|
1.004 (0.990–1.018)
|
|
Percent Medicaid
|
1.001 (0.987–1.017)
|
1.005 (0.989–1.021)
|
1.013 (0.994–1.032)
|
1.020 (0.997–1.042)
|
|
Percent uninsured
|
0.993 (0.976–1.011)
|
1.002 (0.983–1.020)
|
0.974 (0.949–1.001)
|
0.997 (0.973–1.022)
|
|
Percent FPL <138%
|
0.991 (0.982–0.999)
|
0.988 (0.979–0.997)
|
0.990 (0.981–0.999)
|
0.977 (0.963–0.991)
|
|
Percent rural
|
0.893 (0.219–3.142)
|
0.753 (0.181–2.724)
|
0.351 (0.053–2.325)
|
0.471 (0.112–1.989)
|
|
Implementation impact
|
|
Rates of CC screening
|
|
Highest tertile (>57.3%)
|
–
|
Reference
|
–
|
Reference
|
|
Middle tertile (39.7–57.3%)
|
–
|
0.723 (0.438–1.192)
|
–
|
0.357 (0.194–0.657)
|
|
Lowest tertile (<39.7%)
|
–
|
0.249 (0.138–0.441)
|
–
|
0.042 (0.017–0.108)
|
|
Rates of abnormal Pap results
|
|
Highest tertile (>17.8%)
|
–
|
Reference
|
–
|
Reference
|
|
Middle tertile (12.6–17.8%)
|
–
|
0.692 (0.408–1.165)
|
–
|
0.378 (0.205–0.695)
|
|
Lowest tertile (<12.6)
|
–
|
0.566 (0.327–0.972)
|
–
|
0.955 (0.470–1.941)
|
Abbreviations: CC, cervical cancer screening; FPL, federal poverty level.
Note: Model A: model with panel mix indicators. Model B: model with panel mix indicators
and implementation impact variables. 95% confidence intervals are denoted inside parentheses.
Bold denotes statistical significance (p < 0.05). Users are clinics with clinic staff members who used the CC-tool at least
once in the study period.
Two themes emerged from the qualitative results: lack of awareness of the tool's existence
and lack of knowledge about how to use it in an optimal manner.
Many clinical team members involved in the qualitative data collection processes were
not aware of the CC-tool. One clinician who uses the CC-tool shared, “I can tell you
the majority of my colleagues don't know that exists, let alone do they know they
have to use it.”
Those who did know about the tool learned about it by accident or from a colleague.
Few participants who used the CC-tool felt they knew how to use it effectively. Many
noted they received no formal training on its use; as one midwife describes, “I would
say that right now for our clinic, our training is user error. You just get in, and
you figure it out.” Participants reported lacking a standardized system for tracking
abnormal Pap results, indicating a gap in understanding the CC-tool's purpose. One
clinician described the impact on patient care from such gaps, “We had two patients
this year that I unfortunately had to call about abnormal Pap Smear results done over
a year ago…and that's worrisome.” Although many CC-tool users indicated that using
it was easy once they knew how to access it, several noted difficulties in re-accessing
the tool once they closed it. A clinician shared, “I like how the [CC-tool] is right
now because it's four clicks and then done. The only thing I hate is that I can never
find it again if I forget to put it in before I review the lab.” Others indicated
accidentally not completing all fields in the entry form because it was so long that
seeing all fields required scrolling down, which added time burden. Some noted that
if the results from the last Pap were not entered correctly in the cytology form,
the preventive care section of Health Maintenance defaults to a standard, and at times
erroneous, date for the next recommended CC screening. One nurse practitioner said,
“For us the biggest issue[s] are the modifiers because we're co-testing, but [the
Health Maintenance tools] default to 3 years instead of 5 years, and then you've got
to go in and change it. Our QI team does all the outreach and patients are being called
in for their Pap smears when they weren't due.” Several participants indicated this
lack of trust in Health Maintenance required them to conduct a time-intensive clinical
review to ensure appropriate screening interval and modality.
Discussion
These results show that a CDS tool targeting CC prevention was used by fewer than
half of clinics with access to it. The CC-tool use was higher in larger clinics and
those with higher CC screening and abnormal CC screening result rates, suggesting
the tool's perceived utility by these users. These results add evidence on CDS adoption
in the community health center setting. Prior research in other settings showed that
user-centered design participants are more likely to adopt tools; future analyses
will not assess those longitudinal outcomes in this setting.
Yet while CC-tool users generally liked it, many were not aware of it. This is concerning
but not unusual, as EHRs are regularly updated and revised to provide new tools and
functionalities meant to assist providers in care delivery and panel management. Such
updates are necessary to ensure that EHRs' CDS functionalities follow current care
guidelines. Yet within a complex EHR system, improvement-focused updates can be easily
missed, particularly when multiple modifications are made at once; one study estimated
that an average of 2.5 EHR updates per day are implemented in a large integrated care
system.[29]
In our study setting, EHR changes are communicated to member community health centers
in a scheduled monthly cycle and include guidance on what the change is, rationale
for change, and potential workflow impacts. These results suggest that this information
may not be disseminated effectively to all potential users. Participants lacked knowledge
on how to use the CC-tool effectively; this might be addressed through alternate means
of disseminating instructions on new tool use, such as targeted training (e.g., live
or on-demand demonstrations, printed step-by-step instructions). Research is needed
on best practices for informing users about new EHR functions, or such updates may
not be used, or only by those who “just get in, and … figure it out.” This is relevant
to diverse CDS tools across all care settings and EHR platforms.
Conclusion
For CDS tools to improve health outcomes, users need to know where such tools are
located in the EHR, what they can do, and how they should be used. Thus, while continuous
EHR updates are needed, they are unlikely to substantially improve care quality without
the concurrent provision of effective communication mechanisms.
Clinical Relevance Statement
Clinical Relevance Statement
Improving communication of new or updated clinical decision support tools is critical
for such tools to be used in clinical care.
Multiple-Choice Questions
Multiple-Choice Questions
-
What barrier(s) impact(s) adopting a clinical decision tool?
Correct Answer: The correct answer is option d. All barriers listed above impact the adoption of
new or revised electronic health record tools.
-
What factors were associated with higher adoption of the CC-tool by clinic staff?
Correct Answer: The correct answer is option c. Clinics that had higher rates of abnormal Pap also
had greater tool adoption rates, which reinforces the benefit of clinical decision
support for cervical cancer screenings.
