Keywords clinical decision support - clinical trial - blunt head trauma - traumatic brain injury
- implementation - child
Background and Significance
Background and Significance
The emergency department (ED) is the most common medical setting for the evaluation
of children with minor blunt head trauma.[1 ] Between 2001 and 2010, ED visits for traumatic brain injury (TBI) among children
aged 14 years and younger increased 40 to 50%.[2 ] Although population-based estimates of computed tomography (CT) use in children
with minor blunt head trauma are lacking, recent data suggest continued overuse of
CT for TBI.[3 ]
[4 ] Overuse of CT scans unnecessarily exposes children to ionizing radiation with its
known potential to induce lethal malignancies. Therefore, children with minor blunt
head trauma represent an ideal at-risk population to target for CT minimization strategies.[3 ]
The Pediatric Emergency Care Applied Research Network (PECARN) developed, validated,
and subsequently studied the implementation of two TBI prediction rules (one for children
younger than 2 years, and the other for children 2–18 years old) to optimize clinician
decision-making regarding the use of CT scans.[5 ]
[6 ] Subsequently, the PECARN implementation study[6 ] was a nonrandomized multicenter clinical trial with concordant controls comparing
CT use before and after the implementation of the PECARN TBI prediction rules. Thirteen
EDs in the United States were included in the implementation study: five were academic
EDs in PECARN and eight were community EDs in the Kaiser Permanente Clinical Research
on Emergency Services and Treatments (CREST) Network.
For the PECARN implementation trial, an electronic health record (EHR)-based CT clinical
decision support (CDS) tool was developed and implemented in the Epic EHR through
Web or native CDS services.[7 ]
[8 ]
[9 ] Tham et al describe the customization of the CDS alert into clinical workflows at
different institutions.[7 ]
The CDS rules provided risk estimates of clinically important TBI (i.e., death from
TBI, TBI requiring neurosurgery, hospitalization ≥ 2 nights, hospitalization with
a positive CT, or intubation for TBI > 24 hours) and recommendations regarding whether
or not a CT was recommended based on the PECARN TBI prediction rules. To inform the
design of the EHR CT CDS tool, we (the PECARN investigative team) previously completed
focus group discussions with local health care providers and thought leaders and interviews
with key stakeholders.[10 ] We also conducted workflow evaluations at participating sites, with the goal of
understanding local practices and workflows, and maximizing the acceptance and usefulness
of the CDS.[10 ] The focus groups, interviews, and observations identified early barriers to adoption
and implementation.[10 ] The completion of the multicenter PECARN implementation trial offered the opportunity
to follow-up with participating clinicians and thought leaders and understand the
implementation of the EHR CT CDS tool in more depth.
To support future dissemination and translation, our aim was to summarize the quantitative
efficacy findings of the multicenter PECARN implementation trial and integrate with
additional contextual/qualitative data to identify and understand the reach, adoption,
implementation, and maintenance of the EHR CT CDS tool.
Methods
Study Design
We used a mixed-methods design that summarized quantitative data sources, including
those from the PECARN implementation trial, and integrated them with postimplementation
semistructured interviews with a sample of key stakeholders. We used the Reach, Efficacy,
Adoption, Implementation, and Maintenance (RE-AIM) framework, which provides a structure
to organize the study of intervention implementations.[11 ]
[12 ] Beyond e
fficacy , the framework focuses on the: r
each of the intervention to a representative proportion of the target population; a
doption of the intervention across a broad and representative proportion of settings; i
mplementation details; and m
aintenance of the intervention post intervention.[11 ]
[12 ] The RE-AIM framework has been customized to clinical informatics[13 ] by adding pertinent questions related to predisposing, enabling, and reinforcing
factors.[14 ] Predisposing factors influence motivation to undertake a behavior, and enabling
factors make it possible for individuals to change either behaviors or the environment.[15 ]
[16 ] In the context of a clinical informatics intervention, this also includes institutional
commitment and leadership support, integration of the EHR CT CDS tool into the organizational
context and workflow, time allowed for learning, investment in the change process,
and user training.[13 ] Reinforcing factors include the individual, setting, and organizational factors
that are required to maintain use of the CT CDS tool.[13 ] With regards to setting, this includes the extent to which the CT CDS tool had become
part of routine practice and was being used at least 6 months after study completion.[13 ]
Quantitative Data Sources and Qualitative Samples
We used the following quantitative sources of data in this analysis: (1) the existing
publication from the PECARN implementation trial[6 ]; (2) a national report of TBIs published by the Centers for Disease Control and
Prevention,[17 ] and (3) a report of the market penetration of the Epic EHR.[18 ]
[19 ] For the qualitative assessments, we conducted semistructured interviews at two academic
teaching hospital EDs in PECARN and four community EDs within the Kaiser Permanente
system in Northern California that participated in the implementation trial.[6 ] The semistructured interview sample consisted of health care providers (attending
faculty physicians, residents, advanced practice nurses, staff nurses, and nursing
and medical ED managers), and key stakeholders in information technology (data analysts,
Chief Medical Information Officer).
Procedures
[Table 1 ] outlines the data sources, procedures used to collect the data, and the applicable
RE-AIM dimensions. For the primary qualitative data collection, we developed a postimplementation
semistructured interview guide based on the adoption, implementation, and maintenance
dimensions of the RE-AIM framework. This included eliciting predisposing, enabling,
and reinforcing factors for the implementation and maintenance of the EHR CT CDS tool.
Within each institution, we requested to interview a range a participants representing
different health care provider and administrative roles and years of experience in
the role. A purposive convenience sample was interviewed by two research team members
with experience conducting qualitative interviews (S.B. and R.M.C.). Written or verbal
informed consent was obtained at each site at the beginning of each interview, depending
on the local Institutional Review Board requirements. We conducted most interviews
(n = 37) in person and one by telephone until thematic saturation was reached. We offered
participants $20 gift cards after their interviews. Interviews averaged 30 minutes
in duration, and were digitally recorded, and professionally transcribed verbatim.
Table 1
Data sources for each dimension of RE-AIM
Data sources
RE-AIM dimension
Procedure
Centers for Disease Control and Prevention Report[17 ]
Reach
Extract age, gender, race, and ethnicity data from the report
HealthIT.gov[18 ] and
Epic data sheet[19 ]
Reach
Obtain market share of Epic EHR to inform generalizability of CT CDS tool
Published data from the PECARN implementation trial[6 ]
Reach, Efficacy/Effectiveness
Extract age, gender, race, and ethnicity data; Summarize primary study results
Semistructured postimplementation trial interviews
Adoption, Implementation, Maintenance
Develop semistructured interview guide;
Conduct 38 interviews at 6 study sites
Supplemental tables from PECARN implementation trial[6 ]
Adoption
Extract user-specific adoption information
Abbreviations: CDS, clinical decision support; CT, computed tomography; EHR, electronic
health record; PECARN, Pediatric Emergency Care Applied Research Network; RE-AIM,
Reach, Efficacy, Adoption, Implementation, and Maintenance.
Data Analysis
The authors analyzed the interview transcripts using a qualitative descriptive approach
to identify common themes related to adoption, implementation, and maintenance of
the EHR CT CDS tool across types of sites (academic and community EDs). The interview
analysis was primarily deductive, based on the adoption, implementation, and maintenance
dimensions of the RE-AIM framework, and then the themes were inductively generated
to further characterize the data. The analysis included three discrete steps. First,
two authors with training in qualitative methods independently read each transcript,
and defined codes in a data dictionary (R.M.C. and S.B.). The data dictionary also
included a priori codes based on the literature, previous work, and the interview
guide. Second, one author applied nodes to all of the transcripts based on the coding
dictionary. Third, the authors met to review, discuss, and arrive at consensus and
ensure that the coded data fit the agreed upon definitions of the five RE-AIM dimensions.
Themes that emerged from three or more interviews were identified as common.
To ensure rigor of the qualitative findings, two authors conducted the majority of
interviews and a peer debriefing immediately after each interview to summarize key
points in the interview; one author confirmed the content of the audio recordings
to ensure accuracy. All authors reviewed the coding procedure to ensure dependability
and credibility and the coders adhered to the coding procedure.
Results
Demographic Characteristics of the Semistructured Interview Sample
We interviewed 38 participants, of whom 21 (55%) were attending physicians with a
mean of 13 years of posttraining clinical experience, 8 (21%) were staff nurses with
a mean of 9 years of experience in the ED, 2 (5%) were nurse practitioners/physician
assistants, 2 were residents (5%), and 5 (13%) were other key stakeholders including
a Chief Information Officer, 2 data analysts, and 2 nurse managers. The majority of
the sample was female (66%) and white (82%).
Reach
The primary assessment of the Reach dimension (and thus generalizability of the study
trial) focused on comparing the absolute number, proportion, and representativeness
of those who participated in the intervention to national estimates of TBIs. A total
of 8,067 participants younger than 18 years were enrolled in the implementation clinical
trial after the EHR CT CDS was implemented.[6 ] [Table 2 ] notes that the demographic characteristics of the enrolled participants were comparable
with national estimates of TBI-related ED visits by age (0–19 years) and race according
to the Centers for Disease Control and Prevention.[17 ]
Table 2
Comparison of TBI-related ED visits in PECARN implementation trial (0–17 years) and
national estimates (0–19 years) (based on CDC)
PECARN n = 8,067[6 ] (%)
CDC[a ] (%)[17 ]
Race
American Indian or Alaskan Native
17 (0.2)
−
Asian
287 (3.6)
3
Black
1,558 (19.3)
12
More than one race
521 (6.5)
−
Native Hawaiian or Other Pacific Islander
44 (0.5)
−
Unknown
1,524 (18.9)
24
White
4,116 (51.0)
61
Ethnicity
Data not available
Non-Hispanic/Latino
6,677 (82.8)
Hispanic/Latino
851 (10.5)
Unknown
539 (6.7)
Gender
Male
5,080 (63.0)
63
Female
2,987 (37.0)
37
Abbreviations: CDC, Centers for Disease Control; ED, emergency department; PECARN,
Pediatric Emergency Care Applied Research Network; TBI, traumatic brain injury.
a Centers for Disease Control and Prevention data (2002–2006).[17 ]
For this study, the Reach dimension also included the market penetration of the Epic
EHR since Epic was the vendor for the EHR CT CDS tool. According to HealthIT.gov,
as of March 2015, Epic was the largest primary hospital-based EHR participating in
the Centers for Medicare and Medicaid Services EHR Incentive Program. Epic was the
third largest EHR vendor in the United States.[18 ]
[19 ] According to a 2017 Epic fact sheet, Epic currently has 350 client health systems
in the United States which provide health care services to over half of the U.S. population
(totaling nearly 190 million patients).[20 ] These data suggest that the potential Reach of EHR CT CDS tool is substantial.
Efficacy
The Efficacy dimension of the RE-AIM framework examines the impact on important primary
and secondary outcomes.[13 ] Complete details on the trial results are found in the PECARN implementation publication.[6 ] In summary, for all children with minor head trauma (when all sites combined) intervention
sites had small decreases in CT use (1.7–6.2% across sites; adjusted odds ratio 0.72
[95% confidence interval [CI] 0.53–0.99]). There were also variable decreases in CT
use at the control sites.[6 ] The potential unintended consequences of implementing the CT decision support tool
appeared to be minimal—there was no increase in CT rates among those who were not
at very low risk of clinically important TBI and no increase in the “miss” rate for
clinically important TBIs (with no missed children with neurosurgeries).[6 ]
Adoption
The Adoption dimension of the RE-AIM framework addresses the absolute number, proportion,
and representativeness of settings and clinicians who were willing to initiate the
EHR CT CDS tool. Based on the PECARN CDS implementation trial,[6 ] provider-specific adoption at each site varied based on ED site-specific staffing,
workflow, and study protocol implementation. Overall, at the Kaiser Permanente (community)
sites, attending faculty members completed most of the data (89% across EDs) followed
by nurses (10%) and residents (1%). At the PECARN EDs, there was much more variation
among sites. Of the 16,635 participants enrolled in the intervention from PECARN EDs,
54% were completed by attending/fellows, 27% by nurses, 10% by residents, 7% by nurse
practitioner/physicians assistants, and 2% by other health care providers.[6 ] [Table 3 ] presents data from the semistructured interviews, noting that adoption of the EHR
CT CDS tool was perceived as high across the sites and was reinforced by a strong
“match with mission” of the institutions in which it was implemented.
Table 3
Select quotations related to the adoption, implementation, and maintenance dimensions
of the RE-AIM framework
RE-AIM dimension
Theme
Select quotations
Adoption: Absolute number, proportion, and representativeness of settings and intervention
agents who are willing to initiate the PECARN EHR CT CDS tool
Match with Mission
“And just keeping up on the literature, we're champions of this, bringing it up and
making it part of our electronic medical records, it's helpful.” (Attending physician
#27)
“So I think as a department, as a health network, we're accepting of it and are comfortable
with the using of it and entering the data and using it to our advantage.” (Attending
physician #15)
Implementation: Extent and consistency to which a program is delivered across programs and settings
as intended after it is implemented
Predisposing factors:
Occur before a behavior and influence motivation to undertake a particular behavior
(factors include knowledge, attitudes, beliefs, values, self-efficacy, behavioral
intentions, and existing skills); can occur at both the individual or organizational
level
Perception of relevance of CDS
“It provides you clinical support to provide better quality of care to the patient;
in the case of the child, talking about risk and benefits of potential intracranial
injury versus that of a CT scan.” (ED Director/Attending physician #26)
“I think that for our documentation as a physician, I think that it's really important
to have decision tools documented in the EMR. I think from a medical legal perspective,
it's important. I try to put in decision rules and things of that nature. Things that
help me calculate risk, I try to incorporate that into my note.” (Resident #9)
“I think decision tools are great…. I think using decision tools is very helpful and
making them fairly accessible with an easy-to-use interface and something that can
give you a quick answer helps support your decision. I think they're very useful tools
to have in your back pocket.” (Attending physician #15)
“From a nursing standpoint, it's…honestly, I don't know that it's that beneficial
to us. It's not going to change what I do, how I care for them, how I assess them.
I think it's more beneficial from the physician's standpoint.” (RN #12)
“I guess if there would have been nurse leader, a nurse manager that supported it
along with [the physician champion] with some of that learning as we went through,
you know, how could we use this to our benefit.” (Nurse manager #28)
“But then they [pediatricians] kind of get trumped and overruled by the trauma team
who is going to do what they want anyways.” (RN #17)
Clinical champion
“You've got to have champions that will support and demonstrate why this is an incredible
tool. And once you get a few onboard, everyone will see the benefits of it.” (Attending
physician #26)
Enabling factors:
Precede and support behavior; include institutional commitment and central leadership
support, integration of system into organizational context, time to allow learning,
investment in change process, and adequate user training
Integrated Workflow
“It's coming at the right time, to the right person, with the right information, using
the right channel, and in the right situation.” (Attending physician #3)
“It's probably in a good location from a resident workflow issue.” (Attending physician
#16)
User training
“Of course, education for us when we first started it…this is why we're doing it,
this is what we need…some posters of things like that to kind of help us. It was helpful
to have pictures of where the head trauma was because at first we weren't sure…does
this part count? So, it was helpful to have that graph.” (RN #12)
Ease of use
“The ease. It's incredibly easy to use.” (Resident #9)
“The fact that it's integrated into our medical records system, our electronic medical
records, it makes it very easy to use; you don't have to pull anything down. And probably
the biggest thing that makes it easy to use is that it's integrated into our electronic
medical system.” (Attending physician #27)
“It was really pretty simple, pretty straightforward, especially with the navigator.
I know it was really easy in that sense.” (RN #29)
“I like it. I think it's utilized appropriately. I think most nurses fill it out.
It's pretty simple, you know. Just the flow sheet, fill it out, and it prompts you
to ask the appropriate questions. I do think it's helpful.” (RN #2)
“It flows very easily in Epic…it's very similar to our workflow for other things,
so the learning curve was no big deal.” (RN #1)
“I think at first it was difficult to use because not all of the physicians were onboard
in the limited use of the CT scans, and being a very collaborative
practice, we always run cases by our colleagues, and some are a little more lax in
their use of the CT scans, and some tended to use the CT scan more than others, so
you were always doubting yourself whether this was the right way. So now, using the
CT scan is easier and it certainly easier on you and helps you sleep at night and
removes any doubt from your head that there was a bleed because you're never sure
unless you get a CT scan, frankly. But the data are very reassuring and it helps the
more numbers that we get and the more the data shows this is safe, so it makes it
easier. But initially, it was not easy because you just weren't sure.” (Attending
physician #27)
“It doesn't always trigger and I've tried before to put in as a chief complaint and
make it trigger. But I have a hard time doing it.” (Attending physician #6)
“The only thing that I get frustrated with is when I want it to fire and I can't because
the chief complaint when they first walked in doesn't match. And if I add a new complaint
for some reason, it doesn't pop. I don't know why. ” (Attending physician #6)
Maintenance: Degree to which a program becomes routine and part of everyday culture and norms
of an organization at both the setting and individual level
Reinforcing factors
Institutional culture
“I think we hit all the 5 rights of CDS on this one. It's coming at the right time,
to the right person, with the right information, using the right channel, and in the
right situation….There's been no interest in turning it off. In fact, we've now had
a request to do something similar for abdominal pain. They want some kind of assessment
tool. The same methodology we did and we're hearing from our pediatric surgeons they
want to do something like that to reduce the use of abdominal CTs for abdominal pain
and switch to doing more ultra sounds or nothing or observation. So, I think it's
been a success.” (Attending physician #3)
Benchmarking tool
“There was feedback to providers to say, hey, you're not a very good user of the clinical
decision rule, so people got some direct feedback about what percentile they were
in…you are using it 75 or more percent of the time, you're using about 50 percent
of the time, you're using it less than 25 percent of the time…and that feedback, I
think, was kind of useful…it kind of spurs, oh, I've got to remember to do that. Just
like any other kinds of ongoing feedback to your clinical performance is important.”
(Attending physician #23)
“I would love to be able to do give a provider specific feedback about their adherence.”
(Attending physician #32)
Informational tool
“I think it would be a great, if not shared decision-making tool, more of an informing,
why we're advising the way we are.” (Attending physician #21)
Usefulness
“I've actually been pretty pleased with using it, and hope that there will be more
decision aids and more charts and things that we can use to help with shared decision
making.” (Attending physician #8)
“So, for me to get the feedback, I don't necessarily need the clinical decisions support
that comes along with it, but I do find that it is beneficial, especially with a masses
[residents from a large training program] that come with the Emergency Department…
I think it's very useful for them to guide their decision making process.” (Attending
physician #34)
Education tool
“I find it really helpful with the residents. I think it's almost a better teaching
tool than a decision-making tool, in some senses, because I think a lot of us do it
automatically in our head now. But I think from a teaching standpoint or in a place
where they don't play with kids all of the time, I think it would be really, really
helpful.” (Attending physician #6)
“I think that the visual aid would be really great for families to understand, to
completely understand, and bring it down into concrete terms.” (Attending physician
#8)
“I think it's most useful to me in being able to tell families that want to know that
we have a tool that we look at and it helps us make a clinical decision about whether
or not based on the child's symptoms in the physical exam, we need to do neuroimaging.
And that seems to alleviate some parent's fears about whether or not they needed to
have a CT scan done.” (Nurse practitioner #20)
“I think when it helps is when they're educating the patient or the family.” (Nurse
manager #28)
Abbreviations: CDS, clinical decision support; CT, computed tomography; ED, emergency
department; EHR, electronic health record; EMR, electronic medical record; PECARN,
Pediatric Emergency Care Applied Research Network; RE-AIM, Reach, Efficacy, Adoption,
Implementation, and Maintenance.
Implementation
The Implementation dimension of RE-AIM at the setting level takes into account the
sites' fidelity to the protocol, including consistency of the delivery as it was originally
intended.[13 ] Consistency of the intervention delivery was achieved by standardizing triggers,
CDS prediction rules, and clinician messages, while allowing intended practice variation
across sites (e.g., who entered the items on the trauma tool).[10 ] The participants identified both predisposing and enabling factors that supported
the implementation of the CDS.
Predisposing Factors
Based on the interviews, the most important predisposing factors for successful implementation
were the presence of an approachable clinical champion at each site and a strong perception
that the EHR CT CDS tool was relevant to clinical practice by supporting quality care
to patients ([Table 3 ] and [Fig. 1 ]). Across all sites, a clinical champion was essential for the implementation of
the tool. There were divergent perspectives between and within providers at the different
sites as to the degree of perceived relevance of EHR CT CDS tool. If health care providers
believed, in general, that CDS tools made a difference in the clinical decision-making,
they reported being more likely to use the tool. At the sites where nurses were responsible
for completing the CDS tool, they felt that it was relevant to their role, but if
they were not responsible for completing the tool, they felt that it was irrelevant
to their scope of responsibility and tasks.
Fig. 1 Study methods and qualitative themes.
Among many providers, the EHR CT CDS tool was perceived as being a reusable knowledge
asset and relevant for the development of a future toolkit of CDS resources. On the
other hand, some providers thought that the EHR CT CDS tool might not be relevant
in a high-volume, high-severity trauma setting where clinical decisions have to be
made immediately without the aid of CDS in the EHR. Providers also felt the CDS tool
would be most relevant to children seen in general rather than pediatric EDs (where
the perception is that clinical comfort with young children is lower and awareness
of the harms of CT to developing brains was perceived to be higher [in general EDs]
than in pediatric EDs).
Enabling Factors
There were several predominant enabling factors described by respondents ([Table 3 ] and [Fig. 1 ]) that supported the use of the EHR CT CDS tool including: (1) institutional investment
in user training, (2) integration into the clinical workflow, and (3) ease of use.
Early on in the implementation, each site invested in user training, which included
a clinical champion (ED site physician investigator) and study champions from information
technology who provided ongoing education to all staff. A standardized message was
created centrally, provided to all sites, delivered through a Grand Rounds-type format,
and circulated to all physician and nursing staff. Given the unique workflows at each
site,[10 ] the presentation was also tailored to address unique site features (i.e., charting
at the point-of-care or at a central workstation). The participants felt learning
how to use the tool was facilitated by its similarity to other tools, and to other
workflow processes in Epic. One barrier that a few participants raised was that if
the CDS alert for the EHR CT CDS tool did not trigger automatically, they were unsure
of how to manually activate it.
Maintenance
Maintenance is the extent to which a program or policy becomes part of routine organizational
practice and culture.[13 ] The interviews noted that the EHR CT CDS tool fit well within the institutional
culture and thus became so integrated into the clinical workflow at each of the PECARN
sites that the tool was still in active use despite the completion of the trial. The
interviewees reported actual and potential reinforcing factors to motivate health
care providers to maintain their use of the CDS, including a belief that the EHR CT
CDS tool was educational both for health care providers (specifically residents, nurse
practitioners, and physician assistants) and families. The belief was that the use
of the EHR CT CDS tool engendered greater confidence in health care provider communication
with family members. There was also a strong sense that the EHR CT CDS tool was useful
for supporting education with families. The tool also helped some health care providers
feel more comfortable safely discharging children home by providing the health care
providers with reassurance about their clinical judgment.
A proposed reinforcing factor ([Table 3 ] and [Fig. 1 ]) that may increase use of the EHR CT CDS tool was a benchmark report which would
compare use of the tool between providers, potentially generating extrinsic motivation
to use it as intended. A frequently proposed reinforcing factor was the suggestion
to develop family-friendly visualizations and output to support clear end-user communication
with family members. Automating the output of the EHR CT CDS tool into discharge instructions
was another suggestion for reinforcing use of the tool. For example, when the EHR
CT CDS tool output indicated very low risk of TBI, it should be incorporated into
the discharge instructions; when it indicated risk other than very-low risk, that
information could activate an observation pathway.
Proposed barriers to the maintenance of the EHR CT CDS tool included the technical
challenges of migrating the output of the tool into the clinical note. Health care
providers believed that clearer training on how to automatically include the EHR CT
CDS tool output into the note might sustain the use of the tool. Another barrier to
maintenance was the lack of clarity as to which health care provider should complete
the EHR CT CDS tool. In the current implementation of the EHR CT CDS tool, there was
variation across sites regarding which providers completed the EHR CT CDS tool to
allow for flexible integration into each site's workflow.
Discussion
Using the RE-AIM framework, this study evaluated factors that support the implementation
and dissemination of the EHR CT CDS tool into emergency medical care to scale it to
other EDs in the future. The results provide concrete strategies for the adoption,
implementation, and maintenance of the EHR CT CDS tool to improve its application
into practice. [Fig. 1 ] summarizes the methods for the study (including data sources, settings, and sites),
themes from the qualitative analysis, and future suggestions for the implementation
of the EHR CT CDS tool. These data address one of the largest challenges with CDS:
namely, disseminating best practices for the design, development, implementation,
maintenance, and evaluation of these tools so that health care institutions can learn
from one another.[20 ]
Adoption
Based on our data, successful adoption of the EHR CT CDS tool was strongly supported
by close alignment with the institutional culture of the organizations participating
in the PECARN and CREST network, which value evidence-based practice and are open
to implementing informatics-based CDS tools. According to a recent meta-synthesis
of qualitative studies, one of the most common issues with adoption of CDS is poor
clinician–patient–tool integration of the CDS within the sociotechnical system.[22 ] This barrier was addressed prior to adoption of the EHR CT CDS tool through extensive
analysis of the sociotechnical environment at the sites.[10 ]
Implementation
Usability is among the most common barriers to the implementation of CDS tools, including
issues related to the nuisance of alerts and system immaturity (i.e., interoperability).[22 ] These barriers were considered and addressed prior to the adoption and implementation
phases through the systematic development of the prediction rules and EHR CT CDS tool.[5 ]
[6 ]
[9 ]
[10 ]
[23 ]
[24 ] The implementation of the EHR CT CDS tool was supported by a strong perceived relevance
among stakeholders, and an active clinical champion to support its use and value.
Our finding of the importance of an active clinical champion for the successful CDS
implementation is consistent with findings from a prior study conducted in the CREST
Network that evaluated a CDS tool for managing pulmonary embolisms.[26 ] During the implementation process of the EHR CT CDS tool, important features that
were identified as critical to the success of CDS implementation were carefully attended
to, including: providing the tool to health care providers based on the primary chief
complaint (rather than having them seek it out); having it integrated with the charting
and order entry system (rather than a stand-alone system); and providing a specific
recommendation and the decision support at the time and location of decision making.[26 ]
Maintenance
In general, when moving an informatics application from a clinical trial into routine
practice, seamless integration into clinician workflow is important so that it is
not perceived as an additional disruptive task. In this case, the EHR CT CDS tool
was so well integrated into the clinical workflow that there was little interest in
turning it off after the study was completed. Moreover, clinician perception that
the tool was useful reinforced its use. In another RE-AIM analysis completed for a
multiple sclerosis falls-prevention intervention, factors that influenced long-term
maintenance were: attentiveness to branding and promotion of the intervention; using
behavior change theory to guide intervention development and delivery; building ongoing
support, and small reminders; and documenting the cost-effectiveness and cost-benefit
of the intervention.[27 ] Though the EHR CT CDS tool implementation did not include all of these aspects to
support long-term maintenance of tool use, its development was guided by behavior
change theory, and it had ongoing support and reminders from a clinical champion.
To support the maintenance of the tool, the suggestion from health care providers
to receive feedback of their utilization of the CDS tool (and CT rates) compared with
their peers (benchmark report) is consistent with best practices for performance improvement
and driving knowledge translation.[28 ]
[29 ]
[30 ] The optimal feedback loop appears to be repeated feedback from multiple modalities
by both supervisors and peers.[29 ]
[30 ]
Limitations
One of the limitations to understanding the reach of the multicenter PECARN implementation
clinical trial was the challenge comparing the clinical and demographic characteristics
of the patients in the trial with robust national statistics. At this time, national
statistics are not up-to-date and the characterization of the head injuries and TBIs
are not as granular as in the PECARN data. Limitations of the efficacy of the study
are published elsewhere.[6 ] There were some differences in the implementation of the study at each site, but
these differences were part of the intended variation within a pragmatic clinical
trial design. For the qualitative interviews, there may have been some selection bias
related to interviewing a sample of health care providers who were engaged in the
PECARN and from within the CREST Network who opted to be interviewed. In addition,
our study was focused on results across sites rather than between groups of sites
(i.e., academic vs. community EDs). More broadly, another limitation to generalizability
is all of the study sites belonged to a research network, so they may have been more
inclined to have evidence-based practices and incorporate more innovation into their
practice. On the other hand, the strength of the study is that both academic medical
centers and community hospitals provided data and enrolled patients. Another strength
of the study is that the CDS was developed and tested in Epic, an EHR with substantial
penetration (reach), thus providing support for scaling the intervention. In addition,
provider-specific selection bias was mitigated by automatically triggering the CDS
alert based on chief complaints.
Future Implications
By providing the methods for implementation, we see many logical next steps for this
EHR CT CDS tool, as well as others like it. Throughout the interviews, health care
providers voiced interest in applying the same methodology for diagnoses such as abdominal
pain, asthma, otitis media, fever, and appendicitis. Another suggestion for the current
tool is to provide tailored messaging to different types of health care members. Overall,
one physician summarized it well with a common sentiment on perceived impact; “I think
getting it out there to beyond the pediatric emergency department is probably where
the big impact will be felt.” This was followed up by specific recommendations to
get the EHR CT CDS tool into urgent care clinics because many referrals come from
urgent care for which a CT scan is not indicated. Next steps for this research also
include evaluating whether CDS tools reduce unnecessary health care expenditure and
improve the efficiency of patient flow through the ED by decreasing the duration of
the evaluation and shortening time to discharge. More research is also needed on differences
between academic and community EDs. Finally, further study could clarify if this EHR
CT CDS tool helps to decrease anxiety for children and parents.
Conclusion
We identified multiple concrete features to facilitate and support the adoption, implementation,
and maintenance of the EHR CT CDS tool for children with minor blunt head trauma.[6 ] These features can support the future implementation of the EHR CT CDS tool into
general and pediatric EDs around the country.
Multiple Choice Questions
Multiple Choice Questions
What are the benefits of applying an implementation framework to complement the findings
of an efficacy study?
Provide additional information about factors that influenced adoption, implementation,
and maintenance.
Generate best practices for future implementation studies.
Advance the science of implementation and dissemination.
All of the above.
Correct Answer: The correct answer is option d.
What is an example of a predisposing factor for the implementation of the CT CDS tool
in emergency departments?
Institutional investment in user training.
Approachable clinical champion.
Integration into the clinical workflow.
Ease of use of the CT CDS tool.
Correct Answer: The correct answer is option b, Approachable clinical champion. Responses a, c, and
d, are three enabling, not predisposing, factors for implementation. Predisposing
factors occur before the behavior and influence motivation to undertake a particular
behavior.