Keywords
predictive analytics - continuous predictive monitoring - neonatal intensive care
- sepsis - Diffusion of Innovations - innovation attributes - implementation
Background and Significance
Background and Significance
New technologies, such as predictive analytics, hold the potential to dramatically
improve knowledge about illnesses and their effective treatment.[1]
[2]
[3]
[4] Predictive technologies are designed to assess for and warn of patient risk hours
to days in advance of clinical signs toward the goals of early clinical intervention
and improved patient outcomes.[5]
[6]
[7]
[8] However, risk of illness does not guarantee a patient will develop that illness,
thus clinicians must balance the benefits of early intervention (e.g., resolving infection
prior to sepsis onset) with the negative consequences of delayed treatment or unnecessary
treatment (e.g., developing antibiotic-resistant organisms). To date, predictive analytics
studies primarily focused on statistical model development and accuracy, rather than
on clinicians' acceptance and adoption of predictive technologies into care.[9]
[10]
[11]
[12]
[13]
[14] Among studies that did evaluate clinician use, results underscore clinicians' difficulty
in translating risk prediction into medically actionable interventions.[6]
[15] Known technology implementation challenges (e.g., poor design,[16] misalignment between system design and care processes,[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24] changes to communication and care processes[25]) may further complicate use of these emerging innovations, negatively impact efficacy
trials,[26]
[27] and delay systematic adoption.[6] This article describes neonatal intensive care unit (NICU) clinician perceptions
of a continuous predictive analytics technology and how those perceptions influenced
clinician adoption.
Continuous Predictive Analytics: Heart Rate Observation
Using streaming electrocardiograph (ECG) monitoring heart rate data (RR interval,
sample asymmetry, standard deviation, and calculations of sample entropy) from bedside
monitoring technology, a University of Virginia (UVA) interdisciplinary team (neonatology,
cardiology, statistics, biomedical engineering) developed mathematical algorithms
to discriminated neonatal sepsis and sepsis-like illness.[28] The team then created a monitor to visualize the algorithm, heart rate observation
(HeRO), a predictive analytics technology and refined technology functions in response
to neonatologist feedback ([Fig. 1]). HeRO calculated and displayed a neonate's fold-increased risk of developing sepsis
in the next 24 hours where a score of 1 represents baseline sepsis risk among all
neonates. This score was known as the heart rate characteristic (HRC) index. Updated
hourly, the monitor was designed to display a 5-day trend ([Fig. 1]: orange, top) and indicate highest HRC ([Fig. 1]: yellow vertical line) with corresponding raw heart rate data ([Fig. 1]: green, bottom) with controls to allow users to scroll back through time.[29]
Fig. 1 Heart rate observation (HeRO) monitor-visualizing heart rate characteristics index,
corresponding heart rate pattern, and controls to scroll forward and backward in time.
Following Food and Drug Administration approval, the HeRO research team conducted
a parallel, two-group, individually randomized control trial (RCT) (NCT00307333) among
3,003 very low birth weight (< 1,500 g) neonates from 9 U.S. NICUs to determine if
HeRO improved neonatal sepsis outcomes.[29] At UVA, a single HeRO monitor ([Fig. 1]) was mounted in a central location in each of six pods to maximize visibility from
the 6 to 9 beds contained in each pod.
At the beginning of the clinical trial (April, 2004), the research team provided NICU
clinicians with information regarding how the score was calculated and that a rising
score might indicate the need to assess the patient and, as needed, to test or treat
as appropriate.[29] The research team did not use other implementation strategies (e.g., programmatic
training, decision aids, treatment protocols) to promote or improve provider engagement
with HeRO or to influence the use of HeRO data in clinical care due to concerns about
unnecessary sepsis evaluations or overuse of blood cultures and antibiotics.[29] Concluded in May, 2010, the RCT resulted in significant reduction in sepsis-related
mortality (22%) among very low birth weight neonates (hazard ratio= 0.7; 95% confidence
interval [CI], 0.61–0.99, p = 0.04) and among extremely low birth weight neonates (26%; hazard ratio = 0.74;
95% CI, 0.57–0.95; p = 0.02).[29] Monitored infants experienced a slight increase in drawn blood cultures (10%) and
days on antibiotics (5%), the difference from nonmonitored infants was not significant
(p = 0.31).[29] HeRO remains in use and has been associated with early recognition of other significant
neonatal illnesses such as necrotizing enterocolitis[30] and respiratory decompensation.[30]
[31]
[32] HeRO represents an early example of predictive analytics using continuously available
bedside ECG monitor data and may be the first to be routinely used in care delivery.
Members of the UVA NICU were the first to use HeRO and represented an ideal user group
from which to understand user perceptions of newly developed predictive monitoring
technology.
Guiding Framework: Diffusion of Innovation
Diffusion of Innovations (DOI) served as the theoretical lens specifically because
this theory considers clinical team members' needs, motivation, values and goals,
skills, learning style, and networks as core components influencing adoption of new
practices.[19] The theory has proven useful for understanding adoption of care cueing,[33] surgical checklists,[34] after-visit summaries,[35] and technology in geriatric care.[36] DOI research notes that an innovation's attributes (i.e., complexity, compatibility,
trialability, observability, relative advantage) influences if and how quickly an
innovation will be adopted ([Table 1]).[33]
[34]
[36]
[37]
[38]
Table 1
Innovation characteristics defined[a]
|
Complexity
|
Degree to which an innovation is perceived as difficult to understand and use; complexity
can be reduced by practical experience and demonstration, or adopted piecemeal (p.
596)
|
|
Compatibility
|
Degree to which an innovation is perceived as consistent with existing values, past
experiences, and needs of potential adopters (p. 596)
|
|
Trialability
|
Degree to which an innovation may be experimented with on a limited basis; reduces
risk (p. 596)
|
|
Observability
|
Degree to which the results of an innovation are visible to others; results are visible;
stimulates peer discussion of a new idea (p. 596)
|
|
Relative advantage
|
Degree to which an innovation is perceived as better than the one it supersedes; can
be measured in economic terms, social prestige, convenience, satisfaction (p. 595)
|
a Rogers, E. Diffusion of Innovations.5th ed. New York: Free Press; 2003.
If an innovation is perceived by clinicians as difficult to use, not integrated with
existing workflow, and/or does not offer an advantage over existing practices, clinicians
are less likely to adopt it.[16]
[27]
[34]
[39]
[40] And thus, careful attention to users' perceptions of and reactions to an innovation's
attributes may lead to better design and improved integration into care delivery.
An innovation's diffusion within a social system, such as a patient care unit, is
influenced by communication among members.[37] Negative providers' perceptions may create implementation issues, a significant
challenge when newly developed technology undergo clinical trial effectiveness evaluation.
For example, Kappen et al's study of a newly developed predictive screening tool recommending
a preemptive approach to postanesthesia nausea and vomiting, found that clinicians'
preference for their usual, trusted treatment lead to failure to use the screening
tool data, actions that may have contributed to the study's null findings.[26]
[27] Among studies of surgical safety checklists and innovative surgical procedures,
conflict between new and existing processes led clinicians to deviate from best practice[16]
[34] or abandon the new surgical procedure altogether.[41] Clinicians' willingness to trial a newly developed innovation is a critical component
of efficacy studies, predominant technology implementation strategies (classroom training,
protocols) may be inappropriate because the technology lacks evidence of best use
in practice.
Methods
Study Design
This study employed a cross-sectional qualitative descriptive design using individual
interviews collected from an academic NICU in central Virginia.[42] Participants were recruited through a convenience sampling strategy that included
any point of care clinician (registered nurse, respiratory therapist, nurse practitioner,
attending physician) who worked in the unit and were exposed to the HeRO display monitoring
for any period of time. There were no exclusion criteria.
Setting
UVA Health System is a regional academic medical center located in Charlottesville,
Virginia, United States. UVA's NICU was involved in the HeRO clinical trial from April
2004 to May 2010. At the time of the trial, this 45-bed, level IV NICU admitted approximately
600 neonates annually and was organized into 8 sections, or pods of 6 to 9 beds each.
Participants
Following permission from the UVA NICU medical director, NICU members (nurses, nurse
practitioners, and resident, fellow, and attending physicians) were contacted by email
inviting participation in a qualitative study of medical decisionmaking using HeRO.
A follow-up email was sent 2weeks later. Respondents were scheduled for in-person
or telephonic semistructured interviews during January and February 2012. Consent
was obtained from each participant at the time of the interview. The study received
ethics approval (UVA Institutional review board- SBS #2015–0352).
Data Collection
Following an interview guide ([Appendix A]), semistructured interviews with open-ended questions were conducted in-person and
telephonically, audio recorded, transcribed verbatim, and imported into ATLAS.ti (Scientific
Software Development GmbH, Berlin, Germany). These data were not subsequently analyzed
due to study staff turnover.
Appendix A
Semistructured interview guide
|
Date:
|
|
|
Profession
|
MD-Attending; MD-Fellow; MD-Resident Medical Student;
Nurse Practitioner; Registered Nurse
|
|
How long have you been a [physician, nurse, student]?
|
|
How long have you worked in the neonatal intensive care unit (NICU)?
|
|
Please describe your routine on a typical day in the NICU.
When you walk up to a baby, what do you do to assess the situation? What's the very
first thing that you physically do?
|
|
There are a lot of devices and data displayed in the NICU, which ones do you pay the
most attention to? What kind of information do you get from them?
|
|
If a new piece of equipment arrived in the NICU, how would you integrate a new approach
into your practice of taking care of infants?
|
|
Do you feel there are any personal obstacles that could prevent you from incorporating
a new approach into your practice?
|
|
Can you tell me a little bit about when and how you were introduced to the HeRO monitor?
Do you remember when you were introduced to the HeRO monitor and how that took place?
How did you learn to use the data available through the HeRO monitor?
Since you were first introduced to the HeRO monitor, how do new members of the care
team learn about HeRO?
|
|
How does HeRO play a role in your practice?
What information do you get from HeRO?
What role does HeRO play in terms of all the other monitor information you use?
|
|
Does the literature on the HeRO monitor currently, or when you first were introduced
to it, inform your practice?
|
|
Is there anything else you would like to tell me about your practice when it comes
to the HeRO monitor?
|
|
Would you share your observations about how other members of the NICU healthcare team
use HeRO?
Is there a difference in how they use it?
Do you find that certain members of the team look more at the trend compared with
the absolute value?
HeRO data are presented or used differently by the nurses compared with nurse practitioners?
Do parents provide any information about HeRO scores?
|
Data Analysis
Because these data were collected to answer a different research question, R.A., R.K.,
C.L., and J.M. read five interviews (registered nurse, nurse practitioner, resident,
fellow, and attending) to determine if appropriate data existed to answer study aims.[43]
[44] Due to the open-ended nature of the questions and the topics covered in the interview
guide, this preliminary review found descriptions of participants' perceptions of
HeRO, interactions with HeRO, and use of HeRO data in practice.
Informed by the desire to understand NICU clinicians' perceptions of HeRO and the
relationship between perceptions and subsequent adoption of HeRO, the team developed
four broad a priori codes to explore how clinicians first became aware of HeRO (awareness);
learned to interpret HeRO data (interpretation); used HeRO (use); and how HeRO data
guided care decisions and actions (decision-making and action).[45] The team then read a subset of interviews and identified additional codes to capture
contextual information about professional roles and routines, and care responsibilities.
Following code and code definition agreement, the teams were divided into pairs to
conduct descriptive analysis using the full code book. Each interview was read in
its entirety by both coders, and then in a second reading, codes were applied to text
segments. Coding pairs conducted code agreement meetings at two separate times during
the coding process. The entire team met on a weekly basis to review code level text
segments, evolving memos, and emerging themes. Guided by Miles et al,[46] text segments were abstracted for each code into matrices as matrices facilitate
sorting and grouping segments to identify themes informed by DOI's five innovation
attributes. The team frequently returned to original material to uncover assumptions
and explore alternate hypotheses. The team used four strategies to enhance trustworthiness:
(1) multiple team members coded the same interview data with cross-validation of code
use; (2) assumptions and questions about the data were captured in memos and reviewed
with the team; (3) all aspects of the study design were open for review by the members
of the research team; and (4) all members used ATLAS.ti to provide an audit trail.[46]
Results
The 22 participants represented a cross-section of healthcare professionals: registered
nurses (n = 3), nurse practitioners (n = 3), resident physicians (n = 3), neonatology fellows (n = 4), and neonatology attending physicians (n = 7). Participants ranged in professional experience from 2 to 30 years and worked
in the UVA NICU between 1 month and 15 years. Seven participants were employed at
the inception of the clinical trial, seven joined during the clinical trial, and six
joined after the trial concluded. No participants were members of the UVA research
team. Participants' perceptions of HeRO are organized according to the five DOI innovation
attributes (complexity, compatibility, trialability, observability, relative advantage)
([Table 2]).[37]
Table 2
Select quotations related to DOI innovation attributes
|
Complexity
|
|
Location
|
“It's very different pod by pod. I think that it sometimes can be kind of hard to
see since especially if you're going to a—you go to a bedside, you hear all the things
and you say, “Oh, what's the HeRO score?” Either they haven't looked or you're trying
to look and sometimes if you're [in] the middle of B pod you're trying to stand on
tippy toes and look through the glass or around the corner or something and see, and
count of beds so that you can kind of see what the HeRO score is from a distance and
it's kind of hard to do.” (Attending)
|
|
“The nurses weren't focusing on it. The residents really didn't know much about it.
[Research team] realized that there were times when the HeRO score was just getting
ignored.” (Attending)
|
|
Understanding
|
“Until somebody says to you and takes you by the hand…this is how to approach these
screens; these are the questions you can answer with this technology, I won't use
it just cause it's there.” (Fellow)
|
|
“As long as I understand what it's there for and I understand how it works, that it's
been well-taught and that there's some evidence behind its use, then I'm all for it.
I think if it's been well-explained I can latch onto things pretty quickly as long
as I get to play with it a little bit before it has to be on a real patient, if I
get to really sort of see it in action or whatever.” (RN) **cotheme: Trialability
|
|
“Cause we've seen the results. That frequently the blood cultures will come back positive.
The baby did have an infection. Particularly when the study was going on and you'd
have babies that were blinded that weren't on the Hero you wished, when they showed
signs of infection. You wished you could've seen what the Hero was doing there, because
you just knew it would've gone up. So, I feel like we have certainly seen that it
does seem have a predictive value.” (RN)
|
|
“For example, like I said, the fact that Hero is on a wall and I can scan nine patients
and pretty much instantly know what's going on, that, to me, is a testament to the
power of the graphic, versus the fact that I can go into Epic and find it but the
visualization process is different.” (Fellow)
|
|
Compatibility
|
|
Care tasks
|
“Initially, our physician group wanted nursing staff to begin to just document that
number. I (said) You are delegating a responsibility of observing a numerical trend,
but you have not provided any direction as to what constitutes need for a response.
Until you can articulate that to the nursing staff, the nursing staff cannot assume
accountability without knowing what your response algorithm is. I think they, as a
team, determined what those parameters would be for response.” (RN)
|
|
“Then the nurses were told if the HeRO score goes up by a certain amount they need
to alert a clinician—a nurse practitioner, or a resident, or a fellow, or an attending.”
(Attending)
|
|
“…routine care involves vital signs every so many hours, depending upon your patient
population, and the HeRO score is a part of vital signs monitoring.” (RN)
|
|
“It actually now appears automatically. There's a way to automatically get it put
into the progress note. Even just in the last six months when I've been rounding with
Epic, I've noticed that the residents are much more aware of what the HeRO score is
and what it means than they were three years ago.” (Attending)
|
|
Communication
|
“We actually had the fellows responsible for reviewing the HeRO trending overnight
so they would know what might have transpired with the baby's monitoring overnight.
So that would be part of their presentation. Even if it's the matter of the HeRO remains
below two they were looking at it.” (Attending)
|
|
“You know, I'm trying to think of who doesn't use it. We're a pretty—it's pretty engrained
in our practice at this point that everybody, even in our report as nurses when we
hand off, will make a comment; HeRO stable or HeRO went up overnight, but this is
what we're doing about it. So I really can't talk to very many instances where it
hasn't come up.” RN
|
|
Trialability
|
|
Clinical reasoning
|
“…especially at the beginning when it was over two we were doing a full blown workup,
I felt like there was a lot of unnecessary workups. That just in itself predisposes
the baby to—you stick in a catheter in their urethra, you probably—a little bit more
prone to bladder infections then.” (Nurse Practitioner)
|
|
“On call at night when there's a kid that's not doing well, we have some suspicion
of sepsis, the nurse practitioners or the fellows would take a look at the HeRO monitors.
Then we'd talk a little bit about what the significance of those numbers were and
whether or not that push[es] us one way or another in our decision making.” (Resident)
|
|
“It's something that if I go to them [physicians] with the information and say, there's
been a change in the HeRO score that draws their attention to it. They'll look on
that and try and incorporate that as one more piece of the puzzle in trying to make
their decisions.” (RN)
|
|
“Then there are some babies where they might do one little odd thing or something
that's maybe a little bit concerning but it's only the one event. Then you go back
and you look at the HeRO score and say, well, did it go up?” (Attending)
|
|
“What [HeRO's] really done is shown me that I think putting both together, using HeRO
score and something else is a lot more predictive, or it guides my care and my decision
making more so.” (Attending)
|
|
It has shown that it [HeRO] makes a difference, and we obviously believe in it strongly
here, so we pay attention when the HeRO goes about two. We don't necessarily-and if
it's just a HeRO, then we get a CBC, but if there's more clinical symptoms that are
correlating with the HeRO then we go ahead and do blood and urine and potentially
start antibiotics” (Nurse Practitioner)
|
|
Observability
|
|
“We've all learned about it from the same attendings and fellows and nurse practitioners,
so if you learn something from the same people, I think your practice with it tends
to be at least similar.” (Resident)
|
|
“[Attendings] spent some time explaining to me what it is, how it works, how you can
look at it…so, just learning in which clinical aspects would you do this versus that
I've learned from the attendings.” (Fellow)
|
|
“Nobody really explained it. I learned about it from just the routine of once in a
while people would go and check on it (HeRO score), or a nurse would say, oh, the
HeRO score's up, and I'd be like, ah, what does that mean? I don't know. What's a
HeRO score? Then just from being there, gradually I picked up that it was about heart
rate variability.” (Fellow)
|
|
“The HeRO score is a part of vital signs monitoring. So the ability to critically
analyze that for a new hire is supported by an experienced nurse helping [them] along
the way to interpret that.” (RN)
|
|
“When nurses are brought into our unit, if they're a novice new grad, they get six
months of a precepted orientation. So that means, they are paired with a person. At
the very beginning, routine care involves vital signs every so many hours, depending
upon your patient population, and the HeRO score is a part of vital signs monitoring.”
(RN)
|
|
Relative advantage
|
|
Supports clinical judgment
|
“The vital signs of the baby, as far as monitors go. Hero is helpful sometimes. But
lots of times I feel that the baby tells you first. Especially after having a good
bit of experience, the Hero can kinda help back up your feeling that the baby's getting
sick. But at this point I can kind of get a feeling.” (RN)
|
|
“…it can always kind of help out my case I think. If I think that a baby who … is
becoming ill, or he needs respiratory support further than what he's already on. I
can kind of grab the docs and be like; this is what I'm seeing. Oh by the way the
Hero score is up. Then that kind of helps them say, oh okay well let's go ahead and
get septic work, or whatever needs to be done.” (RN)
|
|
“I think it's just another thing to add to their [RN] story to get me concerned…I
think that's reasonable but then that prompts us to go in and investigate it.” (Fellow)
|
|
Surveillance
|
“…one of the things that it's [HeRO Score] done is both shown how something that's
non-invasive that can be active all the time can be helpful.” (Attending)
|
|
“…I'm giving tours at the NICU to families and they're worried about their baby hooked
up to these monitors—what's reassuring to some of them is saying “Here is the monitor,
here are all the numbers we're looking at and we're getting data on your baby. I'm
not even touching your baby. I'm not poking or prodding your baby. I can see what
the heart rate is, I can see what the respiratory pattern is, see what the blood pressure
is, … we have all these methods of evaluating your baby without having to wake the
baby up and take a blood sample. I think the HeRO is one of those ways that we can
assess the baby without hurting the baby so to speak.an advantage is not hurting the
baby.” (RN)
|
|
“There really was not anything like Hero that they used before in terms of its predictive
quality. the only close comparison is a human caregiver having an instinct that something
MIGHT happen.” (Fellow)
|
|
Evidence base
|
“I know there has been a recent review in [journal] by [doctor]. I have not had a
chance to read that. But we talk about HeRO all the time. Before they present some
data in big national meetings.” (Attending)
|
|
“[The published study] has shown that it makes a difference, and we obviously believe
in it strongly here, so we pay attention when the HeRO goes above two.” (Nurse Practitioner)
|
|
“We're always proud to say it's the biggest randomized clinical trial of very low
birth weight infants ever with 3,000 patients. The fact that that showed mortality
reduction, I mean there's really not much that reduces mortality in preemies.” (Attending)
|
Abbreviations: CBC, complete blood count; DOI, Diffusion of Innovation; HeRO, heart
rate observation; NICU, neonatal intensive care unit; RN, Registered Nurse.
Complexity
Innovations that are easily understood and used, can be learned incrementally, or
can be experimented with are more likely to be adopted.[37] At clinical trial inception, participants (n = 9) reported receiving a brief presentation provided by a member of the HeRO research
team. This presentation described the monitor's function and score meaning: the fold-increased
risk that a neonate will develop sepsis in the next 24 hours. Although participants
received initial information about the display and score, participants did not know
whether a changing score should drive clinical decision making or whether it should
merely contribute to overall clinical impressions. A neonatology fellow reflected
on the introduction of technology into care, suggesting that use may take more than
simply understanding how predictive data are calculated and what the data represents:
“Until somebody says to you and takes you by the hand…this is how to approach these
screens; these are the questions you can answer with this technology, I won't use
it just cause it's there.”
Prior to HeRO's introduction, users were accustomed to making care decisions based
on physiologic data that provided information on the neonate's current status (i.e.,
respiratory rate, heart rate, laboratory values, etc.). Because this was the first
application of HeRO in a clinical environment, participants had no experience with
HeRO scores or their association with patient symptoms. Neither could participants
rely on other members of the care team to help them learn about or use HeRO. This
lack of experience may have negatively affected initial engagement with HeRO data.
In fact, the lack of use was pervasive across the entire care team. It appears that
participants' initial engagement with the data was not influenced by knowledge about
HeRO provided when the monitors were installed or by the monitors' presence on the
unit.
Compatibility
Innovations that align with users' values, needs, or past experiences are more likely
to be adopted.[33]
[34]
[37] Clinicians noted that the location of HeRO differed from the location of other devices
they used in day-to-day care delivery. Physiologic monitors that displayed heart and
respiratory rates, oxygenation, blood pressure, etc., resided at each neonate's bedside.
The single HeRO display was centrally located in each pod. To see the data, clinicians
describe the need to move away from the bedside, stand on “tippy toes,” or to walk
to the monitor. Thus, physical location may have deterred routine engagement with
HeRO.
The research team and unit managers agreed to undertake initial steps to increase
participant attention to the HeRO score, yet refrained from requesting specific care
interventions in response to the data. Participants described strategies that align
with or were compatible with routine care practices. Nurses were instructed to record
the score every 4 hours and alert nurse practitioners and physicians if the score
reached two or increased by two. Fellows were required to observe and report on score
trends and care actions during morning patient rounds. These strategies appear to
have influenced participant behavior as noted by one nurse practitioner, “Having the
nurses writing it down was key to us being successful in reacting appropriately to
the spikes and so forth as they happen. I think that was a turning point.”
At first, nurses documented HeRO data on the paper vital signs flow sheet, then later
in the electronic health record. Similarities between HeRO and vital signs collection
pattern (every 4 hours) or its documentation in close proximity to vital signs data
eventually caused nurses to view HeRO as a vital sign. Prior to these requirements,
HeRO data did not have a place in the routine assessment, documentation, or daily
conversations about patients. Overtime, data collection and communication became embedded
in care routines and team interactions.
Trialability
The trialability, or the ability of participants to experiment[37] with HeRO during the RCT, allowed for a more nuanced understanding of the application
to develop over the course of the study. Several attendings and nurse practitioners
observed that initial reactions to rising HeRO scores may have led to unnecessary
testing due to inexperience. “…especially at the beginning when it [HeRO Score] was
over two we were doing a full blown workup, I felt like there was a lot of unnecessary
workups” (nurse practitioner). They attribute this perception to inexperience with
HeRO as well as clinical inexperience among some team members.
The clinical team eventually learned that a score of two or a rise of two did not
necessarily mean that a neonate had sepsis. Through documentation and data presentation
during patient rounds, HeRO data became integrated as a component of the overall dataset
routinely used in care decision making. Further, participants eventually learned that
not all neonates with rising HeRO scores would develop sepsis. As they gained experience,
participants developed critical judgment about the relationship between HeRO scores
and signs to guide when to undertake diagnostic testing and treatment. Over time,
participants came to rely on the score to help them understand uncertain emerging
symptoms and used HeRO in ongoing communication and decision making about next care
interventions.
Observability
The more readily a user can see or observe the results of using an innovation, the
more likely it will be adopted.[37] Members of the NICU also learned how to interpret and react to HeRO data by observing
the practices of more experienced clinicians. Less experienced clinicians observed
if and how senior participants used HeRO data. “[Attending] spent some time explaining
to me what it is, how it works, how you can look at it…so, just learning in which
clinical aspects would you do this versus that I've learned from the attendings” (fellow).
Less experienced participants appear to have benefited most when senior members shared
how they use HeRO data in care decision making. Both less experienced as well new
members of the NICU reported observing the practices of experienced participants to
figure out how to interpret and use HeRO data in care delivery.
Relative Advantage
Observable, substantiated advantage of an innovative and newly introduced technology
is seen as a pivotal attribute for influencing its adoption.[37] In the case of HeRO, there was no evidence base or even experienced participants
upon whom the NICU team could rely. Through interaction with the data and observation
of neonatal symptoms and outcomes, HeRO data served different purposes for different
types of participants. For example, the data confirmed nurses' emerging clinical impressions
and helped nurses determine when to share their observations to other members of the
care team. Physicians came to expect nurses to use HeRO data when communicating about
a patient's status. The inclusion of HeRO data in nurses' communication about patients
seemed to serve as a trigger for physician team members because this may have prompted
patient assessment or closer examination of patient data. Over time, participants
recognized the benefit of noninvasive, continuous monitoring.
As evidence from the RCT emerged, NICU clinicians identified themselves as contributors
to a significant improvement in neonatal care delivery. RCT findings may also have
served to reinforce that participants made the correct decision to use HeRO in care
delivery as it implies that use has scientific merit.
Discussion
This study examined NICU clinicians' perceptions of a predictive analytics monitoring
technology following the conclusion of the RCT establishing its efficacy. In light
of the novel nature of HeRO, evidence of effectiveness as well as guidance for its
application in care delivery was limited.[28]
[47] Use of prediction in care delivery often requires a balance between the benefits
and risk of taking action. Consistent with DOI research, study results suggest that
HeRO's attributes were key to influencing its use in the NICU. The findings highlight
participants' initial reaction to HeRO, the effect of minimal prompts on participant
engagement with HeRO data, how the care team learned to interpret and use HeRO to
guide care decisions, and how the benefits, or relative advantage, of HeRO data emerged
over time with experience.
Reduce Complexity: Provide Simple Guidelines for Engaging with HeRO
Knowledge about the usefulness of an innovation may not be sufficient to promote an
innovation's use. Although sepsis remains a significant cause of death for neonates,
the presence of HeRO in relative proximity to the bedside was insufficient to promote
use among study participants. Prior research indicates that if an innovation is difficult
to use or understand, adoption may occur slowly or not at all.[26]
[48] Because HeRO provided an early alert for the increasing potential for sepsis, but
was not a definitive test for sepsis, participants may have had difficulty knowing
if and when an increasing HeRO score warranted medical action. Combined with the physical
location of HeRO, difficulty interpreting the score in the context of care delivery
may have been a contributing factor for the initial lack of attention. Simple, mandated
interaction with the data, such as documenting and communicating, increased both written
and verbal visibility and was seen by study participants as a turning point. Further,
guidelines for when to report score changes, a “call out” procedure, likely reduced
nurse uncertainty or worry about raising false alarms and engaged several types of
care providers in the evaluation of HeRO trends and patient status. Call out procedures,
a type of decision aid, are associated with effective clinician communication, early
care intervention,[49] and reduced mortality among hospitalized patients.[50] However, several participants expressed concern that initial reactions to HeRO led
to unnecessary sepsis work-ups. The RCT did note a nonsignificant increase in blood
cultures and antibiotics[29]; it may be that the concern voiced by more experienced clinicians actually curtailed
overreaction to rising HeRO scores. Decision aids, such as the one described in our
study, may promote engagement, while avoiding mandated care actions and may provide
a more effective means of introducing predictive analytics technologies into complex
healthcare settings.
Enhance Compatibility: Align HeRO-Related Tasks with Existing Clinician Experience
Studies note that congruence, or compatibility, with user norms, values, and experiences
increases the likelihood of adoption.[36]
[51] In this study, nurses were asked to document the HeRO score every 4 hours, a pattern
and task nurses routinely perform. Fellows were tasked with observing HeRO trends
and reporting their observations as a component of their daily patient presentation
to the care team. Through documentation, HeRO data were in the same medical record
location as other relevant clinical data (e.g., heart and respiratory rate) that,
in turn, may have influenced clinician perspective perhaps because it overcame the
misalignment between HeRO's location on the unit when compared with other physiologic
devices. Over time, HeRO data gained credibility and were eventually viewed as another
vital sign that became a component of care communication and decision making. Thus,
defining HeRO as a vital sign connected the innovation to existing clinician practice
and understanding of physiologic data.[52]
Foster Trialability: Promote Observation and Association
Users desire the opportunity to trial an innovation because it lessens uncertainty,
promotes trust, and may confirm the benefits of using an innovation.[37]
[53] Further, seeking feedback from users' trial experiences may provide opportunities
to improve functionality.[54] HeRO may be a particularly difficult technology with which to experiment because
scores rise as much as 24 hours in advance of symptom presentation.[55] Thus, early forecast of sepsis may be incongruent with clinical assessment. Yet,
it appears that clinicians engaged in a form of trialability. Through active evaluation
of HeRO trends, emerging signs, and neonates' responses to care actions, clinicians
made sense of HeRO and developed judgment about when to wait, undertake further testing,
or initiate treatment.[56] This finding suggests that developing learning cases may provide opportunities to
trail clinical decision making by allowing clinicians to look back over time, explore
patterns, and associated care actions as well as neonates' responses.[57] This may be particularly important for patients who exhibit a high degree of score
variability and present an uncertain clinical picture. Further, deliberation about
HeRO data in conjunction with patient signs often took place among a few collaborating
clinicians, thus the larger NICU community missed the opportunity for collective learning,
a situation that may be improved by consistent use of learning cases.
Increase Observability: Respected Leaders Provide Meaningful Examples
During their training, residents work with attendings from different clinical specialties.
Study attendings integrated HeRO into their education practices including one-on-one
mentoring, conducting patient rounds, demonstrating clinical reasoning, and providing
formalized classes. Resident and fellow participants noted that they valued “hearing”
attendings' cognitive processing of HeRO data and indicated that they followed attendings'
examples. In addition to participation in patient rounds, nurses learning took the
form of orientation, one-on-one work with a nurse preceptor, and protocolized tasks.
Direct observation of respected, successful “other's” innovation use is associated
with increased likelihood of adoption.[53] Although not planned, attendings served as champions, key individuals who supported
the innovation through observable HeRO use and verbalization about HeRO within clinical
reasoning. Formally assigning HeRO data collection and communication tasks to fellows
and nurses may have implied that attendings and senior nurses valued HeRO data.[53] Unlike technology training methods that move clinicians to the classroom and separate
the professions, senior clinicians served as role models and provided HeRO training
and learning within the context of care.[58]
Demonstrate Relative Advantage: Experience and Evidence
Through continued use, HeRO was eventually viewed as advantageous to clinicians' care
communication and decision making. Studies identify relative advantage, defined as
an innovation's benefit to the user, as an essential innovation attribute linked to
adoption.[53]
[59] In the absence of firm evidence of benefit (e.g., monetary, quality, efficiency,
satisfaction), uptake of innovations is prolonged.[60] In the case of HeRO, relative advantage had not yet been established, thus clinicians
had to discover the advantage as they worked with and learned about HeRO. Fitzgerald
et al suggest that in medical contexts ambiguous new scientific knowledge is socially
mediated, meaning that an innovation's benefits are established through use and ongoing
dialog between clinicians.[61] In this study, social mediation occurred through the use of call out procedures,
daily patient rounds, and clinicians' day-to-day collaborative clinical reasoning.
HeRO data provided evidence to support emerging clinical impressions that nurses communicated
to providers. Physicians and nurse practitioners came to expect nurses to use HeRO
data as evidence of clinical concern. Clinicians came to value the noninvasive nature
of HeRO as it provided continuous monitoring without the pain or risk of infection
associated with laboratory testing. Positive clinical trial results may have served
as a form of affirmation. Since evidence of effectiveness influences clinician willingness
to use prediction in practice, frequent review of case examples may provide sufficient
evidence to promote initial clinician engagement with new innovations such as HeRO.
Strengths and Limitations
Study limitations include generalizability, strong interest in HeRO at UVA, small
sample size, and recall bias. While selecting participants from a single hospital
unit limits generalizability, this units' extensive experience with HeRO allowed us
to explore the experience of implementing a newly developed technology. Our participants
were the first to be recruited into the RCT, the first to encounter HeRO data and
knowledge in the care environment, and therefore, had the greatest experiences to
share with the study team. Although our sample was small, it represented a cross-section
of professions, neonatal experience, tenure on unit, and experience with HeRO. Due
to the wide variation in experience with HeRO, we achieved thematic saturation only
in terms of HeRO use in clinical decisionmaking: HeRO was a “piece of the puzzle,”
on data point among many considered when developing a medical course of action. Because
participant interviews took place a year following the conclusion of the clinical
trial, participant data were at risk for recall bias. However, we found consistent
descriptions of the original implementation; the strategies unit managers first established;
as well as how members learned through role modeling and dialog across participants.
A next step to understanding the use of HeRO in hospitals might include direct observation
of care team actions. Finally, this study examined a single type of prediction, sepsis,
a particularly persistent, devastating illness. Future research efforts should consider
evaluating other types of negative patient experiences, such as hemorrhage, where
early intervention is associated with improved survival.
Conclusion
Other than a handful of prediction studies, the majority focused on model development
and validation, therefore there is little evidence to guide integrating predictive
data into providers' care routines.[11]
[27] Tools such as the Acute Physiology and Chronic Health Evaluation, are typically
used to benchmark ICU quality and not alert providers to patient decline.[11] Further, there is little evidence to guide interventions to promote clinician acceptance
and use of predictive technologies as the majority of studies focus upon model development
and accuracy, not on providers' acceptance of prediction as an element of care decision
making.[9]
[11]
[14]
[62]
[63] Because the success of predictive technologies such as HeRO rely on the human system
for interpretation and action,[26]
[49]
[64] processes to build human capacity to interpret predictive data in the context of
clinical reasoning are essential. Simple strategies designed to engage clinicians'
attention and promote data communication may be foundational to helping clinicians
to learn how to effectively use new tools in care delivery.
Multiple Choice Questions
Multiple Choice Questions
-
What attributes of an innovation influence user adoption?
-
Relative advantage, trialability, observation, accountability, complexity.
-
Observation, relative advantage, sustainability, complexity, compatibility.
-
Complexity, compatibility, observation, trialability, relative advantage.
-
Affordability, relative advantage, observation, trialability, complexity.
Correct Answer: The correct answer is option c. Although there is no order or ranking, the five DOI
theory innovation attributes include complexity, compatibility, observation, trialability,
and relative advantage.
-
When implementing new technology into health care settings, what strategy would most
likely promote its use in care delivery?
-
Training classes scheduled to meet the needs of varying shifts.
-
Reminder emails that included best practice materials.
-
Integration into existing data collection, documentation and communication.
-
Vendor provided online tutorials that includes case examples.
Correct Answer: The correct answer is option c. In general, integration into workflow is essential
for implementation of any kind of technology, evidenced-based practice, or care protocol.
While other options provide knowledge about an innovation, integration provides opportunities
to develop skills.
