Diffusing an Innovation: Clinician Perceptions of Continuous Predictive Analytics Monitoring in Intensive Care

Rebecca R. Kitzmiller; Ashley Vaughan; Angela Skeeles-Worley; Jessica Keim-Malpass; Tracey L. Yap; Curt Lindberg; Susan Kennerly; Claire Mitchell; Robert Tai; Brynne A. Sullivan; Ruth Anderson; Joseph R. Moorman

doi:10.1055/s-0039-1688478

Applied Clinical Informatics, Table of Contents

CC BY-NC-ND 4.0 · Appl Clin Inform 2019; 10(02): 295-306
DOI: 10.1055/s-0039-1688478

Research Article

Georg Thieme Verlag KG Stuttgart · New York

Diffusing an Innovation: Clinician Perceptions of Continuous Predictive Analytics Monitoring in Intensive Care

Authors

Rebecca R. Kitzmiller

¹School of Nursing, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States
Ashley Vaughan

¹School of Nursing, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States
Angela Skeeles-Worley

²Curry School of Education and Human Development, University of Virginia, Charlottesville, Virginia, United States
Jessica Keim-Malpass

³School of Nursing, University of Virginia, Charlottesville, Virginia, United States
Tracey L. Yap

⁴School of Nursing, Duke University, Durham, North Carolina, United States
Curt Lindberg

⁵Billings Clinic, Billings, Montana, United States
Susan Kennerly

⁶College of Nursing, East Carolina University, Greenville, North Carolina¸ United States
Claire Mitchell

²Curry School of Education and Human Development, University of Virginia, Charlottesville, Virginia, United States
Robert Tai

²Curry School of Education and Human Development, University of Virginia, Charlottesville, Virginia, United States
Brynne A. Sullivan

⁷Division of Neonatology, University of Virginia, Charlottesville, Virginia, United States
Ruth Anderson

¹School of Nursing, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States
Joseph R. Moorman

⁸Departments of Cardiology and Biomedical Engineering, University of Virginia, Charlottesville, Virginia, United States

⁹Center for Advanced Medical Analytics, University of Virginia, Charlottesville, Virginia, United States

Abstract

Background The purpose of this article is to describe neonatal intensive care unit clinician perceptions of a continuous predictive analytics technology and how those perceptions influenced clinician adoption. Adopting and integrating new technology into care is notoriously slow and difficult; realizing expected gains remain a challenge.

Methods Semistructured interviews from a cross-section of neonatal physicians (n = 14) and nurses (n = 8) from a single U.S. medical center were collected 18 months following the conclusion of the predictive monitoring technology randomized control trial. Following qualitative descriptive analysis, innovation attributes from Diffusion of Innovation Theory-guided thematic development.

Results Results suggest that the combination of physical location as well as lack of integration into work flow or methods of using data in care decisionmaking may have delayed clinicians from routinely paying attention to the data. Once data were routinely collected, documented, and reported during patient rounds and patient handoffs, clinicians came to view data as another vital sign. Through clinicians' observation of senior physicians and nurses, and ongoing dialogue about data trends and patient status, clinicians learned how to integrate these data in care decision making (e.g., differential diagnosis) and came to value the technology as beneficial to care delivery.

Discussion The use of newly created predictive technologies that provide early warning of illness may require implementation strategies that acknowledge the risk–benefit of treatment clinicians must balance and take advantage of existing clinician training methods.

Keywords

predictive analytics - continuous predictive monitoring - neonatal intensive care - sepsis - Diffusion of Innovations - innovation attributes - implementation

Full Text

References

References
1 Agency for Healthcare Research and Quality. AHRQ Issue Brief: Harnessing the Power of Data. Agency for Healthcare Research and Quality; August 2015
2 Seely A, Newman K, Herry C. Monitoring variability and complexity at the bedside. In: Sturmberg JP. , ed. The Value of Systems and Complexity Sciences for Healthcare. New York: Springer; 2016
3 Green GC, Bradley B, Bravi A, Seely AJE. Continuous multiorgan variability analysis to track severity of organ failure in critically ill patients. J Crit Care 2013; 28 (05) 879.e1-879.e11
4 Henry KE, Hager DN, Pronovost PJ, Saria S. A targeted real-time early warning score (TREWScore) for septic shock. Sci Transl Med 2015; 7 (299) 299ra122
5 Kappen TH, van Klei WA, van Wolfswinkel L, Kalkman CJ, Vergouwe Y, Moons KGM. Evaluating the impact of prediction models: lessons learned, challenges, and recommendations. Diagnostic and Prognostic Research 2018; 2 (01) 11
6 Kennedy G, Gallego B. Clinical prediction rules: asystematic review of healthcare provider opinions and preferences. Int J Med Inform 2019; 123: 1-10
7 Nakamura F, Nakai M. Prediction models. 　- why are they used or not used?. Circ J 2017; 81 (12) 1766-1767
8 Shelov E, Muthu N, Wolfe H. , et al. Design and implementation of a pediatric ICU acuity scoring tool as clinical decision support. Appl Clin Inform 2018; 9 (03) 576-587
9 Chalmers JD, Singanayagam A, Akram AR. , et al. Severity assessment tools for predicting mortality in hospitalised patients with community-acquired pneumonia. Systematic review and meta-analysis. Thorax 2010; 65 (10) 878-883
10 Hosein FS, Bobrovitz N, Berthelot S, Zygun D, Ghali WA, Stelfox HT. A systematic review of tools for predicting severe adverse events following patient discharge from intensive care units. Crit Care 2013; 17 (03) R102
11 Breslow MJ, Badawi O. Severity scoring in the critically ill: part 1--interpretation and accuracy of outcome prediction scoring systems. Chest 2012; 141 (01) 245-252
12 van den Boogaard M, Schoonhoven L, Maseda E. , et al. Recalibration of the delirium prediction model for ICU patients (PRE-DELIRIC): a multinational observational study. Intensive Care Med 2014; 40 (03) 361-369
13 Wassenaar A, van den Boogaard M, van Achterberg T. , et al. Multinational development and validation of an early prediction model for delirium in ICU patients. Intensive Care Med 2015; 41 (06) 1048-1056
14 Seely AJ, Bravi A, Herry C. , et al; Canadian Critical Care Trials Group (CCCTG). Do heart and respiratory rate variability improve prediction of extubation outcomes in critically ill patients?. Crit Care 2014; 18 (02) R65
15 Müller-Riemenschneider F, Holmberg C, Rieckmann N. , et al. Barriers to routine risk-score use for healthy primary care patients: survey and qualitative study. Arch Intern Med 2010; 170 (08) 719-724
16 Koppel R, Wetterneck T, Telles JL, Karsh BT. Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety. J Am Med Inform Assoc 2008; 15 (04) 408-423
17 Barber N, Cornford T, Klecun E. Qualitative evaluation of an electronic prescribing and administration system. Qual Saf Health Care 2007; 16 (04) 271-278
18 Bar-Lev S, Harrison MI. Negotiating time scripts during implementation of an electronic medical record. Health Care Manage Rev 2006; 31 (01) 11-17
19 Doolin B. Power and resistance in the implementation of a medical management information system. Inf Syst J 2004; 14 (04) 343-362
20 Lapointe L, Rivard S. A multilevel model of resistance to information technology implementation. Manage Inf Syst Q 2005; 29 (03) 461-491
21 Lau F, Penn A, Wilson D, Noseworthy T, Vincent D, Doze S. The diffusion of an evidence-based disease guidance system for managing stroke. Int J Med Inform 1998; 51 (2-3): 107-116
22 Littlejohns P, Wyatt JC, Garvican L. Evaluating computerised health information systems: hard lessons still to be learnt. BMJ 2003; 326 (7394): 860-863
23 Koppel R, Metlay JP, Cohen A. , et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; 293 (10) 1197-1203
24 Novak L, Brooks J, Gadd C, Anders S, Lorenzi N. Mediating the intersections of organizational routines during the introduction of a health IT system. Eur J Inf Syst 2012; 21 (05) 552-569
25 Edmondson AC, Bohmer RM, Pisano GP. Disrupted routines: team learning and new technology implementation in hospitals. Adm Sci Q 2001; 46 (04) 685
26 Kappen TH, Moons KG, van Wolfswinkel L, Kalkman CJ, Vergouwe Y, van Klei WA. Impact of risk assessments on prophylactic antiemetic prescription and the incidence of postoperative nausea and vomiting: a cluster-randomized trial. Anesthesiology 2014; 120 (02) 343-354
27 Kappen TH, van Loon K, Kappen MA. , et al. Barriers and facilitators perceived by physicians when using prediction models in practice. J Clin Epidemiol 2016; 70: 136-145
28 Griffin MP, Moorman JR. Toward the early diagnosis of neonatal sepsis and sepsis-like illness using novel heart rate analysis. Pediatrics 2001; 107 (01) 97-104
29 Moorman JR, Carlo WA, Kattwinkel J. , et al. Mortality reduction by heart rate characteristic monitoring in very low birth weight neonates: a randomized trial. J Pediatr 2011; 159 (06) 900-906
30 Sullivan BA, Grice SM, Lake DE, Moorman JR, Fairchild KD. Infection and other clinical correlates of abnormal heart rate characteristics in preterm infants. J Pediatr 2014; 164 (04) 775-780
31 Goel N, Chakraborty M, Watkins WJ, Banerjee S. Predicting extubation outcomes-a model incorporating heart rate characteristics index. J Pediatr 2018; 195: 53-58
32 Clark MT, Vergales BD, Paget-Brown AO. , et al. Predictive monitoring for respiratory decompensation leading to urgent unplanned intubation in the neonatal intensive care unit. Pediatr Res 2013; 73 (01) 104-110
33 Yap TL, Kennerly S, Corazzini K, Porter K, Toles M, Anderson RA. Evaluation of Cueing Innovation for Pressure Ulcer Prevention Using Staff Focus Groups. Paper presented at: Healthcare 2014
34 Dharampal N, Cameron C, Dixon E, Ghali W, Quan ML. Attitudes and beliefs about the surgical safety checklist: just another tick box?. Can J Surg 2016; 59 (04) 268-275
35 Federman AD, Sanchez-Munoz A, Jandorf L, Salmon C, Wolf MS, Kannry J. Patient and clinician perspectives on the outpatient after-visit summary: a qualitative study to inform improvements in visit summary design. J Am Med Inform Assoc 2017; 24 (e1): e61-e68
36 Vedel I, Akhlaghpour S, Vaghefi I, Bergman H, Lapointe L. Health information technologies in geriatrics and gerontology: a mixed systematic review. J Am Med Inform Assoc 2013; 20 (06) 1109-1119
37 Rogers E. Diffusion of Innovations. 5th ed. New York: Free Press; 2003
38 Simunovic M, Coates A, Smith A, Thabane L, Goldsmith CH, Levine MN. Uptake of an innovation in surgery: observations from the cluster-randomized Quality Initiative in Rectal Cancer trial. Can J Surg 2013; 56 (06) 415-421
39 Mikkelsen ME, Gaieski DF, Goyal M. , et al. Factors associated with nonadherence to early goal-directed therapy in the ED. Chest 2010; 138 (03) 551-558
40 Bunkenborg G, Poulsen I, Samuelson K, Ladelund S, Åkeson J. Mandatory early warning scoring--implementation evaluated with a mixed-methods approach. Appl Nurs Res 2016; 29: 168-176
41 Pisano GP, Bohmer RM, Edmondson AC. Organizational differences in rates of learning: evidence from the adoption of minimally invasive cardiac surgery. Manage Sci 2001; 47 (06) 752-768
42 Sandelowski M. Whatever happened to qualitative description?. Res Nurs Health 2000; 23 (04) 334-340
43 Hinds PS, Vogel RJ, Clarke-Steffen L. The possibilities and pitfalls of doing a secondary analysis of a qualitative data set. Qual Health Res 1997; 7 (03) 408-424
44 Szabo V, Strang VR. Secondary analysis of qualitative data. ANS Adv Nurs Sci 1997; 20 (02) 66-74
45 Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005; 15 (09) 1277-1288
46 Miles MB, Huberman AM, Saldaña J. Qualitative Data Analysis: A Methods Sourcebook. Thousand Oaks, CA: SAGE Publications, Incorporated; 2013
47 Griffin MP, Lake DE, Bissonette EA, Harrell Jr FE, O'Shea TM, Moorman JR. Heart rate characteristics: novel physiomarkers to predict neonatal infection and death. Pediatrics 2005; 116 (05) 1070-1074
48 Ford EW, McAlearney AS, Phillips MT, Menachemi N, Rudolph B. Predicting computerized physician order entry system adoption in US hospitals: can the federal mandate be met?. Int J Med Inform 2008; 77 (08) 539-545
49 Gardner-Thorpe J, Love N, Wrightson J, Walsh S, Keeling N. The value of Modified Early Warning Score (MEWS) in surgical in-patients: a prospective observational study. Ann R Coll Surg Engl 2006; 88 (06) 571-575
50 Priestley G, Watson W, Rashidian A. , et al. Introducing critical care outreach: a ward-randomised trial of phased introduction in a general hospital. Intensive Care Med 2004; 30 (07) 1398-1404
51 Guilbert ER, Robitaille J, Guilbert AC, Morin D. Determinants of the implementation of a new practice in hormonal contraception by Quebec nurses. Can J Hum Sex 2014; 23 (01) 34-48
52 Weick KE, Sutcliffe KM, Obstfeld D. Organizing and the process of sensemaking. Organ Sci 2005; 16 (04) 409
53 Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004; 82 (04) 581-629
54 Rehr CA, Wong A, Seger DL, Bates DW. Determining inappropriate medication alerts from “inaccurate warning” overrides in the intensive care unit. Appl Clin Inform 2018; 9 (02) 268-274
55 Moss TJ, Lake DE, Calland JF. , et al. Signatures of subacute potentially catastrophic illness in the ICU: model development and validation. Crit Care Med 2016; 44 (09) 1639-1648
56 Weick KE. Sensemaking in Organizations. London: Sage; 1995
57 Norman G. Research in clinical reasoning: past history and current trends. Med Educ 2005; 39 (04) 418-427
58 Cruess SR, Cruess RL, Steinert Y. Role modelling--making the most of a powerful teaching strategy. BMJ 2008; 336 (7646): 718-721
59 Masterson Creber RM, Dayan PS, Kuppermann N. , et al; Pediatric Emergency Care Applied Research Network (PECARN) and the Clinical Research on Emergency Services and Treatments (CREST) Network. Applying the RE-AIM framework for the evaluation of a clinical decision support tool for pediatric head trauma: a mixed-methods study. Appl Clin Inform 2018; 9 (03) 693-703
60 Menachemi N, Burke DE, Ayers DJ. Factors affecting the adoption of telemedicine--a multiple adopter perspective. J Med Syst 2004; 28 (06) 617-632
61 Fitzgerald L, Ferlie E, Wood M, Hawkins C. Interlocking interactions, the diffusion of innovations in health care. Hum Relat 2002; 55 (12) 1429
62 Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM 2001; 94 (10) 521-526
63 Heitz CR, Gaillard JP, Blumstein H, Case D, Messick C, Miller CD. Performance of the maximum modified early warning score to predict the need for higher care utilization among admitted emergency department patients. J Hosp Med 2010; 5 (01) E46-E52
64 Keim-Malpass J, Kitzmiller RR, Skeeles-Worley A. , et al. Advancing continuous predictive analytics monitoring: Moving from implementation to clinical action in a learning health system. Crit Care Nurs Clin North Am 2018; 30 (02) 273-287