Behavioral Health Decision Support Systems and User Interface Design in the Emergency Department

Abstract

Objective The objective of this qualitative study is to gauge physician sentiment about an emergency department (ED) clinical decision support (CDS) system implemented in multiple adult EDs within a university hospital system. This CDS system focuses on predicting patients' likelihood of ED recidivism and/or adverse opioid-related events.

Methods The study was conducted among adult emergency physicians working in three EDs of a single academic health system in Rhode Island. Qualitative, semistructured interviews were conducted with ED physicians. Interviews assessed physicians' prior experience with predictive analytics, thoughts on the alert's placement, design, and content, the alert's overall impact, and potential areas for improvement. Responses were aggregated and common themes identified.

Results Twenty-three interviews were conducted (11 preimplementation and 12 postimplementation). Themes were identified regarding each physician familiarity with predictive analytics, alert rollout, alert appearance and content, and on alert sentiments. Most physicians viewed these alerts as a neutral or positive EHR addition, with responses ranging from neutral to positive. The alert placement was noted to be largely intuitive and nonintrusive. The design of the alert was generally viewed positively. The alert's content was believed to be accurate, although the decision to respond to the alert's call-to-action was physician dependent. Those who tended to ignore the alert did so for a few reasons, including already knowing the information the alert contains, the alert offering information that is not relevant to this particular patient, and the alert not containing enough information to be useful.

Conclusion Ultimately, this alert appears to have a marginally positive effect on ED physician workflow. At its most beneficial, the alert reminded physicians to deeply consider the care provided to high-risk populations and to potentially adjust their care and referrals. At its least beneficial, the alert did not affect physician decision-making but was not intrusive to the point of negatively impacting workflow.

Keywords

decision support systems - clinical - emergency department - hospital - informatics - attitude - interview

Protection of Human and Animal Subjects

Study participants were deidentified prior to analysis, with identifying information contained in a password-protected file within a secure file-sharing environment. This study was reviewed and approved by the relevant Institutional Review Board. Study participants were compensated for their participation via gift card.

Publikationsverlauf

Eingereicht: 27. Februar 2023

Angenommen: 06. Juni 2023

Artikel online veröffentlicht:
06. September 2023

• References

• 1 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
• 2 Shah SN, Amato MG, Garlo KG, Seger DL, Bates DW. Renal medication-related clinical decision support (CDS) alerts and overrides in the inpatient setting following implementation of a commercial electronic health record: implications for designing more effective alerts. J Am Med Inform Assoc 2021; 28 (06) 1081-1087
• 3 Rosenthal B, Skrbin J, Fromkin J. et al. Integration of physical abuse clinical decision support at 2 general emergency departments. J Am Med Inform Assoc 2019; 26 (10) 1020-1029
• 4 Fletcher GS, Aaronson BA, White AA, Julka R. Effect of a real-time electronic dashboard on a rapid response system. J Med Syst 2017; 42 (01) 5
• 5 Whitcomb WF, Lucas JE, Tornheim R, Chiu JL, Hayward P. Association of decision support for hospital discharge disposition with outcomes. Am J Manag Care 2019; 25 (06) 288-294
• 6 Amland RC, Hahn-Cover KE. Clinical decision support for early recognition of sepsis. Am J Med Qual 2019; 34 (05) 494-501
• 7 Henry KE, Hager DN, Pronovost PJ, Saria S. A targeted real-time early warning score (TREWScore) for septic shock. Sci Transl Med 2015; 7 (299) 299ra122
• 8 Jenssen BP, Kelleher S, Karavite DJ. et al. A clinical decision support system for motivational messaging and tobacco cessation treatment for parents: pilot evaluation of use and acceptance. Appl Clin Inform 2023; 14 (03) 439-447
• 9 Shear K, Rice H, Garabedian PM. et al. Usability testing of an interoperable computerized clinical decision support tool for fall risk management in primary care. Appl Clin Inform 2023; 14 (02) 212-226
• 10 Levin S, Toerper M, Hamrock E. et al. Machine-learning-based electronic triage more accurately differentiates patients with respect to clinical outcomes compared with the emergency severity index. Ann Emerg Med 2018; 71 (05) 565-574.e2
• 11 Amarasingham R, Patzer RE, Huesch M, Nguyen NQ, Xie B. Implementing electronic health care predictive analytics: considerations and challenges. Health Aff (Millwood) 2014; 33 (07) 1148-1154
• 12 Bates DW, Saria S, Ohno-Machado L, Shah A, Escobar G. Big data in health care: using analytics to identify and manage high-risk and high-cost patients. Health Aff (Millwood) 2014; 33 (07) 1123-1131
• 13 Blackburn J, Ousey K, Goodwin E. Information and communication in the emergency department. Int Emerg Nurs 2019; 42: 30-35
• 14 Geissert P, Hallvik S, Van Otterloo J. et al. High-risk prescribing and opioid overdose: prospects for prescription drug monitoring program-based proactive alerts. Pain 2018; 159 (01) 150-156
• 15 Weiner SG, Baker O, Bernson D, Schuur JD. One-year mortality of patients after emergency department treatment for nonfatal opioid overdose. Ann Emerg Med 2020; 75 (01) 13-17
• 16 Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 2019; 366 (6464) 447-453
• 17 Manrai AK, Funke BH, Rehm HL. et al. Genetic misdiagnoses and the potential for health disparities. N Engl J Med 2016; 375 (07) 655-665
• 18 Gijsberts CM, Groenewegen KA, Hoefer IE. et al. Race/ethnic differences in the associations of the Framingham risk factors with carotid IMT and cardiovascular events. PLoS One 2015; 10 (07) e0132321
• 19 Hussain MI, Reynolds TL, Zheng K. Medication safety alert fatigue may be reduced via interaction design and clinical role tailoring: a systematic review. J Am Med Inform Assoc 2019; 26 (10) 1141-1149
• 20 Safi S, Thiessen T, Schmailzl KJ. Acceptance and resistance of new digital technologies in medicine: qualitative study. JMIR Res Protoc 2018; 7 (12) e11072
• 21 Benda NC, Das LT, Abramson EL. et al. “How did you get to this number?” Stakeholder needs for implementing predictive analytics: a pre-implementation qualitative study. J Am Med Inform Assoc 2020; 27 (05) 709-716
• 22 Jeffery AD, Novak LL, Kennedy B, Dietrich MS, Mion LC. Participatory design of probability-based decision support tools for in-hospital nurses. J Am Med Inform Assoc 2017; 24 (06) 1102-1110
• 23 Fernandes M, Vieira SM, Leite F, Palos C, Finkelstein S, Sousa JMC. Clinical decision support systems for triage in the emergency department using intelligent systems: a review. Artif Intell Med 2020; 102: 101762
• 24 Weiss A, Jiang HJ. Overview of Clinical Conditions With Frequent and Costly Hospital Readmissions by Payer, 2018. Healthcare Cost & Utilization Project: Agency for Healthcare Research and Quality (AHRQ); 2021
• 25 Soares III WE, Melnick ER, Nath B. et al. Emergency department visits for nonfatal opioid overdose during the COVID-19 pandemic across six US Health Care Systems. Ann Emerg Med 2022; 79 (02) 158-167
• 26 Shortliffe EH, Sepúlveda MJ. Clinical decision support in the era of artificial intelligence. JAMA 2018; 320 (21) 2199-2200
• 27 Nowell LS, Norris JM, White DE, Moules NJ. Thematic analysis: striving to meet the trustworthiness criteria. Int J Qual Methods 2017; 16 (01) 1-13
• 28 Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006; 3 (02) 77-101
• 29 Pourmand A, Quan T, Amini SB, Sikka N. Can emoji's assess patients' mood and emotion in the emergency department? An emoji based study. Am J Emerg Med 2020; 38 (04) 842-843
• 30 Lai D, Lee J, He S. Emoji for the medical community-challenges and opportunities. JAMA 2021; 326 (09) 795-796
• 31 Skiba DJ. Face with tears of joy is word of the year: are emoji a sign of things to come in health care?. Nurs Educ Perspect 2016; 37 (01) 56-57
• 32 Bakken S. Need for innovation in electronic health record-based medication alerts. J Am Med Inform Assoc 2019; 26 (10) 901-902
• 33 Miller SD, Murphy Z, Gray JH. et al. Human-centered design of clinical decision support for anemia screening in children with inflammatory bowel disease. Appl Clin Inform 2023; 14 (02) 345-353
• 34 Levy-Fix G, Kuperman G, Elhadad N. Machine Learning and Visualization in Clinical Decision Support: Current State and Future Directions. arXiv preprint; 2019
• 35 Chan CT, Carlson J, Lee T, Vo M, Nasr A, Hart-Cooper G. Usability and utility of human immunodeficiency virus pre-exposure prophylaxis clinical decision support to increase knowledge and pre-exposure prophylaxis initiations among pediatric providers. Appl Clin Inform 2022; 13 (05) 1141-1150