PDF icon Download PDF
Open Access
CC BY-NC-ND 4.0 · Appl Clin Inform 2025; 16(02): 377-392
DOI: 10.1055/a-2505-7743
Research Article

“Be Really Careful about That”: Clinicians' Perceptions of an Intelligence Augmentation Tool for In-Hospital Deterioration Detection

Jorie M. Butler
1   Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, Utah, United States
2   Salt Lake City VA Informatics Decision-Enhancement and Analytic Sciences (IDEAS) Center for Innovation, Geriatrics Research, Education, and Clinical Center (GRECC), VA Salt Lake City Health Care System, Salt Lake City, Utah, United States
,
Alyssa Doubleday
3   Kasiska Division of Health Sciences, College of Health, Idaho State University, Pocatello, Idaho, United States
,
Usman Sattar
1   Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, Utah, United States
,
Mary Nies
3   Kasiska Division of Health Sciences, College of Health, Idaho State University, Pocatello, Idaho, United States
,
Amanda Jeppesen
4   Kasiska Division of Health Sciences, College of Pharmacy, Idaho State University, Meridian, Idaho, United States
,
Melanie Wright
5   Tunnell Government Services, Inc., Bethesda, Maryland, United States
,
Thomas Reese
6   Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, United States
,
Kensaku Kawamoto
1   Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, Utah, United States
,
Guilherme Del Fiol
1   Department of Biomedical Informatics, School of Medicine, University of Utah, Salt Lake City, Utah, United States
,
Karl Madaras-Kelly
4   Kasiska Division of Health Sciences, College of Pharmacy, Idaho State University, Meridian, Idaho, United States
› Author Affiliations
Funding This study was funded by the U.S. Department of Health and Human Services, National Institutes of Health (grant no.: 1R01GM137083-01).
› Further Information
Also available ateRef
  Permissions and Reprints
Preview

Abstract

Objective This study aimed to explore clinicians' perceptions and preferences of prototype intelligence augmentation (IA)-based visualization displays of in-hospital deterioration risk scores to inform future user interface design and implementation in clinical care.

Methods Prototype visualization displays incorporating an IA-based early warning score (EWS) for in-hospital deterioration were developed using cognitive theory and user-centered design principles. The displays featured variations of EWS and clinical data arranged in multipatient and single-patient views. Physician and nurse participants with at least 5 years of clinical experience were recruited to participate in semistructured qualitative interviews focused on understanding their experiences with IA and thoughts and preferences about the prototype displays. A thematic analysis was performed on these data.

Results Six themes were identified: (1) clinicians perceive IA as valuable with some caveats related to function and context; (2) individual differences among users influence preferences for customizability; (3) EWS are particularly useful for patient triage; (4) need for patient-centered contextual information to complement EWS; (5) perspectives related to understanding the EWS composition; and (6) design preferences that focus on clarity for interpretation of information.

Conclusion This study demonstrates clinicians' interest in and reservations about IA tools for clinical deterioration. The findings underscore the importance of understanding clinicians' cognitive needs and framing IA-generated tools as complementary to support them. A clinician focuses on high-level pattern matching information, and clinician's comments related to the power of consistency with typical views (e.g., this is “how I usually see things”), and questions regarding support of score interpretation (e.g., age of the data, questions about what the model “knows”) suggest some of the challenges of IA implementation. The findings also identify design implications including the need for contextualizing the EWS for the patient's specific situation, incorporating trend information, and explaining the display purpose for clinical use.

Keywords

intelligence augmentation - in-hospital deterioration - patient-centered - visualization displays - artificial intelligence

Protection of Human and Animal Subjects

This project was reviewed and approved by the Institutional Review Board at Idaho State and University of Utah. Participants provided verbal consent for participation at the beginning of each session.


Note

The views expressed in this paper are those of the authors and do not necessarily represent the position or policy of the U.S. Department of Veterans Affairs or the United States Government.


Supplementary Material



Publication History

Received: 08 August 2024

Accepted: 16 December 2024

Article published online:
30 April 2025

© 2025. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Oswald-Hesse-Straße 50, 70469 Stuttgart, Germany