Download PDF
Appl Clin Inform 2025; 16(01): 223-233
DOI: 10.1055/a-2451-9046
Research Article

Exploring Stakeholder Perceptions about Using Artificial Intelligence for the Diagnosis of Rare and Atypical Infections

Aysun Tekin
1   Division of Nephrology and Hypertension, Department of Internal Medicine, Mayo Clinic, Rochester, Minnesota, United States
,
Svetlana Herasevich
2   Department of Anesthesiology and Perioperative Medicine, Mayo Clinic, Rochester, Minnesota, United States
,
Sarah A. Minteer
3   Department of Physical Medicine and Rehabilitation Research, Mayo Clinic, Rochester, Minnesota, United States
,
Ognjen Gajic
4   Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Mayo Clinic, Rochester, Minnesota, United States
,
Amelia K. Barwise
4   Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Mayo Clinic, Rochester, Minnesota, United States
› Author Affiliations Funding This publication was supported by Grant Number UL1 TR002377 from the National Center for Advancing Translational Sciences (NCATS). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.
This work is supported by benefactor funding via the Mayo Clinic Rochester Division of Pulmonary and Critical Care Medicine (O.G.) and by the Minnesota Partnership for Biotechnology and Medical Genomics (P008848012; O.G.).
A.K.B. receives funding from the Agency for Healthcare Research and Quality (AHRQ; R21 HS028475).
The funders had no influence on the study design, data collection, analysis, interpretation, and reporting of pooled data.
› Further Information
Also available at
    Permissions and Reprints

Abstract

Objectives This study aimed to evaluate critical care provider perspectives about diagnostic practices for rare and atypical infections and the potential for using artificial intelligence (AI) as a decision support system (DSS).

Methods We conducted an anonymous web-based survey among critical care providers at Mayo Clinic Rochester between November 25, 2023, and January 15, 2024, to evaluate their experience with rare and atypical infection diagnostic processes and AI-based DSSs. We also assessed the perceived usefulness of AI-based DSSs, their potential impact on improving diagnostic practices for rare and atypical infections, and the perceived risks and benefits of their use.

Results A total of 47/143 providers completed the survey. Thirty-eight out of 47 agreed that there was a delay in diagnosing rare and atypical infections. Among those who agreed, limited assessment of specific patient factors and failure to consider them were the most frequently cited important contributing factors (33/38). Thirty-eight out of 47 reported familiarity with the AI-based DSS applications available to critical care providers. Less than half (18/38) thought AI-based DSSs often provided valuable insights into patient care, but almost three-quarters (34/47) thought AI-based DDSs often provided valuable insight when specifically asked about their ability to improve the diagnosis of rare and atypical infections. All respondents rated reliability as important in enhancing the perceived utility of AI-based DSSs (47/47) and almost all rated interpretability and integration into the workflow as important (45/47). The primary concern about implementing an AI-based DSS in this context was alert fatigue (44/47).

Conclusion Most critical care providers perceive that there are delays in diagnosing rare infections, indicating inadequate assessment and consideration of the diagnosis as the major contributors. Reliability, interpretability, workflow integration, and alert fatigue emerged as key factors impacting the usability of AI-based DSS. These findings will inform the development and implementation of an AI-based diagnostic algorithm to aid in identifying rare and atypical infections.

Keywords

artificial intelligence - atypical infections - critical care providers - decision support system - diagnostic delay

Protection of Human and Animal Subjects

The Mayo Clinic Institutional Review Board reviewed and approved the survey study as minimal risk (#22-009881). The survey instructions stated that continuing the survey would constitute implied consent.


Supplementary Material



Publication History

Received: 16 July 2024

Accepted: 24 October 2024

Accepted Manuscript online:
25 October 2024

Article published online:
05 March 2025

© 2025. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Redmond S, Barwise A, Zornes S. et al. Contributors to diagnostic error or delay in the acute care setting: a survey of clinical stakeholders. Health Serv Insights 2022; 15: 11 786329221123540
    PubMedSearch in Google Scholar
  • 2 Huang C, Barwise A, Soleimani J. et al. Bedside clinicians' perceptions on the contributing role of diagnostic errors in acutely ill patient presentation: a survey of academic and community practice. J Patient Saf 2022; 18 (02) e454-e462
    CrossrefPubMedSearch in Google Scholar
  • 3 van der Meijden SL, de Hond AAH, Thoral PJ. et al. Intensive care unit physicians' perspectives on artificial intelligence-based clinical decision support tools: preimplementation survey study. JMIR Hum Factors 2023; 10: e39114
    CrossrefPubMedSearch in Google Scholar
  • 4 Winters B, Custer J, Galvagno Jr SM. et al. Diagnostic errors in the intensive care unit: a systematic review of autopsy studies. BMJ Qual Saf 2012; 21 (11) 894-902
    CrossrefPubMedSearch in Google Scholar
  • 5 Martynowicz MA, Prakash UB. Pulmonary blastomycosis: an appraisal of diagnostic techniques. Chest 2002; 121 (03) 768-773
    CrossrefPubMedSearch in Google Scholar
  • 6 O'Dowd TR, Mc Hugh JW, Theel ES. et al. Diagnostic methods and risk factors for severe disease and mortality in blastomycosis: a retrospective cohort study. J Fungi (Basel) 2021; 7 (11) 888
    CrossrefPubMedSearch in Google Scholar
  • 7 Richeldi L. An update on the diagnosis of tuberculosis infection. Am J Respir Crit Care Med 2006; 174 (07) 736-742
    CrossrefPubMedSearch in Google Scholar
  • 8 Suneja M, Beekmann SE, Dhaliwal G, Miller AC, Polgreen PM. Diagnostic delays in infectious diseases. Diagnosis (Berl) 2022; 20: 332-339
    CrossrefPubMedSearch in Google Scholar
  • 9 Tekin A, Pinevich Y, Herasevich V. et al. Diagnostic delay in pulmonary blastomycosis: a case series reflecting a referral center experience. Infection 2022; 51: 193-201
    CrossrefPubMedSearch in Google Scholar
  • 10 Peterson ED. Machine learning, predictive analytics, and clinical practice: can the past inform the present?. JAMA 2019; 322 (23) 2283-2284
    CrossrefPubMedSearch in Google Scholar
  • 11 Lin T, Zhang X, Gong J. et al. A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients. BMC Med Inform Decis Mak 2023; 23 (01) 81
    CrossrefPubMedSearch in Google Scholar
  • 12 Xu Z, Guo J, Qin L. et al. Predicting ICU interventions: a transparent decision support model based on multivariate time series graph convolutional neural network. IEEE J Biomed Health Inform 2024; 28 (06) 3709-3720
    CrossrefPubMedSearch in Google Scholar
  • 13 Pinsky MR, Bedoya A, Bihorac A. et al. Use of artificial intelligence in critical care: opportunities and obstacles. Crit Care 2024; 28 (01) 113
    CrossrefPubMedSearch in Google Scholar
  • 14 VanGeest JB, Johnson TP, Welch VL. Methodologies for improving response rates in surveys of physicians: a systematic review. Eval Health Prof 2007; 30 (04) 303-321
    CrossrefPubMedSearch in Google Scholar
  • 15 Barwise AK, Curtis S, Diedrich DA, Pickering BW. Using artificial intelligence to promote equitable care for inpatients with language barriers and complex medical needs: clinical stakeholder perspectives. J Am Med Inform Assoc 2024; 31 (03) 611-621
    CrossrefPubMedSearch in Google Scholar
  • 16 van de Sande D, Van Genderen ME, Smit JM. et al. Developing, implementing and governing artificial intelligence in medicine: a step-by-step approach to prevent an artificial intelligence winter. BMJ Health Care Inform 2022; 29 (01) e100495
    CrossrefPubMedSearch in Google Scholar
  • 17 Murphree DH, Quest DJ, Allen RM, Ngufor C, Storlie CB. Deploying predictive models in a healthcare environment - an open source approach. Annu Int Conf IEEE Eng Med Biol Soc 2018; 2018: 6112-6116
    PubMedSearch in Google Scholar
  • 18 Tekin A, Joghataee M, Rovati L. et al. Development and validation of a preliminary multivariable diagnostic model for identifying unusual infections in hospitalized patients. Biomol Biomed 2024; 24 (05) 1387-1399
    CrossrefPubMedSearch in Google Scholar
  • 19 Oh S, Kim JH, Choi SW, Lee HJ, Hong J, Kwon SH. Physician confidence in artificial intelligence: an online mobile survey. J Med Internet Res 2019; 21 (03) e12422
    CrossrefPubMedSearch in Google Scholar
  • 20 Benda NC, Das LT, Abramson EL. et al. “How did you get to this number?” Stakeholder needs for implementing predictive analytics: a pre-implementation qualitative study. J Am Med Inform Assoc 2020; 27 (05) 709-716
    CrossrefPubMedSearch in Google Scholar
  • 21 Abdullah R, Fakieh B. Health care employees' perceptions of the use of artificial intelligence applications: survey study. J Med Internet Res 2020; 22 (05) e17620
    CrossrefPubMedSearch in Google Scholar
  • 22 Castagno S, Khalifa M. Perceptions of artificial intelligence among healthcare staff: a qualitative survey study. Front Artif Intell 2020; 3: 578983
    CrossrefPubMedSearch in Google Scholar
  • 23 Chen M, Zhang B, Cai Z. et al. Acceptance of clinical artificial intelligence among physicians and medical students: a systematic review with cross-sectional survey. Front Med (Lausanne) 2022; 9: 990604
    CrossrefPubMedSearch in Google Scholar
  • 24 Barwise A, Caples S, Jensen J, Pickering B, Herasevich V. Information needs for the rapid response team electronic clinical tool. BMC Med Inform Decis Mak 2017; 17 (01) 142
    CrossrefPubMedSearch in Google Scholar
  • 25 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
    CrossrefPubMedSearch in Google Scholar
  • 26 Alpern JD, Bahr NC, Vazquez-Benitez G, Boulware DR, Sellman JS, Sarosi GA. Diagnostic delay and antibiotic overuse in acute pulmonary blastomycosis. Open Forum Infect Dis 2016; 3 (02) ofw078
    CrossrefPubMedSearch in Google Scholar
  • 27 Benedict K, Gibbons-Burgener S, Kocharian A. et al. Blastomycosis surveillance in 5 states, United States, 1987-2018. Emerg Infect Dis 2021; 27 (04) 999-1006
    CrossrefPubMedSearch in Google Scholar
  • 28 Dworkin MS, Duckro AN, Proia L, Semel JD, Huhn G. The epidemiology of blastomycosis in Illinois and factors associated with death. Clin Infect Dis 2005; 41 (12) e107-e111
    CrossrefPubMedSearch in Google Scholar
  • 29 Ireland M, Klumb C, Smith K, Scheftel J. Blastomycosis in Minnesota, USA, 1999-2018. Emerg Infect Dis 2020; 26 (05) 866-875
    CrossrefPubMedSearch in Google Scholar
  • 30 Barwise A, Leppin A, Dong Y. et al. What contributes to diagnostic error or delay? A qualitative exploration across diverse acute care settings in the United States. J Patient Saf 2021; 17 (04) 239-248
    CrossrefPubMedSearch in Google Scholar
  • 31 Estrada Alamo CE, Diatta F, Monsell SE, Lane-Fall MB. Artificial intelligence in anesthetic care: a survey of physician anesthesiologists. Anesth Analg 2024; 138: 938-950
    CrossrefPubMedSearch in Google Scholar
  • 32 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
    CrossrefPubMedSearch in Google Scholar
  • 33 Naik H, Murray TM, Khan M. et al. Population-based trends in complexity of hospital inpatients. JAMA Intern Med 2024; 184 (02) 183-192
    CrossrefPubMedSearch in Google Scholar
  • 34 Ploug T, Sundby A, Moeslund TB, Holm S. Population preferences for performance and explainability of artificial intelligence in health care: choice-based conjoint survey. J Med Internet Res 2021; 23 (12) e26611
    CrossrefPubMedSearch in Google Scholar
  • 35 Choudhury A. Factors influencing clinicians' willingness to use an AI-based clinical decision support system. Front Digit Health 2022; 4: 920662
    CrossrefPubMedSearch in Google Scholar
  • 36 Liang H, Tsui BY, Ni H. et al. Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence. Nat Med 2019; 25 (03) 433-438
    CrossrefPubMedSearch in Google Scholar
  • 37 Blease C, Kaptchuk TJ, Bernstein MH, Mandl KD, Halamka JD, DesRoches CM. Artificial intelligence and the future of primary care: exploratory qualitative study of UK general practitioners' views. J Med Internet Res 2019; 21 (03) e12802
    CrossrefPubMedSearch in Google Scholar
  • 38 Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R. with the HITEC Investigators. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak 2017; 17 (01) 36
    CrossrefPubMedSearch in Google Scholar
  • 39 Murad DA, Tsugawa Y, Elashoff DA, Baldwin KM, Bell DS. Distinct components of alert fatigue in physicians' responses to a noninterruptive clinical decision support alert. J Am Med Inform Assoc 2022; 30 (01) 64-72
    CrossrefPubMedSearch in Google Scholar
  • 40 Fernandes CO, Miles S, Lucena CJP, Cowan D. Artificial intelligence technologies for coping with alarm fatigue in hospital environments because of sensory overload: algorithm development and validation. J Med Internet Res 2019; 21 (11) e15406
    CrossrefPubMedSearch in Google Scholar
  • 41 Hussain MI, Reynolds TL, Zheng K. Medication safety alert fatigue may be reduced via interaction design and clinical role tailoring: a systematic review. J Am Med Inform Assoc 2019; 26 (10) 1141-1149
    CrossrefPubMedSearch in Google Scholar
  • 42 Jabbour S, Fouhey D, Shepard S. et al. Measuring the impact of AI in the diagnosis of hospitalized patients: a randomized clinical vignette survey study. JAMA 2023; 330 (23) 2275-2284
    CrossrefPubMedSearch in Google Scholar
  • 43 Chauhan C, Gullapalli RR. Ethics of AI in pathology: current paradigms and emerging issues. Am J Pathol 2021; 191 (10) 1673-1683
    CrossrefPubMedSearch in Google Scholar
  • 44 Ouanes K, Farhah N. Effectiveness of artificial intelligence (AI) in clinical decision support systems and care delivery. J Med Syst 2024; 48 (01) 74
    CrossrefPubMedSearch in Google Scholar
  • 45 Lam TYT, Cheung MFK, Munro YL, Lim KM, Shung D, Sung JJY. Randomized controlled trials of artificial intelligence in clinical practice: systematic review. J Med Internet Res 2022; 24 (08) e37188
    CrossrefPubMedSearch in Google Scholar
  • 46 Kwan JL, Lo L, Ferguson J. et al. Computerised clinical decision support systems and absolute improvements in care: meta-analysis of controlled clinical trials. BMJ 2020; 370: m3216
    CrossrefPubMedSearch in Google Scholar
  • 47 Harutyunyan R, Jeffries SD, Ramírez-GarcíaLuna JL, Hemmerling TM. Clinical performance of decision support systems in anesthesia, intensive care, and emergency medicine: a systematic review and meta-analysis. Anesth Analg 2023; 136 (06) 1084-1095
    CrossrefPubMedSearch in Google Scholar
  • 48 Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med 2019; 17 (01) 195
    CrossrefPubMedSearch in Google Scholar
  • 49 Shaw J, Rudzicz F, Jamieson T, Goldfarb A. Artificial intelligence and the implementation challenge. J Med Internet Res 2019; 21 (07) e13659
    CrossrefPubMedSearch in Google Scholar
  • 50 Esmaeilzadeh P. Challenges and strategies for wide-scale artificial intelligence (AI) deployment in healthcare practices: a perspective for healthcare organizations. Artif Intell Med 2024; 151: 102861
    CrossrefPubMedSearch in Google Scholar
  • 51 Vo V, Chen G, Aquino YSJ, Carter SM, Do QN, Woode ME. Multi-stakeholder preferences for the use of artificial intelligence in healthcare: a systematic review and thematic analysis. Soc Sci Med 2023; 338: 116357
    CrossrefPubMedSearch in Google Scholar
  • 52 Meyer VM, Benjamens S, Moumni ME, Lange JFM, Pol RA. Global overview of response rates in patient and health care professional surveys in surgery: a systematic review. Ann Surg 2022; 275 (01) e75-e81
    CrossrefPubMedSearch in Google Scholar
  • 53 Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol 1997; 50 (10) 1129-1136
    CrossrefPubMedSearch in Google Scholar
  • 54 Leece P, Bhandari M, Sprague S. et al. Internet versus mailed questionnaires: a controlled comparison (2). J Med Internet Res 2004; 6 (04) e39
    CrossrefPubMedSearch in Google Scholar
  • 55 Kim HL, Gerber GS, Patel RV, Hollowell CM, Bales GT. Practice patterns in the treatment of female urinary incontinence: a postal and internet survey. Urology 2001; 57 (01) 45-48
    CrossrefPubMedSearch in Google Scholar
  • 56 McMahon SR, Iwamoto M, Massoudi MS. et al. Comparison of e-mail, fax, and postal surveys of pediatricians. Pediatrics 2003; 111: e299-e303
    CrossrefPubMedSearch in Google Scholar
  • 57 Raziano DB, Jayadevappa R, Valenzula D, Weiner M, Lavizzo-Mourey R. E-mail versus conventional postal mail survey of geriatric chiefs. Gerontologist 2001; 41 (06) 799-804
    CrossrefPubMedSearch in Google Scholar
  • 58 Crouch S, Robinson P, Pitts M. A comparison of general practitioner response rates to electronic and postal surveys in the setting of the National STI Prevention Program. Aust N Z J Public Health 2011; 35 (02) 187-189
    CrossrefPubMedSearch in Google Scholar
  • 59 Edwards PJ, Roberts I, Clarke MJ. et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009; 2009 (03) MR000008
    PubMedSearch in Google Scholar
  • 60 Herasevich S, Pinevich Y, Lipatov K. et al. Evaluation of digital health strategy to support clinician-led critically ill patient population management: a randomized crossover study. Crit Care Explor 2023; 5 (05) e0909
    CrossrefPubMedSearch in Google Scholar
  • 61 Boswell CL, Minteer SA, Herasevich S. et al. Early prevention of critical illness in older adults: adaptation and pilot testing of an electronic risk score and checklist. J Prim Care Community Health 2024; 15: 21 501319241231238
    CrossrefPubMedSearch in Google Scholar