Subscribe to RSS
DOI: 10.1055/a-2451-9046
Exploring Stakeholder Perceptions about Using Artificial Intelligence for the Diagnosis of Rare and Atypical Infections
Funding This publication was supported by Grant Number UL1 TR002377 from the National Center for Advancing Translational Sciences (NCATS). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the NIH.This work is supported by benefactor funding via the Mayo Clinic Rochester Division of Pulmonary and Critical Care Medicine (O.G.) and by the Minnesota Partnership for Biotechnology and Medical Genomics (P008848012; O.G.).
A.K.B. receives funding from the Agency for Healthcare Research and Quality (AHRQ; R21 HS028475).
The funders had no influence on the study design, data collection, analysis, interpretation, and reporting of pooled data.

Abstract
Objectives This study aimed to evaluate critical care provider perspectives about diagnostic practices for rare and atypical infections and the potential for using artificial intelligence (AI) as a decision support system (DSS).
Methods We conducted an anonymous web-based survey among critical care providers at Mayo Clinic Rochester between November 25, 2023, and January 15, 2024, to evaluate their experience with rare and atypical infection diagnostic processes and AI-based DSSs. We also assessed the perceived usefulness of AI-based DSSs, their potential impact on improving diagnostic practices for rare and atypical infections, and the perceived risks and benefits of their use.
Results A total of 47/143 providers completed the survey. Thirty-eight out of 47 agreed that there was a delay in diagnosing rare and atypical infections. Among those who agreed, limited assessment of specific patient factors and failure to consider them were the most frequently cited important contributing factors (33/38). Thirty-eight out of 47 reported familiarity with the AI-based DSS applications available to critical care providers. Less than half (18/38) thought AI-based DSSs often provided valuable insights into patient care, but almost three-quarters (34/47) thought AI-based DDSs often provided valuable insight when specifically asked about their ability to improve the diagnosis of rare and atypical infections. All respondents rated reliability as important in enhancing the perceived utility of AI-based DSSs (47/47) and almost all rated interpretability and integration into the workflow as important (45/47). The primary concern about implementing an AI-based DSS in this context was alert fatigue (44/47).
Conclusion Most critical care providers perceive that there are delays in diagnosing rare infections, indicating inadequate assessment and consideration of the diagnosis as the major contributors. Reliability, interpretability, workflow integration, and alert fatigue emerged as key factors impacting the usability of AI-based DSS. These findings will inform the development and implementation of an AI-based diagnostic algorithm to aid in identifying rare and atypical infections.
Keywords
artificial intelligence - atypical infections - critical care providers - decision support system - diagnostic delayProtection of Human and Animal Subjects
The Mayo Clinic Institutional Review Board reviewed and approved the survey study as minimal risk (#22-009881). The survey instructions stated that continuing the survey would constitute implied consent.
Publication History
Received: 16 July 2024
Accepted: 24 October 2024
Accepted Manuscript online:
25 October 2024
Article published online:
05 March 2025
© 2025. Thieme. All rights reserved.
Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany
-
References
- 1 Redmond S, Barwise A, Zornes S. et al. Contributors to diagnostic error or delay in the acute care setting: a survey of clinical stakeholders. Health Serv Insights 2022; 15: 11 786329221123540
- 2 Huang C, Barwise A, Soleimani J. et al. Bedside clinicians' perceptions on the contributing role of diagnostic errors in acutely ill patient presentation: a survey of academic and community practice. J Patient Saf 2022; 18 (02) e454-e462
- 3 van der Meijden SL, de Hond AAH, Thoral PJ. et al. Intensive care unit physicians' perspectives on artificial intelligence-based clinical decision support tools: preimplementation survey study. JMIR Hum Factors 2023; 10: e39114
- 4 Winters B, Custer J, Galvagno Jr SM. et al. Diagnostic errors in the intensive care unit: a systematic review of autopsy studies. BMJ Qual Saf 2012; 21 (11) 894-902
- 5 Martynowicz MA, Prakash UB. Pulmonary blastomycosis: an appraisal of diagnostic techniques. Chest 2002; 121 (03) 768-773
- 6 O'Dowd TR, Mc Hugh JW, Theel ES. et al. Diagnostic methods and risk factors for severe disease and mortality in blastomycosis: a retrospective cohort study. J Fungi (Basel) 2021; 7 (11) 888
- 7 Richeldi L. An update on the diagnosis of tuberculosis infection. Am J Respir Crit Care Med 2006; 174 (07) 736-742
- 8 Suneja M, Beekmann SE, Dhaliwal G, Miller AC, Polgreen PM. Diagnostic delays in infectious diseases. Diagnosis (Berl) 2022; 20: 332-339
- 9 Tekin A, Pinevich Y, Herasevich V. et al. Diagnostic delay in pulmonary blastomycosis: a case series reflecting a referral center experience. Infection 2022; 51: 193-201
- 10 Peterson ED. Machine learning, predictive analytics, and clinical practice: can the past inform the present?. JAMA 2019; 322 (23) 2283-2284
- 11 Lin T, Zhang X, Gong J. et al. A dosing strategy model of deep deterministic policy gradient algorithm for sepsis patients. BMC Med Inform Decis Mak 2023; 23 (01) 81
- 12 Xu Z, Guo J, Qin L. et al. Predicting ICU interventions: a transparent decision support model based on multivariate time series graph convolutional neural network. IEEE J Biomed Health Inform 2024; 28 (06) 3709-3720
- 13 Pinsky MR, Bedoya A, Bihorac A. et al. Use of artificial intelligence in critical care: opportunities and obstacles. Crit Care 2024; 28 (01) 113
- 14 VanGeest JB, Johnson TP, Welch VL. Methodologies for improving response rates in surveys of physicians: a systematic review. Eval Health Prof 2007; 30 (04) 303-321
- 15 Barwise AK, Curtis S, Diedrich DA, Pickering BW. Using artificial intelligence to promote equitable care for inpatients with language barriers and complex medical needs: clinical stakeholder perspectives. J Am Med Inform Assoc 2024; 31 (03) 611-621
- 16 van de Sande D, Van Genderen ME, Smit JM. et al. Developing, implementing and governing artificial intelligence in medicine: a step-by-step approach to prevent an artificial intelligence winter. BMJ Health Care Inform 2022; 29 (01) e100495
- 17 Murphree DH, Quest DJ, Allen RM, Ngufor C, Storlie CB. Deploying predictive models in a healthcare environment - an open source approach. Annu Int Conf IEEE Eng Med Biol Soc 2018; 2018: 6112-6116
- 18 Tekin A, Joghataee M, Rovati L. et al. Development and validation of a preliminary multivariable diagnostic model for identifying unusual infections in hospitalized patients. Biomol Biomed 2024; 24 (05) 1387-1399
- 19 Oh S, Kim JH, Choi SW, Lee HJ, Hong J, Kwon SH. Physician confidence in artificial intelligence: an online mobile survey. J Med Internet Res 2019; 21 (03) e12422
- 20 Benda NC, Das LT, Abramson EL. et al. “How did you get to this number?” Stakeholder needs for implementing predictive analytics: a pre-implementation qualitative study. J Am Med Inform Assoc 2020; 27 (05) 709-716
- 21 Abdullah R, Fakieh B. Health care employees' perceptions of the use of artificial intelligence applications: survey study. J Med Internet Res 2020; 22 (05) e17620
- 22 Castagno S, Khalifa M. Perceptions of artificial intelligence among healthcare staff: a qualitative survey study. Front Artif Intell 2020; 3: 578983
- 23 Chen M, Zhang B, Cai Z. et al. Acceptance of clinical artificial intelligence among physicians and medical students: a systematic review with cross-sectional survey. Front Med (Lausanne) 2022; 9: 990604
- 24 Barwise A, Caples S, Jensen J, Pickering B, Herasevich V. Information needs for the rapid response team electronic clinical tool. BMC Med Inform Decis Mak 2017; 17 (01) 142
- 25 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
- 26 Alpern JD, Bahr NC, Vazquez-Benitez G, Boulware DR, Sellman JS, Sarosi GA. Diagnostic delay and antibiotic overuse in acute pulmonary blastomycosis. Open Forum Infect Dis 2016; 3 (02) ofw078
- 27 Benedict K, Gibbons-Burgener S, Kocharian A. et al. Blastomycosis surveillance in 5 states, United States, 1987-2018. Emerg Infect Dis 2021; 27 (04) 999-1006
- 28 Dworkin MS, Duckro AN, Proia L, Semel JD, Huhn G. The epidemiology of blastomycosis in Illinois and factors associated with death. Clin Infect Dis 2005; 41 (12) e107-e111
- 29 Ireland M, Klumb C, Smith K, Scheftel J. Blastomycosis in Minnesota, USA, 1999-2018. Emerg Infect Dis 2020; 26 (05) 866-875
- 30 Barwise A, Leppin A, Dong Y. et al. What contributes to diagnostic error or delay? A qualitative exploration across diverse acute care settings in the United States. J Patient Saf 2021; 17 (04) 239-248
- 31 Estrada Alamo CE, Diatta F, Monsell SE, Lane-Fall MB. Artificial intelligence in anesthetic care: a survey of physician anesthesiologists. Anesth Analg 2024; 138: 938-950
- 32 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
- 33 Naik H, Murray TM, Khan M. et al. Population-based trends in complexity of hospital inpatients. JAMA Intern Med 2024; 184 (02) 183-192
- 34 Ploug T, Sundby A, Moeslund TB, Holm S. Population preferences for performance and explainability of artificial intelligence in health care: choice-based conjoint survey. J Med Internet Res 2021; 23 (12) e26611
- 35 Choudhury A. Factors influencing clinicians' willingness to use an AI-based clinical decision support system. Front Digit Health 2022; 4: 920662
- 36 Liang H, Tsui BY, Ni H. et al. Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence. Nat Med 2019; 25 (03) 433-438
- 37 Blease C, Kaptchuk TJ, Bernstein MH, Mandl KD, Halamka JD, DesRoches CM. Artificial intelligence and the future of primary care: exploratory qualitative study of UK general practitioners' views. J Med Internet Res 2019; 21 (03) e12802
- 38 Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R. with the HITEC Investigators. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak 2017; 17 (01) 36
- 39 Murad DA, Tsugawa Y, Elashoff DA, Baldwin KM, Bell DS. Distinct components of alert fatigue in physicians' responses to a noninterruptive clinical decision support alert. J Am Med Inform Assoc 2022; 30 (01) 64-72
- 40 Fernandes CO, Miles S, Lucena CJP, Cowan D. Artificial intelligence technologies for coping with alarm fatigue in hospital environments because of sensory overload: algorithm development and validation. J Med Internet Res 2019; 21 (11) e15406
- 41 Hussain MI, Reynolds TL, Zheng K. Medication safety alert fatigue may be reduced via interaction design and clinical role tailoring: a systematic review. J Am Med Inform Assoc 2019; 26 (10) 1141-1149
- 42 Jabbour S, Fouhey D, Shepard S. et al. Measuring the impact of AI in the diagnosis of hospitalized patients: a randomized clinical vignette survey study. JAMA 2023; 330 (23) 2275-2284
- 43 Chauhan C, Gullapalli RR. Ethics of AI in pathology: current paradigms and emerging issues. Am J Pathol 2021; 191 (10) 1673-1683
- 44 Ouanes K, Farhah N. Effectiveness of artificial intelligence (AI) in clinical decision support systems and care delivery. J Med Syst 2024; 48 (01) 74
- 45 Lam TYT, Cheung MFK, Munro YL, Lim KM, Shung D, Sung JJY. Randomized controlled trials of artificial intelligence in clinical practice: systematic review. J Med Internet Res 2022; 24 (08) e37188
- 46 Kwan JL, Lo L, Ferguson J. et al. Computerised clinical decision support systems and absolute improvements in care: meta-analysis of controlled clinical trials. BMJ 2020; 370: m3216
- 47 Harutyunyan R, Jeffries SD, Ramírez-GarcíaLuna JL, Hemmerling TM. Clinical performance of decision support systems in anesthesia, intensive care, and emergency medicine: a systematic review and meta-analysis. Anesth Analg 2023; 136 (06) 1084-1095
- 48 Kelly CJ, Karthikesalingam A, Suleyman M, Corrado G, King D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med 2019; 17 (01) 195
- 49 Shaw J, Rudzicz F, Jamieson T, Goldfarb A. Artificial intelligence and the implementation challenge. J Med Internet Res 2019; 21 (07) e13659
- 50 Esmaeilzadeh P. Challenges and strategies for wide-scale artificial intelligence (AI) deployment in healthcare practices: a perspective for healthcare organizations. Artif Intell Med 2024; 151: 102861
- 51 Vo V, Chen G, Aquino YSJ, Carter SM, Do QN, Woode ME. Multi-stakeholder preferences for the use of artificial intelligence in healthcare: a systematic review and thematic analysis. Soc Sci Med 2023; 338: 116357
- 52 Meyer VM, Benjamens S, Moumni ME, Lange JFM, Pol RA. Global overview of response rates in patient and health care professional surveys in surgery: a systematic review. Ann Surg 2022; 275 (01) e75-e81
- 53 Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol 1997; 50 (10) 1129-1136
- 54 Leece P, Bhandari M, Sprague S. et al. Internet versus mailed questionnaires: a controlled comparison (2). J Med Internet Res 2004; 6 (04) e39
- 55 Kim HL, Gerber GS, Patel RV, Hollowell CM, Bales GT. Practice patterns in the treatment of female urinary incontinence: a postal and internet survey. Urology 2001; 57 (01) 45-48
- 56 McMahon SR, Iwamoto M, Massoudi MS. et al. Comparison of e-mail, fax, and postal surveys of pediatricians. Pediatrics 2003; 111: e299-e303
- 57 Raziano DB, Jayadevappa R, Valenzula D, Weiner M, Lavizzo-Mourey R. E-mail versus conventional postal mail survey of geriatric chiefs. Gerontologist 2001; 41 (06) 799-804
- 58 Crouch S, Robinson P, Pitts M. A comparison of general practitioner response rates to electronic and postal surveys in the setting of the National STI Prevention Program. Aust N Z J Public Health 2011; 35 (02) 187-189
- 59 Edwards PJ, Roberts I, Clarke MJ. et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009; 2009 (03) MR000008
- 60 Herasevich S, Pinevich Y, Lipatov K. et al. Evaluation of digital health strategy to support clinician-led critically ill patient population management: a randomized crossover study. Crit Care Explor 2023; 5 (05) e0909
- 61 Boswell CL, Minteer SA, Herasevich S. et al. Early prevention of critical illness in older adults: adaptation and pilot testing of an electronic risk score and checklist. J Prim Care Community Health 2024; 15: 21 501319241231238