Clinical Decision Support Enhanced by User Acceptance Testing to Promote Situational Awareness for Pediatric Patients with a Difficult Airway

Abstract

Background

Children with a difficult airway are at high risk of decompensation in the setting of respiratory distress. Situational awareness among all team members, and a shared plan in case of an emergency, can reduce the chance of catastrophic outcomes.

Objectives

This study aimed to improve difficult airway situational awareness while minimizing alert burden in a quaternary care pediatric healthcare system through the application of clinical decision support (CDS).

Methods

Three iterative designs were developed and implemented from 2015 through 2023. We measured interruptive alert burden and performed observations between each implementation to estimate point prevalence among hospitalized patients with a difficult airway of three desired behaviors: presence of a difficult airway sign at the head of the bed, orders placed for appropriate equipment nearby, and primary nurse awareness of the difficult airway.

Results

Over the course of the redesign, the alert burden decreased from 12,316 firings per month to 125 firings per month from the first alert to the second redesign and final iteration. There was a statistically significant increase in the proportion of difficult airway patients with orders for appropriate equipment from 51.4 to 83.9% (p < 0.001). There was no significant change in difficult airway sign placement (71.4–87.1%, p = 0.29) or observed nurse awareness of difficult airway status of the patient (80.0–87.1%, p = 0.447). The greatest improvements in alert burden and rates of desired user action occurred after redesigning based on usability testing.

Conclusion

CDS redesign using popular frameworks alone reduced alert burden without significantly worsening situational awareness. Redesign through guerilla in situ usability testing led to much more substantial reductions in alert burden and greater improvements in desired user action.

Keywords

decision support systems - clinical awareness - user-centered design - alert fatigue - health personnel

Protection of Human and Animal Subjects

This study was approved by the Children's Healthcare of Atlanta Institutional Review Board as Non-Human Subjects Research Study no.: 00001887.

Publication History

Received: 27 December 2024

Accepted: 11 June 2025

Accepted Manuscript online:
12 June 2025

Article published online:
13 August 2025

• References

• 1 Nolan JP, Deakin CD, Soar J, Böttiger BW, Smith G. European Resuscitation Council. European Resuscitation Council guidelines for resuscitation 2005. Section 4. Adult advanced life support. Resuscitation 2005; 67 (Suppl. 01) S39-S86
• 2 Kollmeier BR, Boyette LC, Beecham GB, Desai NM, Khetarpal S. Difficult Airway. StatPearls; 2025
• 3 Hsu G, von Ungern-Sternberg BS, Engelhardt T. Pediatric airway management. Curr Opin Anaesthesiol 2021; 34 (03) 276-283
• 4 Apfelbaum JL, Hagberg CA, Connis RT. et al. 2022 American Society of Anesthesiologists practice guidelines for management of the difficult airway. Anesthesiology 2022; 136 (01) 31-81
• 5 McGreevey III JD, Mallozzi CP, Perkins RM, Shelov E, Schreiber R. Reducing alert burden in electronic health records: state of the art recommendations from four health systems. Appl Clin Inform 2020; 11 (01) 1-12
• 6 Endsley MR. A systematic review and meta-analysis of direct objective measures of situation awareness: a comparison of SAGAT and SPAM. Hum Factors 2021; 63 (01) 124-150
• 7 Feller S, Feller L, Bhayat A, Feller G, Khammissa RAG, Vally ZI. Situational awareness in the context of clinical practice. Healthcare (Basel) 2023; 11 (23) 3098
• 8 Rameau A, Wang E, Saraswathula A, Pageler N, Perales S, Sidell DR. Enhancing pediatric airway safety using the electronic medical record. Laryngoscope 2018; 128 (12) 2885-2892
• 9 Endsley MR. Situation awareness misconceptions and misunderstandings. J Cogn Eng Decis Mak 2015;9(01):
• 10 Chaparro JD, Beus JM, Dziorny AC. et al. Clinical decision support stewardship: best practices and techniques to monitor and improve interruptive alerts. Appl Clin Inform 2022; 13 (03) 560-568
• 11 Zhang J, Walji MF. TURF: toward a unified framework of EHR usability. J Biomed Inform 2011; 44 (06) 1056-1067
• 12 Kandaswamy S, Williams H, Thompson SA, Dawson TE, Muthu N, Orenstein EW. Realizing the full potential of clinical decision support: translating usability testing into routine practice in health care operations. Appl Clin Inform 2024; 15 (05) 1039-1048
• 13 Soberano BT, Brady P, Yunger T. et al. The effects of care team roles on situation awareness in the pediatric intensive care unit: a prospective cross-sectional study. J Hosp Med 2020; 15 (10) 594-597
• 14 Rasooly I, Orenstein E, Grundmeier R, Muthu N. Measuring performance in an electronic health record (EHR) based simulation using the situation awareness global assessment technique. Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care. 2019; 8 (01) 92-95
• 15 Sosa T, Mayer B, Chakkalakkal B. et al. Improving shared situation awareness for high-risk therapies in hospitalized children. Hosp Pediatr 2022; 12 (01) 37-46
• 16 Sosa T, Sitterding M, Dewan M. et al. Optimizing situation awareness to reduce emergency transfers in hospitalized children. Pediatrics 2021; 148 (04) e2020034603
• 17 Dewan M, Soberano B, Sosa T. et al. Assessment of a situation awareness quality improvement intervention to reduce cardiac arrests in the PICU. Pediatr Crit Care Med 2022; 23 (01) 4-12