Redesigning Clinical Decision Support for Retinopathy of Prematurity Screening After Alert Failure

Abstract

Background

Retinopathy of prematurity (ROP) is the leading cause of preventable childhood blindness. Guidelines recommend screening for infants with gestational age at birth <31 weeks or birth weight ≤1,500 g. However, ensuring timely screening during readmissions after birth is challenging.

Objectives

To analyze the performance of an interruptive alert at a large academic pediatric hospital for identifying premature infants needing ROP screening upon hospital readmission and to describe how data informed the transition to a non-interruptive dashboard.

Methods

The alert appeared for patients 1 to 365 days of age hospitalized in acute care or pediatric intensive care and instructed providers to order an ophthalmology consult from within the alert and to call ophthalmology for at-risk patients. For quality improvement, the clinical decision support (CDS) advisory group evaluated the effectiveness and efficiency of the alert. We extracted alert metrics from the hospital's enterprise data warehouse, including the user response and feedback, patient characteristics (age, birth gestational age, and birth weight), and any ophthalmology consultations. We analyzed the percentage of encounters seen by ophthalmology using a statistical process control chart during alert implementation and 6 months before and after.

Results

The alert appeared 3,309 times during 2,194 patient encounters usually. Users chose “Accept and place order” for 43% (943/2,194) of encounters, but only 11% (102/943) had an ophthalmology consult; 34% (53/155) of ophthalmology consultations occurred in encounters with a final response other than “Accept and place order.” The intervention was redesigned using a non-interruptive surveillance dashboard with greater specificity, and the alert was de-implemented.

Conclusion

Analysis of a failed interruptive alert for identifying patients at risk for ROP led to a transition to targeted surveillance using a dashboard. This case emphasizes the importance of aligning the CDS modality to the clinical workflow, information availability, and user decision-making needs and should be supported by governance.

Keywords

clinical decision support - quality improvement - retinopathy of prematurity - alert fatigue - dashboard

Protection of Human and Animal Subjects

This data was collected as part of a continuous quality improvement initiative in alignment with institutional policies and ethical standards for healthcare improvement projects. No research involving human or animal subjects was performed, and data collection focused on de-identified, routine clinical operations. The case report adhered to the principles outlined in the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects.

Publication History

Received: 31 December 2024

Accepted: 24 April 2025

Accepted Manuscript online:
25 April 2025

Article published online:
05 September 2025

• References

• 1 Fierson WM. American Academy of Pediatrics Section on Ophthalmology, American Academy of Ophthalmology, American Association for Pediatric Ophthalmology and Strabismus, American Association of Certified Orthoptists. Screening examination of premature infants for retinopathy of prematurity. Pediatrics 2018; 142 (06) e20183061
• 2 Osheroff Jerome A, Teich J, Levick D. et al. Improving Outcomes with Clinical Decision Support. 2nd ed. HIMSS Publishing; 2012
• 3 Berner ES. ed. Clinical Decision Support Systems: Theory and Practice. 3rd ed. Springer International Publishing; 2016
• 4 Bates DW, Kuperman GJ, Wang S. et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 10 (06) 523-530
• 5 Chaparro JD, Hussain C, Lee JA, Hehmeyer J, Nguyen M, Hoffman J. Reducing interruptive alert burden using quality improvement methodology. Appl Clin Inform 2020; 11 (01) 46-58
• 6 McGreevey III JD, Mallozzi CP, Perkins RM, Shelov E, Schreiber R. Reducing alert burden in electronic health records: state of the art recommendations from four health systems. Appl Clin Inform 2020; 11 (01) 1-12
• 7 Chaparro JD, Beus JM, Dziorny AC. et al. Clinical decision support stewardship: best practices and techniques to monitor and improve interruptive alerts. Appl Clin Inform 2022; 13 (03) 560-568
• 8 International HL7 Clinical Documentation Architecture (CDA). August 28, 2024 . Accessed December 2, 2024 at: https://www.hl7.org/cda/
• 9 Barton GP, Chandra A, Sanchez-Solano N, Berry JD, Goss KN. Smaller left ventricular size but preserved function in adolescents and adults born preterm. J Am Heart Assoc 2024; 13 (18) e035529
• 10 Blecker S, Pandya R, Stork S. et al. Interruptive versus noninterruptive clinical decision support: usability study. JMIR Hum Factors 2019; 6 (02) e12469
• 11 Lammert J, Dreyer T, Mathes S. et al. Expert-guided large language models for clinical decision support in precision oncology. JCO Precis Oncol 2024; 8 (08) e2400478
• 12 Hu Y, Chen Q, Du J. et al. Improving large language models for clinical named entity recognition via prompt engineering. J Am Med Inform Assoc 2024; 31 (09) 1812-1820
• 13 Wright A, Sittig DF, Ash JS. et al. Governance for clinical decision support: case studies and recommended practices from leading institutions. J Am Med Inform Assoc 2011; 18 (02) 187-194
• 14 Van Dort BA, Zheng WY, Sundar V, Baysari MT. Optimizing clinical decision support alerts in electronic medical records: a systematic review of reported strategies adopted by hospitals. J Am Med Inform Assoc 2021; 28 (01) 177-183