PDF icon PDF herunterladen
Appl Clin Inform 2025; 16(04): 838-847
DOI: 10.1055/a-2630-4192
Research Article

Evaluating Equity in Usage and Effectiveness of the CONCERN Early Warning System

Rachel Y. Lee 
1   Department of Biomedical Informatics, Columbia University, New York, New York, United States
,
Kenrick D. Cato
2   School of Nursing, Columbia University, New York, New York, United States
3   School of Nursing, University of Pennsylvania, Philadelphia, Pennsylvania, United States
4   Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
,
Patricia C. Dykes
5   Center for Patient Safety, Brigham and Women's Hospital, Boston, Massachusetts, United States
6   Harvard Medical School, Boston, Massachusetts, United States
,
Graham Lowenthal
5   Center for Patient Safety, Brigham and Women's Hospital, Boston, Massachusetts, United States
,
Haomiao Jia
2   School of Nursing, Columbia University, New York, New York, United States
,
Temiloluwa Daramola
1   Department of Biomedical Informatics, Columbia University, New York, New York, United States
,
Sarah C. Rossetti
1   Department of Biomedical Informatics, Columbia University, New York, New York, United States
2   School of Nursing, Columbia University, New York, New York, United States
› Institutsangaben
Funding American Nurses Foundation Reimagining Nursing Initiative approved grant for this study.
› Weitere Informationen
Auch verfügbar aufeRef
   Lizenzen und Reprints
Preview

Abstract

Background

The CONCERN Early Warning System (CONCERN EWS) is an artificial intelligence-based clinical decision support system (AI-CDSS) for the prediction of clinical deterioration, leveraging signals from nursing documentation patterns. While a recent multisite randomized controlled trial (RCT) demonstrated its effectiveness in reducing inpatient mortality and length of stay, evaluating implementation outcomes is essential to ensure equitable results across patient populations.

Objectives

This study aims to (1) assess whether clinicians' usage of the CONCERN EWS, as measured by CONCERN Detailed Prediction Screen launches, varied by patient demographic characteristics, including sex, race, ethnicity, and primary language; (2) evaluate whether CONCERN EWS's effectiveness in reducing the risk of in-hospital mortality varied across patient demographic groups.

Methods

We conducted a retrospective observational analysis of electronic health record log files and clinical outcomes from a multisite, pragmatic, cluster-RCT involving four hospitals across two health care systems. Equity in usage was assessed by comparing CONCERN Detailed Prediction Screen launches across demographic groups, and effectiveness was examined by comparing the risk of in-hospital mortality between intervention and usual care groups using Cox proportional hazards models adjusted for patient characteristics.

Results

Clinicians' CONCERN Detailed Prediction Screen launches did not significantly differ by patients' demographic characteristics, suggesting equitable usage. The CONCERN EWS was significantly associated with reduced risk of in-hospital mortality overall (adjusted hazard ratio [HR] = 0.644, 95% CI: 0.532–0.778, p < 0.0001), with consistent effectiveness across most groups. Notably, patients whose primary language was not English experienced a greater reduction of mortality risk compared to patients whose primary language was English (adjusted HR = 0.419, 95% CI: 0.287–0.610, p = 0.0082).

Conclusion

This study presents a case of evaluating equity in AI-CDSS usage and effectiveness, contributing to the limited literature. While findings suggest equitable engagement and effectiveness, ongoing evaluations are needed to understand the observed variability and ensure responsible implementation.

Keywords

clinical decision support systems - artificial intelligence - implementation science - electronic health records - nursing informatics

Protection of Human and Animal Subjects

Institutional review boards at the researchers' institution (blinded for review) approved the protocol with a waiver of consent.


Authors' Contributions

Conceptualization: R.Y.L., S.C.R., K.D.C., P.C.D. Data curation, formal analysis, methodology: R.Y.L., H.J., G.L., S.C.R. Writing—original draft: R.Y.L. Writing—review and editing: R.Y.L., K.D.C., P.C.D., G.L., H.J., T.D., and S.C.R.




Publikationsverlauf

Eingereicht: 21. Februar 2025

Angenommen: 07. Juni 2025

Accepted Manuscript online:
10. Juni 2025

Artikel online veröffentlicht:
13. August 2025

© 2025. Thieme. All rights reserved.

Georg Thieme Verlag KG
Oswald-Hesse-Straße 50, 70469 Stuttgart, Germany