Download PDF
Am J Perinatol 2024; 41(05): 598-605
DOI: 10.1055/a-1745-1348
Original Article

External Validation of Postpartum Hemorrhage Prediction Models Using Electronic Health Record Data

Sean R. Meyer 
1   Michigan Institute for Data Science, University of Michigan, Ann Arbor, Michigan
,
Alissa Carver 
2   Department of Obstetrics and Gynecology, University of Michigan, Ann Arbor, Michigan
,
Hyeon Joo 
3   Department of Anesthesiology, University of Michigan, Ann Arbor, Michigan
,
Kartik K. Venkatesh 
4   Department of Obstetrics and Gynecology, The Ohio State University, Columbus, Ohio
,
J. Eric Jelovsek 
5   Department of Obstetrics and Gynecology, Duke University, Durham, North Carolina
,
Thomas T. Klumpner    *
6   Department of Anesthesiology, University of Michigan, Ann Arbor, Michigan
,
Karandeep Singh    *
7   Department of Learning Health Sciences, University of Michigan, Ann Arbor, Michigan
› Author Affiliations Funding This study was funded by a University of Michigan internal grant: University of Michigan Precision Health Investigator Award 2019. K.K.V. was supported by the Care Innovation and Community Improvement Program at The Ohio State University. J.E.J. was supported by the NIDDK Symptoms of Lower Urinary Dysfunction Network at Duke University. These funding sources had no role in study design, data collection, analysis and interpretation, in writing of the report, and in the decision to submit the article for publication.
› Further Information
Also available at
    Permissions and Reprints

Abstract

Objective A recent study leveraging machine learning methods found that postpartum hemorrhage (PPH) can be predicted accurately at the time of labor admission in the U.S. Consortium for Safe Labor (CSL) dataset, with a C-statistic as high as 0.93. These CSL models were developed in older data (2002–2008) and used an estimated blood loss (EBL) of ≥1,000 mL to define PPH. We sought to externally validate these models using a more recent cohort of births where blood loss was measured using quantitative blood loss (QBL) methods.

Study Design Using data from 5,261 deliveries between February 1, 2019 and May 11, 2020 at a single tertiary hospital, we mapped our electronic health record (EHR) data to the 55 predictors described in previously published CSL models. PPH was defined as QBL ≥1,000 mL within 24 hours after delivery. Model discrimination and calibration of the four CSL models were measured using our cohort. In a secondary analysis, we fit new models in our study cohort using the same predictors and algorithms as the original CSL models.

Results The original study cohort had a substantially lower rate of PPH, 4.8% (7,279/228,438) versus 25% (1,321/5,261), possibly due to differences in measurement. The CSL models had lower discrimination in our study cohort, with a C-statistic as high as 0.57 (logistic regression). Models refit in our study cohort achieved better discrimination, with a C-statistic as high as 0.64 (random forest). Calibration improved in the refit models as compared with the original models.

Conclusion The CSL models' accuracy was lower in a contemporary EHR where PPH is assessed using QBL. As institutions continue to adopt QBL methods, further data are needed to understand the differences between EBL and QBL to enable accurate prediction of PPH.

Key Points

  • Machine learning methods may help predict PPH.

  • EBL models do not generalize when QBL is used.

  • Blood loss estimation alters model accuracy.

Keywords

electronic health record - external validation - machine learning - postpartum hemorrhage - prediction model

* These authors contributed equally.


Supplementary Material



Publication History

Received: 14 July 2021

Accepted: 17 January 2022

Accepted Manuscript online:
19 January 2022

Article published online:
02 March 2022

© 2022. Thieme. All rights reserved.

Thieme Medical Publishers, Inc.
333 Seventh Avenue, 18th Floor, New York, NY 10001, USA