Reply to “Developing a Risk Prediction Model for Intensive Care Unit Mortality after Cardiac Surgery”

Validation of Three Postoperative Risk Prediction Models for Intensive Care Unit Mortality after Cardiac Surgery

Developing a Risk Prediction Model for Intensive Care Unit Mortality after Cardiac Surgery

We thank Liu et al for their letter regarding our manuscript “Validation of three postoperative risk prediction models for intensive care unit mortality after cardiac surgery.”[1] As they point out, the suboptimal calibration of the models we analyzed is likely to be multifactorial. The mortality rate after cardiac surgery has fallen over time resulting in calibration drift which has significantly affected overall model calibration.[2] The reasons behind the suboptimal calibration following local recalibration are more complex and are likely related to the design of the original models. We agree with Liu et al that postoperative mortality after cardiac surgery is influenced by a range of important factors. Including as many of the postoperative complications known to affect mortality risk as possible would be likely to improve model performance. In addition to the complications discussed by Liu et al, our group has recently demonstrated that postoperative sepsis diagnosed using the most recent Sepsis-3 definition is also associated with poor outcomes following cardiac surgery.[3] However, inclusion of all potentially relevant factors within a mortality model is often not possible due to the large number of potential variables and limitations of the data available.

The number of risk factors included in the models is primarily limited by the statistical approach employed. All models in our analysis employed multivariable logistic regression and the design of such models is limited by the number of outcomes observed in the groups of the development dataset.[4] The lower the number of outcomes in the development set, the fewer variables can be included in the model without increasing the risk of overfitting. In cardiac surgery, low mortality rates often limit the number of factors which can be entered into models for mortality prediction. The development of mortality models containing more variables will therefore require larger development datasets.

The models analyzed were primarily based on physiological parameters and treatments provided rather than the occurrence of postoperative complications. Excluding postoperative complications from such models reduces the likelihood of errors being introduced through diagnostic misclassification and also facilitates real-time risk prediction. Mortality models which include complications as variables may be more statistically accurate but are likely to be less useful clinically as they only show an increase in risk once a significant complication has already occurred.

Future research should focus on developing postoperative models designed for specific purposes. For example, models which work in real time to predict adverse events based on physiological data could enable changes in clinical management strategies to prevent complications occurring. Alternative models based on combined event and physiological data may allow the most accurate risk adjustment of postoperative outcome data. Our group is hoping to pursue work in these areas following the award of a British Heart Foundation grant to combine continuous physiological data with nursing observations and postoperative complications for the development of predictive models for use following cardiac surgery.

• References

• 1 Howitt SH, Caiado C, McCollum C. , et al. Validation of three postoperative risk prediction models for intensive care unit mortality after cardiac surgery. Thorac Cardiovasc Surg 2018; DOI: 10.1055/s-0037-1608897.
• 2 Hickey GL, Grant SW, Murphy GJ. , et al. Dynamic trends in cardiac surgery: why the logistic EuroSCORE is no longer suitable for contemporary cardiac surgery and implications for future risk models. Eur J Cardiothorac Surg 2013; 43 (06) 1146-1152
• 3 Howitt SH, Herring M, Malagon I, McCollum CN, Grant SW. Incidence and outcomes of sepsis after cardiac surgery as defined by the Sepsis-3 guidelines. Br J Anaesth 2018; 120 (03) 509-516
• 4 Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 1996; 49 (12) 1373-1379