New machine-learning models outperform conventional risk assessment tools in gastrointestinal bleeding

Aims Rapid and accurate identification of high-risk acute gastrointestinal bleeding (GIB) patients is essential. We developed two machine-learning (ML) models to calculate the risk of in-hospital mortality in patients admitted due to overt GIB.

Methods We analyzed the prospective, multicenter Hungarian GIB Registry's data collected between October 2019 and September 2022. The predictive performance of XGBoost and CatBoost machine-learning algorithms with the Glasgow-Blatchford (GBS) and pre-endoscopic Rockall scores were compared. We evaluated our models using five-fold cross-validation, and performance was measured by area under receiver operating characteristic curve (AUC) analysis with 95% confidence intervals (CI). The individual predictions and the variables' contribution to the outcome were calculated via the SHapley Additive exPlanations (SHAP) method.

Results Overall, we included 1,021 patients in the analysis; 52% had nonvariceal upper GIB, 30% had lower GIB, and 9% had variceal bleeding. The median age was 70 years (IQR:61-80); 60% were men. In-hospital death occurred in 108 cases (11%). The XGBoost and the CatBoost model identified patients who died with an AUC of 0.84 (CI:0.76-0.90; 0.77-0.90; respectively) in the internal validation set, whereas the GBS and pre-endoscopic Rockall clinical scoring system's performance was significantly lower, AUC values of 0.68 (CI:0.62 – 0.74) and 0.62 (CI:0.56 – 0.67), respectively. The XGBoost model had a specificity of 0.96 (CI:0.92-0.98) at a sensitivity of 0.25 (CI:0.10-0.43) compared with the CatBoost model, which had a specificity of 0.74 (CI:0.66-0.83) at a sensitivity of 0.78 (CI:0.57-0.95). Our XGBoost and CatBoost model identified the patient's CRP level as the most powerful characteristic influencing mortality.

Conclusions XGBoost and the CatBoost model better identified patients with high mortality risk than GBS and pre-endoscopic Rockall risk scores. The accuracy of the two ML models was not significantly different. Due to its better sensitivity, we prefer the CatBoost model in determining the in-hospital mortality risk of GIB.

Publication History

Article published online:
27 March 2025