Appl Clin Inform 2020; 11(04): 570-577
DOI: 10.1055/s-0040-1715827
Research Article

Implementation of Artificial Intelligence-Based Clinical Decision Support to Reduce Hospital Readmissions at a Regional Hospital

Santiago Romero-Brufau
1   Mayo Clinic Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, United States
2   Department of Biostatistics, Harvard T.H. Chan School of Public Health, Harvard University, Cambridge, Massachusetts, United States
,
Kirk D. Wyatt
3   Division of Pediatric Hematology/Oncology, Department of Pediatric and Adolescent Medicine, Mayo Clinic, Rochester, Minnesota, United States
,
Patricia Boyum
1   Mayo Clinic Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, United States
,
Mindy Mickelson
1   Mayo Clinic Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, United States
,
Matthew Moore
1   Mayo Clinic Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, United States
,
Cheristi Cognetta-Rieke
4   Department of Nursing, Mayo Clinic Health System, La Crosse, La Crosse, Wisconsin, United States
› Author Affiliations
Funding This study received its financial support from Mayo Clinic research funds.

Abstract

Background Hospital readmissions are a key quality metric, which has been tied to reimbursement. One strategy to reduce readmissions is to direct resources to patients at the highest risk of readmission. This strategy necessitates a robust predictive model coupled with effective, patient-centered interventions.

Objective The aim of this study was to reduce unplanned hospital readmissions through the use of artificial intelligence-based clinical decision support.

Methods A commercially vended artificial intelligence tool was implemented at a regional hospital in La Crosse, Wisconsin between November 2018 and April 2019. The tool assessed all patients admitted to general care units for risk of readmission and generated recommendations for interventions intended to decrease readmission risk. Similar hospitals were used as controls. Change in readmission rate was assessed by comparing the 6-month intervention period to the same months of the previous calendar year in exposure and control hospitals.

Results Among 2,460 hospitalizations assessed using the tool, 611 were designated by the tool as high risk. Sensitivity and specificity for risk assignment were 65% and 89%, respectively. Over 6 months following implementation, readmission rates decreased from 11.4% during the comparison period to 8.1% (p < 0.001). After accounting for the 0.5% decrease in readmission rates (from 9.3 to 8.8%) at control hospitals, the relative reduction in readmission rate was 25% (p < 0.001). Among patients designated as high risk, the number needed to treat to avoid one readmission was 11.

Conclusion We observed a decrease in hospital readmission after implementing artificial intelligence-based clinical decision support. Our experience suggests that use of artificial intelligence to identify patients at the highest risk for readmission can reduce quality gaps when coupled with patient-centered interventions.

Protection of Human and Animal Subjects

This study was reviewed by the Mayo Clinic Institutional Review Board and deemed “exempt.”




Publication History

Received: 29 April 2020

Accepted: 17 July 2020

Article published online:
02 September 2020

Georg Thieme Verlag KG
Stuttgart · New York

 
  • References

  • 1 Car J, Sheikh A, Wicks P, Williams MS. Beyond the hype of big data and artificial intelligence: building foundations for knowledge and wisdom. BMC Med 2019; 17 (01) 143
  • 2 Jamei M, Nisnevich A, Wetchler E, Sudat S, Liu E. Predicting all-cause risk of 30-day hospital readmission using artificial neural networks. PLoS One 2017; 12 (07) e0181173
  • 3 Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA. The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med 2018; 24 (11) 1716-1720
  • 4 Seymour CW, Kennedy JN, Wang S. , et al. Derivation, validation, and potential treatment implications of novel clinical phenotypes for sepsis. JAMA 2019; 321 (20) 2003-2017
  • 5 Hasan M. Readmission of patients to hospital: still ill defined and poorly understood. Int J Qual Health Care 2001; 13 (03) 177-179
  • 6 Clarke A. Readmission to hospital: a measure of quality or outcome?. Qual Saf Health Care 2004; 13 (01) 10-11
  • 7 Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med 2009; 360 (14) 1418-1428
  • 8 Kripalani S, Theobald CN, Anctil B, Vasilevskis EE. Reducing hospital readmission rates: current strategies and future directions. Annu Rev Med 2014; 65: 471-485
  • 9 Desai NR, Ross JS, Kwon JY. , et al. Association between hospital penalty status under the hospital readmission reduction program and readmission rates for target and nontarget conditions. JAMA 2016; 316 (24) 2647-2656
  • 10 Lee EW. Selecting the best prediction model for readmission. J Prev Med Public Health 2012; 45 (04) 259-266
  • 11 Cholleti S, Post A, Gao J. , et al. Leveraging derived data elements in data analytic models for understanding and predicting hospital readmissions. AMIA Annu Symp Proc 2012; 2012: 103-111
  • 12 Kulkarni P, Smith LD, Woeltje KF. Assessing risk of hospital readmissions for improving medical practice. Health Care Manage Sci 2016; 19 (03) 291-299
  • 13 Swain MJ, Kharrazi H. Feasibility of 30-day hospital readmission prediction modeling based on health information exchange data. Int J Med Inform 2015; 84 (12) 1048-1056
  • 14 Artetxe A, Beristain A, Graña M. Predictive models for hospital readmission risk: a systematic review of methods. Comput Methods Programs Biomed 2018; 164: 49-64
  • 15 van Walraven C, Dhalla IA, Bell C. , et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ 2010; 182 (06) 551-557
  • 16 Cotter PE, Bhalla VK, Wallis SJ, Biram RW. Predicting readmissions: poor performance of the LACE index in an older UK population. Age Ageing 2012; 41 (06) 784-789
  • 17 Damery S, Combes G. Evaluating the predictive strength of the LACE index in identifying patients at high risk of hospital readmission following an inpatient episode: a retrospective cohort study. BMJ Open 2017; 7 (07) e016921
  • 18 Hopkins BS, Yamaguchi JT, Garcia R. , et al. Using machine learning to predict 30-day readmissions after posterior lumbar fusion: an NSQIP study involving 23,264 patients. J Neurosurg Spine 2019; 1-8
  • 19 Mahajan SM, Ghani R. Using ensemble machine learning methods for predicting risk of readmission for heart failure. Stud Health Technol Inform 2019; 264: 243-247
  • 20 Wolff P, Graña M, Ríos SA, Yarza MB. Machine learning readmission risk modeling: a pediatric case study. BioMed Res Int 2019; 2019: 8532892
  • 21 Eckert C, Nieves-Robbins N, Spieker E. , et al. Development and prospective validation of a machine learning-based risk of readmission model in a large military hospital. Appl Clin Inform 2019; 10 (02) 316-325
  • 22 Awan SE, Bennamoun M, Sohel F, Sanfilippo FM, Dwivedi G. Machine learning-based prediction of heart failure readmission or death: implications of choosing the right model and the right metrics. ESC Heart Fail 2019; 6 (02) 428-435
  • 23 Min X, Yu B, Wang F. Predictive modeling of the hospital readmission risk from patients' claims data using machine learning: a case study on COPD. Sci Rep 2019; 9 (01) 2362
  • 24 Kalagara S, Eltorai AEM, Durand WM, DePasse JM, Daniels AH. Machine learning modeling for predicting hospital readmission following lumbar laminectomy. J Neurosurg Spine 2018; 30 (03) 344-352
  • 25 Kakarmath S, Golas S, Felsted J, Kvedar J, Jethwani K, Agboola S. Validating a machine learning algorithm to predict 30-day re-admissions in patients with heart failure: protocol for a prospective cohort study. JMIR Res Protoc 2018; 7 (09) e176
  • 26 Mahajan SM, Mahajan AS, King R, Negahban S. Predicting risk of 30-day readmissions using two emerging machine learning methods. Stud Health Technol Inform 2018; 250: 250-255
  • 27 Baig MM, Hua N, Zhang E. , et al. A machine learning model for predicting risk of hospital readmission within 30 days of discharge: validated with LACE index and patient at risk of hospital readmission (PARR) model. Med Biol Eng Comput 2020; 58 (07) 1459-1466
  • 28 Gupta S, Ko DT, Azizi P. , et al. Evaluation of machine learning algorithms for predicting readmission after acute myocardial infarction using routinely collected clinical data. Can J Cardiol 2020; 36 (06) 878-885
  • 29 Baig MM, Hua N, Zhang E. , et al. Machine learning-based risk of hospital readmissions: predicting acute readmissions within 30 days of discharge. Conf Proc IEEE Eng Med Biol Soc 2019; 2019: 2178-2181
  • 30 Goto T, Jo T, Matsui H, Fushimi K, Hayashi H, Yasunaga H. Machine learning-based prediction models for 30-day readmission after hospitalization for chronic obstructive pulmonary disease. COPD 2019; 16 (5-6): 338-343
  • 31 Awan SE, Bennamoun M, Sohel F, Sanfilippo FM, Chow BJ, Dwivedi G. Feature selection and transformation by machine learning reduce variable numbers and improve prediction for heart failure readmission or death. PLoS One 2019; 14 (06) e0218760
  • 32 Morgan DJ, Bame B, Zimand P. , et al. Assessment of machine learning vs standard prediction rules for predicting hospital readmissions. JAMA Netw Open 2019; 2 (03) e190348
  • 33 Golas SB, Shibahara T, Agboola S. , et al. A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data. BMC Med Inform Decis Mak 2018; 18 (01) 44
  • 34 Rojas JC, Carey KA, Edelson DP, Venable LR, Howell MD, Churpek MM. Predicting intensive care unit readmission with machine learning using electronic health record data. Ann Am Thorac Soc 2018; 15 (07) 846-853
  • 35 Desautels T, Das R, Calvert J. , et al. Prediction of early unplanned intensive care unit readmission in a UK tertiary care hospital: a cross-sectional machine learning approach. BMJ Open 2017; 7 (09) e017199
  • 36 Garcia-Arce A, Rico F, Zayas-Castro JL. Comparison of machine learning algorithms for the prediction of preventable hospital readmissions. J Healthc Qual 2018; 40 (03) 129-138
  • 37 Mortazavi BJ, Downing NS, Bucholz EM. , et al. Analysis of machine learning techniques for heart failure readmissions. Circ Cardiovasc Qual Outcomes 2016; 9 (06) 629-640
  • 38 Shameer K, Johnson KW, Yahi A. , et al. Predictive modeling of hospital readmission rates using electronic medical record-wide machine learning: a case-study using Mount Sinai Heart Failure Cohort. Pac Symp Biocomput 2017; 22: 276-287
  • 39 Frizzell JD, Liang L, Schulte PJ. , et al. Prediction of 30-day all-cause readmissions in patients hospitalized for heart failure: comparison of machine learning and other statistical approaches. JAMA Cardiol 2017; 2 (02) 204-209
  • 40 Hood CM, Gennuso KP, Swain GR, Catlin BB. County health rankings: relationships between determinant factors and health outcomes. Am J Prev Med 2016; 50 (02) 129-135
  • 41 Negarandeh R, Zolfaghari M, Bashi N, Kiarsi M. Evaluating the effect of monitoring through telephone (tele-monitoring) on self-care behaviors and readmission of patients with heart failure after discharge. Appl Clin Inform 2019; 10 (02) 261-268
  • 42 Institute of Medicine. Evidence-Based Medicine and the Changing Nature of Healthcare: 2007 IOM Annual Meeting Summary. Washington (DC): National Academies Press; 2008
  • 43 Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and cost. Health Aff (Millwood) 2008; 27 (03) 759-769
  • 44 Chen S, Bergman D, Miller K, Kavanagh A, Frownfelter J, Showalter J. Using applied machine learning to predict healthcare utilization based on socioeconomic determinants of care. Am J Manag Care 2020; 26 (01) 26-31
  • 45 Shortliffe EH, Sepúlveda MJ. Clinical decision support in the era of artificial intelligence. JAMA 2018; 320 (21) 2199-2200
  • 46 Navathe AS, Zhong F, Lei VJ. , et al. Hospital readmission and social risk factors identified from physician notes. Health Serv Res 2018; 53 (02) 1110-1136
  • 47 Romero-Brufau S, Huddleston JM, Escobar GJ, Liebow M. Why the C-statistic is not informative to evaluate early warning scores and what metrics to use. Crit Care 2015; 19: 285
  • 48 Yu S, Farooq F, van Esbroeck A, Fung G, Anand V, Krishnapuram B. Predicting readmission risk with institution-specific prediction models. Artif Intell Med 2015; 65 (02) 89-96
  • 49 Lazer D, Kennedy R, King G, Vespignani A. Big data. The parable of Google Flu: traps in big data analysis. Science 2014; 343 (6176): 1203-1205