Download PDF
Methods Inf Med 2017; 56(05): 377-389
DOI: 10.3414/ME17-01-0024
Paper
Schattauer GmbH

Prediction of Emergency Department Hospital Admission Based on Natural Language Processing and Neural Networks[*]

Xingyu Zhang
1   Department of Surgery, Emory University School of Medicine, Atlanta, GA, USA
,
Joyce Kim
2   Emory University School of Medicine, Atlanta, GA, USA
,
Rachel E. Patzer
1   Department of Surgery, Emory University School of Medicine, Atlanta, GA, USA
3   Department of Epidemiology, Rollins School of Public Health, Atlanta, GA, USA
,
Stephen R. Pitts
4   Department of Emergency Medicine, Emory School of Medicine, Atlanta, GA, USA
,
Aaron Patzer
5   Vital Software Inc., Auckland, New Zealand
,
Justin D. Schrager
4   Department of Emergency Medicine, Emory School of Medicine, Atlanta, GA, USA
› Author Affiliations
Further Information

Publication History

received: 25 February 2017

accepted in revised form: 26 July 2017

Publication Date:
24 January 2018 (online)

    Permissions and Reprints

Summary

Objective: To describe and compare logistic regression and neural network modeling strategies to predict hospital admission or transfer following initial presentation to Emergency Department (ED) triage with and without the addition of natural language processing elements.

Methods: Using data from the National Hospital Ambulatory Medical Care Survey (NHAMCS), a cross-sectional probability sample of United States EDs from 2012 and 2013 survey years, we developed several predictive models with the outcome being admission to the hospital or transfer vs. discharge home. We included patient characteristics immediately available after the patient has presented to the ED and undergone a triage process. We used this information to construct logistic regression (LR) and multilayer neural network models (MLNN) which included natural language processing (NLP) and principal component analysis from the patient’s reason for visit. Ten-fold cross validation was used to test the predictive capacity of each model and receiver operating curves (AUC) were then calculated for each model.

Results: Of the 47,200 ED visits from 642 hospitals, 6,335 (13.42%) resulted in hospital admission (or transfer). A total of 48 principal components were extracted by NLP from the reason for visit fields, which explained 75% of the overall variance for hospitalization. In the model including only structured variables, the AUC was 0.824 (95% CI 0.818-0.830) for logistic regression and 0.823 (95% CI 0.817-0.829) for MLNN. Models including only free-text information generated AUC of 0.742 (95% CI 0.7310.753) for logistic regression and 0.753 (95% CI 0.742-0.764) for MLNN. When both structured variables and free text variables were included, the AUC reached 0.846 (95% CI 0.839-0.853) for logistic regression and 0.844 (95% CI 0.836-0.852) for MLNN.

Conclusions: The predictive accuracy of hospital admission or transfer for patients who presented to ED triage overall was good, and was improved with the inclusion of free text data from a patient’s reason for visit regardless of modeling approach. Natural language processing and neural networks that incorporate patient-reported outcome free text may increase predictive accuracy for hospital admission.

Keywords

Natural language processing - neural networks - logistic regression - prediction - emergency department

* Supplementary material published on our website https://doi.org/10.3414/ME17-01-0024


 

  • References

  • 1 Sun Y, Heng BH, Tay SY, Seow E. Predicting hospital admissions at emergency department triage using routine administrative data. Academic Emergency Medicine 2011; 18 (08) 844-850.
    CrossrefPubMedGoogle Scholar
  • 2 Handly N, Thompson DA, Li J, Chuirazzi DM, Venkat A. Evaluation of a hospital admission prediction model adding coded chief complaint data using neural network methodology. European Journal of Emergency Medicine 2015; 22 (02) 87-91.
    CrossrefPubMedGoogle Scholar
  • 3 Golmohammadi D. Predicting hospital admissions to reduce emergency department boarding. International Journal of Production Economics 2016; 182: 535-544.
    CrossrefPubMedGoogle Scholar
  • 4 Future of Emergency Care: Hospital-Based Emergency Care at the Breaking Point. Washington, DC: Institute of Medicine; 2006
    Google Scholar
  • 5 Sun BC, Hsia RY, Weiss RE. et al. Effect of emergency department crowding on outcomes of admitted patients. Annals of Emergency Medicine 2013; 61 (06) 605-611. e606.
    CrossrefPubMedGoogle Scholar
  • 6 Pines JM, Hilton JA, Weber EJ. et al. International perspectives on emergency department crowding. Academic Emergency Medicine 2011; 18 (12) 1358-1370.
    CrossrefPubMedGoogle Scholar
  • 7 Chalfin DB, Trzeciak S, Likourezos A, Baumann BM, Dellinger RP, group D-Es. Impact of delayed transfer of critically ill patients from the emergency department to the intensive care unit. Critical Care Medicine 2007; 35 (06) 1477-1483.
    CrossrefPubMedGoogle Scholar
  • 8 Rabin E, Kocher K, McClelland M. et al. Solutions to emergency department ‘boarding’ and crowding are underused and may need to be legislated. Health Affairs 2012; 31 (08) 1757-1766.
    CrossrefPubMedGoogle Scholar
  • 9 Forero R, McCarthy S, Hillman K. Access block and emergency department overcrowding. Critical Care 2011; 15 (02) 1.
    PubMedGoogle Scholar
  • 10 Peck JS, Benneyan JC, Nightingale DJ, Gaehde SA. Predicting emergency department inpatient admissions to improve same-day patient flow. Academic Emergency Medicine 2012; 19 (09) E1045-1054.
    CrossrefPubMedGoogle Scholar
  • 11 Leegon J, Jones I, Lanaghan K, Aronsky D. Predicting hospital admission in a pediatric Emergency Department using an Artificial Neural Network. AMIA Annual Symposium Proceedings. 2006: 1004.
    PubMedGoogle Scholar
  • 12 McCallum A. Information extraction: Distilling structured data from unstructured text. Queue 2005; 3 (09) 48-57.
    PubMedGoogle Scholar
  • 13 Cohen AM, Hersh WR. A survey of current work in biomedical text mining. Briefings in Bioin- formatics 2005; 6 (01) 57-71.
    PubMedGoogle Scholar
  • 14 Claster W, Shanmuganathan S, Ghotbi N. Text mining of medical records for radiodiagnostic decision-making. Journal of Computers 2008; 3 (01) 1-6.
    PubMedGoogle Scholar
  • 15 Demner-Fushman D, Chapman WW, McDonald CJ. What can natural language processing do for clinical decision support?. Journal of Biomedical Informatics 2009; 42 (05) 760-772.
    CrossrefPubMedGoogle Scholar
  • 16 Lin C, Karlson EW, Canhao H. et al. Automatic prediction of rheumatoid arthritis disease activity from the electronic medical records. PLoS One 2013; 8 (08) e69932.
    CrossrefPubMedGoogle Scholar
  • 17 Sadat-Hashemi SM, Kazemnejad A, Lucas C, Badie K. Predicting the type of pregnancy using artificial neural networks and multinomial logistic regression: a comparison study. Neural Computing & Applications 2005; 14 (03) 198-202.
    CrossrefPubMedGoogle Scholar
  • 18 Penny W, Frost D. Neural networks in clinical medicine. Medical Decision Making 1996; 16 (04) 386-398.
    CrossrefPubMedGoogle Scholar
  • 19 Heckerling PS, Gerber BS, Tape TG, Wigton RS. Prediction of community-acquired pneumonia using artificial neural networks. Medical Decision Making 2003; 23 (02) 112-121.
    CrossrefPubMedGoogle Scholar
  • 20 Lammers RL, Hudson DL, Seaman ME. Prediction of traumatic wound infection with a neural network-derived decision model. American journal of Emergency Medicine 2003; 21 (01) 1-7.
    PubMedGoogle Scholar
  • 21 Launay C, Rivière H, Kabeshova A, Beauchet O. Predicting prolonged length of hospital stay in older emergency department users: Use of a novel analysis method, the Artificial Neural Network. European Journal of Internal Medicine 2015; 26 (07) 478-482.
    CrossrefPubMedGoogle Scholar
  • 22 Hess EP, Wells GA, Jaffe A, Stiell IG. A study to derive a clinical decision rule for triage of emergency department patients with chest pain: design and methodology. BMC Emergency Medicine 2008; 8: 3.
    CrossrefPubMedGoogle Scholar
  • 23 Green M, Björk J, Forberg J, Ekelund U, Edenbrandt L, Ohlsson M. Comparison between neural networks and multiple logistic regression to predict acute coronary syndrome in the emergency room. Artificial Intelligence in Medicine 2006; 38 (03) 305-318.
    CrossrefPubMedGoogle Scholar
  • 24 Carlson JN, Das S, De la Torre F. et al. A Novel Artificial Intelligence System for Endotracheal Intubation. Prehospital Emergency Care. 2016: 1-5.
    PubMedGoogle Scholar
  • 25 McCaig LF, Burt CW. Understanding and Interpreting the National Hospital Ambulatory Medical Care Survey: Key Questions and Answers. Annals of Emergency Medicine 2012; 60 (06) 716-721.
    CrossrefPubMedGoogle Scholar
  • 26 Lum HD, Studenski SA, Degenholtz HB, Hardy SE. Early hospital readmission is a predictor of one-year mortality in community-dwelling older Medicare beneficiaries. J Gen Intern Med 2012; 27 (11) 1467-1474.
    CrossrefPubMedGoogle Scholar
  • 27 Centers for Disease Control and Prevention NHAMCS micro-data file documentation [cited 2017 Jan 19]. 2013 Available from: ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHAMCS/doc13_ed.pdf.
    PubMedGoogle Scholar
  • 28 Stoltzfus JC. Logistic regression: a brief primer. Academic Emergency Medicine 2011; 18 (10) 1099-1104.
    CrossrefPubMedGoogle Scholar
  • 29 Zhang X, Liu Y, Yang M, Zhang T, Young AA, Li X. Comparative study of four time series methods in forecasting typhoid fever incidence in China. PLoS One 2013; 8 (05) e63116.
    CrossrefPubMedGoogle Scholar
  • 30 Zhao Z, Xin H, Ren Y, Guo X. Application and comparison of BP neural network algorithm in MATLAB. International Conference on Measuring Technology and Mechatronics Automation (ICMTMA). 2010
    PubMedGoogle Scholar
  • 31 Vijayarani S, Ilamathi MJ, Nithya M. Preprocessing Techniques for Text Mining-An Overview. International Journal of Computer Science and Communication Networks 2015; 5 (01) 7-16.
    PubMedGoogle Scholar
  • 32 Zhang Y, Jin R, Zhou Z-H. Understanding bag-of- words model: a statistical framework. International Journal of Machine Learning and Cybernetics 2010; 1 1-4 43-52.
    CrossrefPubMedGoogle Scholar
  • 33 Zhang X, Cowan BR, Bluemke DA. et al. Atlas- based quantification of cardiac remodeling due to myocardial infarction. PLoS One 2014; 9 (10) e110243.
    CrossrefPubMedGoogle Scholar
  • 34 Zhang X, Ambale-Venkatesh B, Bluemke DA. et al. Information maximizing component analysis of left ventricular remodeling due to myocardial infarction. Journal of Translational Medicine 2015; 13 (01) 1.
    CrossrefPubMedGoogle Scholar
  • 35 Leegon J, Jones I, Lanaghan K, Aronsky D. Predicting hospital admission for Emergency Department patients using a Bayesian network. AMIA Annual Symposium Proceedings. 2005: 1022.
    PubMedGoogle Scholar
  • 36 Wuerz RC, Milne LW, Eitel DR, Travers D, Gilboy N. Reliability and validity of a new five-level triage instrument. Academic Emergency Medicine 2000; 7 (03) 236-242.
    CrossrefPubMedGoogle Scholar
  • 37 Botsis T, Nguyen MD, Woo EJ, Markatou M, Ball R. Text mining for the Vaccine Adverse Event Reporting System: medical text classification using informative feature selection. Journal of the American Medical Informatics Association 2011; 18 (05) 631-638.
    CrossrefPubMedGoogle Scholar
  • 38 Husk G, Akhtar S. Chief complaints, emergency department clinical documentation systems, and the challenge of dealing with the patient’s own words. Academic Emergency Medicine 2007; 14 (01) 69-73.
    CrossrefPubMedGoogle Scholar
  • 39 Thompson DA, Yarnold PR, Williams DR, Adams SL. Effects of actual waiting time, perceived waiting time, information delivery, and expressive quality on patient satisfaction in the emergency department. Annals of Emergency Medicine 1996; 28 (06) 657-665.
    CrossrefPubMedGoogle Scholar
  • 40 Day FC, Schriger DL, La M. Automated linking of free-text complaints to reason-for-visit categories and International Classification of Diseases diagnoses in emergency department patient record databases. Annals of Emergency Medicine 2004; 43 (03) 401-409.
    CrossrefPubMedGoogle Scholar
  • 41 Dybowski R, Gant V. Clinical applications of artificial neural networks. Cambridge: Cambridge University Press; 2001
    Google Scholar
  • 42 Lo-Ciganic WH, Donohue JM, Thorpe JM. et al. Using machine learning to examine medication adherence thresholds and risk of hospitalization. Medical Care 2015; 53 (08) 720-728.
    CrossrefPubMedGoogle Scholar
  • 43 Becalick DC, Coats TJ. Comparison of artificial intelligence techniques with UKTRISS for estimating probability of survival after trauma. UK Trauma and Injury Severity Score. The Journal of Trauma 2001; 51 (01) 123-133.
    CrossrefPubMedGoogle Scholar
  • 44 Hsich E, Gorodeski EZ, Blackstone EH, Ishwaran H, Lauer MS. Identifying important risk factors for survival in patient with systolic heart failure using random survival forests. Circulation Cardiovascular Quality and Outcomes 2011; 4 (01) 39-45.
    CrossrefPubMedGoogle Scholar
  • 45 Parsaeian M, Mohammad K, Mahmoudi M, Zeraati H. Comparison of logistic regression and artificial neural network in low back pain prediction: second national health survey. Iranian Journal of Public Health 2012; 41 (06) 86.
    PubMedGoogle Scholar
  • 46 Song JH, Venkatesh SS, Conant EA, Arger PH, Sehgal CM. Comparative analysis of logistic regression and artificial neural network for computer-aided diagnosis of breast masses. Academic Radiology 2005; 12 (04) 487-495.
    CrossrefPubMedGoogle Scholar
  • 47 Forberg JL, Green M, Bj ork J. et al. In search of the best method to predict acute coronary syndrome using only the electrocardiogram from the emergency department. Journal of Electrocardiology 2009; 42 (01) 58-63.
    CrossrefPubMedGoogle Scholar
  • 48 Gholipour C, Rahim F, Fakhree A, Ziapour B. Using an Artificial Neural Networks (ANNs) Model for Prediction of Intensive Care Unit (ICU) Outcome and Length of Stay at Hospital in Traumatic Patients. Journal of Clinical and Diagnostic Research 2015; 9 (04) OC19-23.
    PubMedGoogle Scholar
  • 49 Tsai PF, Chen PC, Chen YY. et al. Length of Hospital Stay Prediction at the Admission Stage for Cardiology Patients Using Artificial Neural Network. Journal of Healthcare Engineering 2016; 2016: 7035463.
    PubMedGoogle Scholar
  • 50 Sinha M, Khor KN, Amresh A, Drachman D, Frechette A. The use of a kiosk-model bilingual self- triage system in the pediatric emergency department. Pediatr Emerg Care 2014; 30 (01) 63-68.
    CrossrefPubMedGoogle Scholar
  • 51 Jaimes F, Farbiarz J, Alvarez D, Martínez C. Comparison between logistic regression and neural networks to predict death in patients with suspected sepsis in the emergency room. Critical Care 2005; 9 (02) 1.
    CrossrefPubMedGoogle Scholar
  • 52 Bartfay E, Mackillop W, Pater J. Comparing the predictive value of neural network models to logistic regression models on the risk of death for small cell lung cancer patients. European Journal of Cancer Care 2006; 15 (02) 115-124.
    CrossrefPubMedGoogle Scholar
  • 53 Dexheimer JW, Leegon J, Aronsky D. Predicting hospital admission at triage in an emergency department. AMIA Annual Symposium Proceedings. 2007: 937.
    PubMedGoogle Scholar
  • 54 Tanabe P, Gimbel R, Yarnold PR, Adams JG. The Emergency Severity Index (version 3) 5-level triage system scores predict ED resource consumption. J Emerg Nurs 2004; 30 (01) 22-29.
    CrossrefPubMedGoogle Scholar
  • 55 McHugh M, Tanabe P, McClelland M, Khare RK. More patients are triaged using the Emergency Severity Index than any other triage acuity system in the United States. Academic Emergency Medicine 2012; 19 (01) 106-109.
    CrossrefPubMedGoogle Scholar
  • 56 Wuerz R. Emergency severity index triage category is associated with six-month survival. ESI Triage Study Group. Academic Emergency Medicine 2001; 8 (01) 61-64.
    PubMedGoogle Scholar
  • 57 Hirshon JM, Warner M, Irvin CB. et al. Research using emergency department-related data sets: current status and future directions. Academic Emergency Medicine 2009; 16 (11) 1103-1109.
    CrossrefPubMedGoogle Scholar
  • 58 Roberts A, Gaizauskas R, Hepple M. et al. The CLEF corpus: semantic annotation of clinical text. AMIA Annual Symposium Proceedings 2007; 2007: 625-629.
    PubMedGoogle Scholar
  • 59 Gurulingappa H, Rajput AM, Roberts A, Fluck J, Hofmann-Apitius M, Toldo L. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. Journal of Biomedical Informatics 2012; 45 (05) 885-892.
    CrossrefPubMedGoogle Scholar
  • 60 Bodenreider O. The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Research 2004; 32 (Suppl. 01) D267-D270.
    CrossrefPubMedGoogle Scholar
  • 61 Wallach HM. Topic modeling: beyond bag-of- words. Proceedings of the 23rd International Conference on Machine Learning. 2006
    PubMedGoogle Scholar
  • 62 Newman D, Karimi S, Cavedon L. Using Topic Models to Interpret MEDLINE’s Medical Subject Headings. Proceedings of the 22nd Australasian Joint Conference on Advances in Artificial Intelligence (AI ‘09).. Berlin, Heidelberg: Springer; 2009: 270-279.
    Google Scholar
  • 63 Nadkarni PM, Ohno-Machado L, Chapman WW. Natural language processing: an introduction. Journal of the American Medical Informatics Association 2011; 18 (05) 544-551.
    CrossrefPubMedGoogle Scholar
  • 64 Yu W, Liu T, Valdez R, Gwinn M, Khoury MJ. Application of support vector machine modeling for prediction of common diseases: the case of diabetes and pre-diabetes. BMC Medical Informatics and Decision Making 2010; 10 (01) 16.
    CrossrefPubMedGoogle Scholar
  • 65 Özçift A. Random forests ensemble classifier trained with data resampling strategy to improve cardiac arrhythmia diagnosis. Computers in Biology and Medicine 2011; 41 (05) 265-271.
    CrossrefPubMedGoogle Scholar
  • 66 Subasi A. Automatic recognition of alertness level from EEG by using neural network and wavelet coefficients. Expert Systems with Applications 2005; 28 (04) 701-711.
    CrossrefPubMedGoogle Scholar
  • 67 Bar Y, Diamant I, Wolf L, Greenspan H. Deep learning with non-medical training used for chest pathology identification. SPIE Medical Imaging. 2015: 94140V.
    PubMedGoogle Scholar