Download PDF
Appl Clin Inform 2022; 13(03): 700-710
DOI: 10.1055/a-1863-7176
Research Article

Application of a Machine Learning–Based Decision Support Tool to Improve an Injury Surveillance System Workflow

Jesani Catchpoole
1   Queensland Injury Surveillance Unit, Royal Brisbane and Women's Hospital, Metro North Hospital and Health Service, Queensland, Australia
2   Jamieson Trauma Institute, Royal Brisbane and Women's Hospital, Metro North Hospital and Health Service, Queensland, Australia
3   Australian Centre for Health Services Innovation (AusHSI), School of Public Health and Social Work
,
Gaurav Nanda
4   Purdue University, School of Engineering Technology, West Lafayette, Indiana, United States
,
Kirsten Vallmuur
2   Jamieson Trauma Institute, Royal Brisbane and Women's Hospital, Metro North Hospital and Health Service, Queensland, Australia
3   Australian Centre for Health Services Innovation (AusHSI), School of Public Health and Social Work
,
Goshad Nand
1   Queensland Injury Surveillance Unit, Royal Brisbane and Women's Hospital, Metro North Hospital and Health Service, Queensland, Australia
,
Mark Lehto
5   Purdue University, School of Industrial Engineering, West Lafayette, Indiana, United States
› Author Affiliations Funding This work was supported by an Australian Research Council Discovery Grant (grant no.: DP170103136).
› Further Information
   Permissions and Reprints

Abstract

Background Emergency department (ED)-based injury surveillance systems across many countries face resourcing challenges related to manual validation and coding of data.

Objective This study describes the evaluation of a machine learning (ML)-based decision support tool (DST) to assist injury surveillance departments in the validation, coding, and use of their data, comparing outcomes in coding time, and accuracy pre- and postimplementations.

Methods Manually coded injury surveillance data have been used to develop, train, and iteratively refine a ML-based classifier to enable semiautomated coding of injury narrative data. This paper describes a trial implementation of the ML-based DST in the Queensland Injury Surveillance Unit (QISU) workflow using a major pediatric hospital's ED data comparing outcomes in coding time and pre- and postimplementation accuracies.

Results The study found a 10% reduction in manual coding time after the DST was introduced. The Kappa statistics analysis in both DST-assisted and -unassisted data shows increase in accuracy across three data fields, that is, injury intent (85.4% unassisted vs. 94.5% assisted), external cause (88.8% unassisted vs. 91.8% assisted), and injury factor (89.3% unassisted vs. 92.9% assisted). The classifier was also used to produce a timely report monitoring injury patterns during the novel coronavirus disease 2019 (COVID-19) pandemic. Hence, it has the potential for near real-time surveillance of emerging hazards to inform public health responses.

Conclusion The integration of the DST into the injury surveillance workflow shows benefits as it facilitates timely reporting and acts as a DST in the manual coding process.

Keywords

injury surveillance - machine learning - decision support

Protection of Human and Animal Subjects

Human and/or animal subjects were not included in the project. In addition, the study analyzed nonidentifiable data; therefore, consent from patients was not required.




Publication History

Received: 31 January 2022

Accepted: 26 May 2022

Accepted Manuscript online:
29 May 2022

Article published online:
13 July 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Chen L, Vallmuur K, Nayak R. Injury narrative text classification using factorization model. BMC Med Inform Decis Mak 2015; 15 (Suppl. 01) S5-S5
    CrossrefPubMedGoogle Scholar
  • 2 Nanda G, Vallmuur K, Lehto M. Improving autocoding performance of rare categories in injury classification: is more training data or filtering the solution?. Accid Anal Prev 2018; 110: 115-127
    CrossrefPubMedGoogle Scholar
  • 3 Nanda G, Vallmuur K, Lehto M. Semi-automated text mining strategies for identifying rare causes of injuries from emergency room triage data. IISE Trans Healthc Syst Eng 2019; 9 (02) 157-171
    CrossrefPubMedGoogle Scholar
  • 4 Nanda G, Vallmuur K, Lehto M. Intelligent human-machine approaches for assigning groups of injury codes to accident narratives. Saf Sci 2020; 125: 104585
    CrossrefPubMedGoogle Scholar
  • 5 Vallmuur K. Machine learning approaches to analysing textual injury surveillance data: A systematic review. Accid Anal Prev 2015; 79: 41-49
    CrossrefPubMedGoogle Scholar
  • 6 Vallmuur K, Marucci-Wellman HR, Taylor JA, Lehto M, Corns HL, Smith GS. Harnessing information from injury narratives in the ‘big data’ era: understanding and applying machine learning for injury surveillance. Inj Prev 2016; 22 (Suppl. 01) i34-i42
    CrossrefPubMedGoogle Scholar
  • 7 National Injury Surveillance Unit. National Data Standards for Injury Surveillance Ver 2.1. Available at: https://nla.gov.au/nla.cat-vn730536
    PubMedGoogle Scholar
  • 8 Mitchell TM. Machine Learning. 1st ed.. Ithaca, NY: McGraw-Hill, Inc.; 1997
    Google Scholar
  • 9 Chae YM, Kim HS, Tark KC, Park HJ, Ho SH. Analysis of healthcare quality indicator using data mining and decision support system. Expert Syst Appl 2003; 24 (02) 167-172
    CrossrefPubMedGoogle Scholar
  • 10 Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: A systematic review. Arch Intern Med 2003; 163 (12) 1409-1416
    CrossrefPubMedGoogle Scholar
  • 11 Kong G, Xu D, Yang J, Wang T, Jiang B. Evidential reasoning rule-based decision support system for predicting ICU admission and in-hospital death of trauma. IEEE Trans Syst Man Cybern Syst 2020; 51 (11) 7131-7142
    CrossrefPubMedGoogle Scholar
  • 12 Romero-Brufau S, Wyatt KD, Boyum P, Mickelson M, Moore M, Cognetta-Rieke C. Implementation of artificial intelligence-based clinical decision support to reduce hospital readmissions at a regional hospital. Appl Clin Inform 2020; 11 (04) 570-577
    Article in Thieme ConnectPubMedGoogle Scholar
  • 13 Shah M, Shu D, Prasath VBS, Ni Y, Schapiro AH, Dufendach KR. Machine learning for detection of correct peripherally inserted central catheter tip position from radiology reports in infants. Appl Clin Inform 2021; 12 (04) 856-863
    Article in Thieme ConnectPubMedGoogle Scholar
  • 14 Soldaini L, Cohan A, Yates A, Goharian N, Frieder O. Retrieving Medical Literature for Clinical Decision Support. Paper presented at: European Conference on Information Retrieval, Advances in Information Retrieval; 2015, 2015;538–549
    PubMedGoogle Scholar
  • 15 Teng AK, Wilcox AB. A review of predictive analytics solutions for sepsis patients. Appl Clin Inform 2020; 11 (03) 387-398
    Article in Thieme ConnectPubMedGoogle Scholar
  • 16 Goh YM, Ubeynarayana CU. Construction accident narrative classification: An evaluation of text mining techniques. Accid Anal Prev 2017; 108: 122-130
    CrossrefPubMedGoogle Scholar
  • 17 Goldberg DM. Characterizing accident narratives with word embeddings: Improving accuracy, richness, and generalizability. J Safety Res 2022; 80: 441-455
    CrossrefPubMedGoogle Scholar
  • 18 Nanda G, Grattan KM, Chu MT, Davis LK, Lehto MR. Bayesian decision support for coding occupational injury data. J Safety Res 2016; 57: 71-82
    CrossrefPubMedGoogle Scholar
  • 19 Zhong B, Pan X, Love PED, Ding L, Fang W. Deep learning and network analysis: Classifying and visualizing accident narratives in construction. Autom Construct 2020; 113: 103089
    CrossrefPubMedGoogle Scholar
  • 20 Bussone A, Stumpf S, O'Sullivan D. The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems. 2015; 160-169 DOI: 10.1109/ICHI.2015.26.
    CrossrefPubMedGoogle Scholar
  • 21 Lehto MR, Sorock GS. Machine learning of motor vehicle accident categories from narrative data. Methods Inf Med 1996; 35 (4-5): 309-316
    Article in Thieme ConnectPubMedGoogle Scholar
  • 22 Zhu W, Lehto MR. Decision support for indexing and retrieval of information in hypertext systems. Int J Hum Comput Interact 1999; 11 (04) 349-371
    CrossrefPubMedGoogle Scholar
  • 23 Leman S, Lehto MR. Interactive decision support system to predict print quality. Ergonomics 2003; 46 (1–3): 52-67
    CrossrefPubMedGoogle Scholar
  • 24 Choe P, Lehto MR, Shin GC, Choi KY. Semiautomated identification and classification of customer complaints. Hum Factors Ergon Manuf Serv Ind 2013; 23 (02) 149-162
    CrossrefPubMedGoogle Scholar
  • 25 McKenzie K, Scott DA, Waller GS, Campbell M. Reliability of routinely collected hospital data for child maltreatment surveillance. BMC Public Health 2011; 11 (01) 8-8
    CrossrefPubMedGoogle Scholar
  • 26 Liberati EG, Ruggiero F, Galuppo L. et al. What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci 2017; 12 (01) 113-113
    Google Scholar
  • 27 May CR, Mair F, Finch T. et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci 2009; 4 (01) 29-29
    CrossrefPubMedGoogle Scholar
  • 28 Mishuris RG, Palmisano J, McCullagh L. et al. Using normalisation process theory to understand workflow implications of decision support implementation across diverse primary care settings. BMJ Health Care Inform 2019; 26 (01) e100088
    CrossrefPubMedGoogle Scholar