Background and Significance
Sepsis is a result of overwhelming immune response to combat an infection, leading
to widespread inflammation and subsequent damage to organs and tissue impairment.[1 ] At least 1.7 million adults in the United States develop sepsis and of those 270,000
results in death annually.[2 ]
Sepsis, besides being associated with high mortality and increased lengths of intensive
care unit (ICU) and hospital stay, is significantly associated with severe morbidity
such as multiple organ failure, critical illness myopathy, and acute delirium.[3 ] Sepsis was the most expensive condition treated in hospitals in the United States
in 2013.[4 ] Sepsis was also the most expensive condition billed to Medicare and Medicaid, with
aggregate hospital costs of $14.55 billion (8.2% of national health-care costs) billed
to Medicare and $3.35 billion (5.3% of national health care costs) billed to Medicaid
in 2013.[4 ] Along with monetary health care costs of sepsis treatment for the individual, hospitals,
government, and the taxpayers, there is a lasting impact on lives of sepsis survivors
and their caregivers affecting their health-related quality of life (HRQOL).[5 ]
Sepsis diagnosis criteria have changed many times due to complexities and lack of
accuracy in diagnosis of sepsis and septic shock. The most recent definition, Sepsis-3
criteria, recommended use of the Quick Sequential Organ Failure Assessment (qSOFA)
score for early diagnosis of sepsis: (1) respiratory rate (RR) ≥22/min (2) change
in mental status, and (3) systolic blood pressure (SBP) ≤100 mm Hg.[6 ] Sepsis-3 defined sepsis as suspected or documented infection and an acute increase
of ≥2 of the SOFA points; and septic shock as subset of sepsis in which the severe
acute circulatory and cellular metabolic failure leads to increased mortality.[6 ] Sepsis-3 also defined septic shock as presence of sepsis and vasopressor therapy
needed to elevate mean arterial pressure ≥ 65 mm Hg and lactate >2 mmol/L (18 mg/dL)
despite adequate fluid resuscitation.[6 ]
Early goal-directed therapy (EGDT) for sepsis includes rapid and early recognition
of sepsis, early resuscitation if applicable, early antibiotics, and early eradication
of the source of infection to improve patient outcomes.[7 ]
[8 ]
[9 ] Recognition of sepsis in ED was associated with higher compliance to the Surviving
Sepsis Campaign 3-hour resuscitation and management bundle leading to decreases in
patient mortality.[9 ] Machine learning (ML) models have been useful in predicting sepsis and in decreasing
sepsis mortality rates and 30-day readmission rates for inpatients.[10 ]
[11 ] Recently, Teng and Wilcox reviewed several models for sepsis prediction, including
different ML model structures, feature selection, and data sample size methods, and
demonstrated such predictive analytics tool are beneficial in early identification
of sepsis and thereby improve patient outcomes.[12 ] This study moves one step further and suggests potential data elements to be included
in such models to improve their prediction/detection performance without introducing
any delay in running the models when applied in clinical workflow.
Prehospital data, also referred as emergency medical service (EMS) data or ambulance
records, are data elements recorded by the ambulance or prehospital care services.
Prehospital data are sources of untapped information that can be valuable in early
identification of sepsis and in facilitating clinical decision-making for EGDT in
ED. Advancement in technology providing interoperability between prehospital and ED
data enable use of prehospital data for health information exchange and clinical decision-making
in ED.[13 ]
Despite several efforts in predicting early detection of sepsis in ED, by using ML
approaches, the importance of using prehospital data has not been reviewed yet. Therefore,
this integrative review of literature first summarizes the current use of prehospital
data in developing ML-based clinical decision models in ED, and then focuses on identifying
potential contributing data elements in prediction of early detection of sepsis in
ED using ML models. Aims of this review of literature are to (1) summarize the use
of prehospital data in developing ED prediction models, (2) review use of ML approaches
in sepsis prediction/early identification in ED, and (3) introduce potential prehospital
data elements that can be helpful in prediction of sepsis in ML models.
Methods
The literature search strategy included two separate search methods related to use
of pre-hospital data in ED and prediction or identification of sepsis in ED using
ML models. [Fig. 1 ] shows the breakdown structure of these studies based on their purposes.
Fig. 1 Search strategy for prehospital data and sepsis in emergency department (ED).
The literature search for the use of prehospital data in ED included English language,
peer-reviewed articles published between the years July 2015 and July 2020 in PubMed,
the Institute of Electrical and Electronics Engineers (IEEE) Xplore, and Google Scholar.
The inclusion criteria for this review included studies that used documented prehospital
data prior to arrival to ED for the purpose of decision-making in ED for adult patients
(age ≥18 years). Similarly, the exclusion criteria for this search related to use
of prehospital data include any nonpeer reviewed articles that did not use prehospital
data or data documented prior to arrival to ED. Also included were articles for clinical
decision-making in ED for the purpose of ED operations, diagnosis in ED, or treatment
in ED. Articles that were not in English language were excluded from the review.
Initial search strategy using key terms [“prehospital data” OR Emergency Service]
AND [Emergency Department] AND [sepsis OR septic] AND [machine learning OR artificial
intelligence] generated 16 articles. “Prehospital data” were used to reduce the number
of nonrelevant articles produced without the quotation marks. After exploring the
articles through manual inspection, none of the articles found used prehospital data
as variables in ML models for clinical decision-making related to sepsis in ED. Therefore,
the search was divided into two different search strategies. One search strategy focused
on the use of prehospital data for decision-making in ED. Second search strategy focused
on sepsis in ED using ML regardless of the use of prehospital data.
In the search related to the use of prehospital data in ED, the key terms included
[“prehospital data” OR “Emergency Medical Service Data”] AND Emergency Department
AND [predict OR prediction model OR machine learning OR artificial intelligence].
In total, 200 articles were found between the three databases which were reduced to
113 with filters for within 5 years and peer-reviewed articles. After removal of duplicates
from Google Scholar, 80 articles were manually assessed for relevance. Relevance was
based on use of prehospital data collected prior to arrival in ED as a data element
or variable in clinical decision-making in ED. In total, 12 articles related to use
of prehospital data in clinical decision-making in ED were included in this review.
The literature search for the ML models for early identification of sepsis in ED included
English language, peer-reviewed articles published between the years 2015 to 2020
in PubMed and IEEE Xplore. Key terms included [sepsis OR Septic] AND Emergency Department
AND [prediction OR machine learning OR Artificial Intelligence]. PubMed initially
generated 28 articles and IEEE generated 9 articles that included peer-reviewed conference
papers. The filters of within 5 years, peer-review, and assessment for duplicates
did not change the total numbers of articles. After manually inspecting for relevance,
12 additional studies were included in the review. Criteria for relevance included
use of ML or artificial intelligence in predictions related to sepsis using data that
are generally collected in ED.
Results
Use of Prehospital Data
Limited prehospital data have been used in developing ED prediction models to improve
ED outcomes. Of the 12 studies reviewed that used prehospital data for ED decision-making;
1 study improved ED operations by forecasting number of arrivals to reduce overcrowding
in ED[14 ]; 4 studies predicted patient outcomes such as in-hospital mortality, survival rate,
and return of spontaneous circulation (ROSC)[15 ]
[16 ]
[17 ]
[18 ]; 2 studies identified specific risk or early warning scores for patient outcomes
such as higher acuity or short-term in-hospital mortality[19 ]
[20 ]; 2 studies made decisions in the field prior to arrival to ED such as triage patient
disposition to specialized centers with appropriate medical capabilities (e.g., trauma
centers or aortic surgery centers)[21 ]
[22 ]; and 3 studies identified time-sensitive conditions.[21 ]
[22 ]
[23 ] All studies reported improvement in ED operations or ED outcomes as described in
[Table 1 ].
Table 1
Summary of studies using prehospital data for clinical decision-making in emergency
department (ED)
Reference
Model
Target(s)
Target class/distribution
Predictors
Performance/result
Pre-EDA
Post-EDA
TTP
[13 ]
Poisson's time-series regression model
No. of arrivals within a forecasting horizon of 1, 2, and 3 hours
Numeric
Arrival notification
Sampling the data on 5-minute intervals; the number of arrivals in the previous 1,
2, and 3 hours
1 hour prior to EDA
Notification prior to EDA contributed to higher forecasting accuracy especially on
a 1-hour horizon
[23 ]
Multivariable logistic regression model and SuperLearner (SL) model
Prediction for acute aortic syndromes (AAS) and performance of undertriage and overtriage
Binary (62.4% positive)
Referred from PC; helicopter Tpt; Htn; smoking; DM; EDA; pain: sudden, abd, thoracic,
ripping/tearing, migrating; severe pain intensity; pain scale rate pre-analgesic;
focal neuro deficit and type; Htn or use of vasodilatators at EDA; mechanical ventilation
at EDA; prehospital CA
EDA date and time; DOB; sex; age; origin of the patient; Hx of abd and thoracic AA;
SG: aortic, valve, CABG; connective tissue disease; aortic manipulation; AD; familial
AD; Inflammatory vasculitides; pulse deficit; SBP differential; new AI murmur; use
of vasoactive Rx; CV collapse; ECG; ST elevation; pericardial effusion; cardiac tamponade;
syncope; GCS < 8; GI bleeding; hemoptysis; LI; In-hospital CA; ADDRS; TTE; CT-scan
from PC and/or AC; need for SG; Time-to-SG; SAPS II; Final VASC Dx and alternative
Dx hospital LOS; status at hospital D/C
24 hours after EDA
SL model with AUC of 0.87 in the training cohort and 0.73 in the validation cohort
outperformed logistic regression model with AUC of 0.68 in the training cohort and
0.67 in the validation cohort (p < 0.05)
[18 ]
The National Early Warning Score (NEWS) logistic regression model
Death within 1 day of emergency medical service dispatch
Binary (1.1% positive)
RR; SpO2 ; use of supplemental oxygen; temperature; SBP; HR; Level of consciousness
–
prior to arrival to ED
The AUC for primary outcome of death within 1 day was 0.840 (95% CI: 0.823-0.858)
[19 ]
Composite risk score based on gradient boosting model
Outcome acuity level (transferred to floor, transferred to critical care) or in hospital
mortality
Multiclass (1. admissions: 52%; 2. critical care: 5.8%; 3. mortality within 2 days:
1.2%)
Age, gender, DOB, hour and month that the call was received, distance to the nearest
ED, and prior contacts with EMD center by the patient, clinical characteristics of
the call, times to reach the incident, on scene, and to the hospital, VS, patient
Hx, Rx and procedures administered, and the clinical findings of ambulance staff
–
Prior to EDA
ML models based on ambulance data with AUC of 0.79 (0.78–0.80) for hospital admission,
AUC of 0.79 (0.77–0.81) for critical care, and AUC of 0.89 (0.87–0.92) for 2-day mortality
outperformed NEWS score with AUC of 0.66 (0.65–0.67), 0.75 (0.73–0.77), and 0.85(0.81–0.88),
respectively
[22 ]
Deep learning approach with CNN and LSTM
Shock/no shock decision. detection of VF
Binary
ECG records from out-of- hospital CA
–
<3 seconds
Enabled an accurate shock/ no shock Dx in <3 seconds; decision to shock/ no shock
was considered an important prehospital data for ED
[16 ]
ROTEM-based logistical regression model
ROSC in ED
Binary (31% positive)
POW during CPR; presence of activation; initial rhythm: VF, VT, PEA, asystole
VF, VT, PEA, asystole; time from onset to blood sampling after ED admission; WBC;
HgB; platelet count; PT; INR; aPTT; fibrinogen; FDP; D-dimer and lactate; EXTEM; FIBTEM
After ROTEM results in ED
Rotem based predictors of ROSC model had low sensitivity (40.9%) but high specificity
(94.7%) and accuracy (75.0%)
[14 ]
Random forest
30-day survival rate post–out-of-hospital CA
Binary
Initial rhyhm; Age; time- CA to CPR, CA to 911 call, EMS dispatch to arrival; AED;
Witness Arrest, AED before ambulance arrival, cause of CA, calendar year, region,
time during day, and sex
–
prior to arrival to ED
Initial rhythm; age; early CPR (CPR, time to CPR and CPR before arrival of EMS) were
top 3 most important predictors to 30-day survival post out of hospital CA
[17 ]
Random forest
1-year survival rate post–out-of-hospital CA
Binary (6.35% positive for training set and 4.27% positive for test set)
Medical Hx; ADL before CA; healthy or mild, moderate, and severe disability; PVS;
Site; POW; bystander CPR; patient status; cardiac pulmonary, cardiac, and resp arrest;
ROSC; presence of SB and pulse; RPD; LPD; PLR; ECG; JCS; CPR; use of CCD; AED; airway
securing and method; VA; adrenaline use; removal of airway obstruction; ROSC during
Tpt; change of ECG waveform during Tpt; no. of EMT; Time: between EMS perception and
contact, EMS perception and CPR, EMS perception and EDA, EMS arrival and contact,
EMS contact and CPR, CPR and EDA; other party
–
After EDA
Trained using 35 variables for 1-year survival showed AUC values of 0.943 (95% CI
[0.930, 0.955]) and 0.958 (95% CI [0.948, 0.969]), respectively
[24 ]
Gradient boosting decision tree
Injury severity score (ISS) ≥ 16, linked to lower mortality rate; resource-based outcome
measure to define need for specialized trauma care
NA
Age, gender, VS, GCS, SBP, DBP, HR, RR, Intubation, SpO2 , Mechanism of injury: MVA, Motorcycle accident, Moped, scooter accident- MVA, pedestrian-
MVA, different, gunshot, stab wound, struck with blunt object, fall at same level
and at higher level; asphyxia, burns, body surface, injury type, penetrating injury
to head, neck, torso, and extremities proximal to elbow and knee, flail chest, paralysis,
open or depressed skull fracture
–
Prior to EDA
Aim of the study is to create a new prediction model to attain acceptable undertriage
rates and to minimize mortality rates for patients in need of specialized trauma care.
Validation and performance results are pending
[21 ]
Prediction of acute coagulopathy of trauma score
Acute traumatic coagulopathy INR > 1.5
Binary (5.9% positive)
Shock index, age, GCS, mechanism of injury, intubation, CPR, chest decompression
–
Prior to EDA
The prediction of acute coagulopathy of trauma score with AUC of 0.80 (0.72–0.88)
outperformed coagulation of severe trauma score with AUC of 0.70 (0.60–0.80; p = 0.032)
[20 ]
Comparison of logistic regression, random forest, support vector, eXtreme gradient
boosting
Presence of large vessel occlusion
Binary (43.3% positive)
Pre EDA (level 1): age; gender; presence of speech deficits; facial weakness; Lt and
Rt sided facial weakness; limb weakness; Lt and Rt sided limb weakness
Post-EDA (Level 2): preexisting medical conditions: DM, Htn; current and Hx of smoking;
SBP and DBP; GCS; AF Hx, atherosclerosis, cardioembolism, and valvular heart disease;
level 3: CT scan results
Level 1 prior to EDA; level 2 after H&P in ED or EMS; Level 3 after CT scan results
are read
Comparing all four machine learning algorithms, the eXtreme Gradient Boosting method
gave robust and accurate performance in all 3 levels.
[15 ]
Regression models
Death within 14 days post trauma
Binary (22.8% positive)
Gender; age; level of pupil reactivity at admission; GCS at the site
GCS at EDA; motor component score of the GCS; presence of hypoxia and hypotension;
midline shift bigger than 5 mm; brain herniation detected on CT (defined as effacement
of the third ventricle or the basal cisterns); hemorrhage: SAH, epidural, subdural
and intracerebral
After CT scan results are read in ED
When predicting in-hospital mortality, random forest was the best performing model
(AUC = 0.838), closely followed by generalized partial least squares (AUC = 0.831),
stochastic gradient boosting (AUC = 0.823), and penalized discriminant analysis (AUC = 0.803)
Abbreviations: AA, aortic aneurysm; Abd, abdominal; AC, aortic center; AD, aortic
dissection; ADDRS, aortic dissection detection risk score; ADL, activities of daily
living; AED, automatic external defibrillation; AF, atrial fibrillation; AI, aortic
insufficiency; aPTT, activated partial thromboplastin time; AUC, area under receiving
operating curve; CA, cardiac arrest; CABG, coronary arter bypass graft; CCD, chest
compression device; CI, confidence interval; CNN, convolutional neural networks; CPR,
cardio-pulmonary resuscitation; CT, computerized tomography; CV, cardiovascular; D/C,
discharge; DBP, diastolic blood pressure; DM, diabetes mellitus; DOB, date of birth;
Dx, diagnosis; ECG, electrocardiogram; ED, emergency department; EDA, ED arrival time;
EMD, emergency medical dispatch; EMS, emergency medical service; EMT, emergency medical
technician; EXTEM, extrinsic coagulation pathway; FDP, fibrin degradation products;
FIBTEM, function of fibrinogen pathway in the extrinsic pathway; GCS, Glasgow coma
scale; GI, gastrointestinal; H&P, history and physical; HgB, hemoglobin; HR, heart
rate; Htn, hypertension; Hx, history; INR, International Normalized Ratio; JCS, Japanese
coma scale; LI, limb ischemia; LOS, length of stay; LPD, left pupil diameter; LSTM,
long short-term memory; Lt, left; MVA, motor vehicle accident; Neuro, neurologic;
PC, primary center; PEA, pulseless electric activity; PLR, pupil light reflex; POW,
presence of witness; PT, prothrombin time; PVS, persistent vegetative state; Resp,
respiratory; ROSC, return of spontaneous circulation; ROTEM, thromboelastometry; RPD,
right pupil diameter; RR, respiratory rate; Rt, right; Rx, drugs; SAH, subarachnoid
hemorrhage; SAPS II, simplified acute physiology score II; SB, spontaneous breathing;
SBP, systolic blood pressure; SG, surgery; Site, site of incidence; SpO2 , oxygen saturation; TPT, transport; TTE, Transthoracic echocardiography; TTP, time
to predict (the earliest time that the prediction model can be run based on the predictors);
VA, vascular access; VASC, vascular; VF, ventricular fibrillation; VS, vital signs;
VT, ventricular tachycardia; WBC, white blood count.
Development and validation of ML models using prehospital data are useful in early
identification of time-sensitive conditions, such as trauma, stroke, and shockable
rhythms post–out-of-hospital cardiac arrest (CA).[16 ]
[21 ]
[22 ] For instance, prediction of acute coagulopathy of trauma (ACT) score, based on prehospital
data, was developed for early identification of ACT, a complication of trauma that
may require early goal-directed treatment of massive transfusion for patient survival.[22 ] Therefore, the early information that the prehospital data provide can be an asset
in improving time to predict time-sensitive conditions in ED.
Machine Learning Models Using Prehospital Data
Of the nine studies included in the review, common prehospital data elements used
in ML models targeting early detection of time-sensitive conditions or prediction
of patient outcomes include the following: (1) demographics, such as age and gender;
chief complaints[15 ]
[16 ]
[20 ]
[21 ]
[22 ]
[24 ]
[25 ]; (2) prehospital level of consciousness such as Glasgow coma scale (GCS)[19 ]
[20 ]
[22 ]
[25 ]; (3) prehospital vital signs recorded in the ambulance or in the field[16 ]
[21 ]
[22 ]; (4) prehospital actions, characterized as intubation, cardiopulmonary resuscitation
(CPR), and chest decompression[16 ]
[18 ]
[21 ]
[22 ]; or (5) prehospital situational assessments, described as mechanism of injury, witnessed
arrest or CPR initiation, and presence of speech deficit and facial and limb weakness
prior to arrival to ED.[15 ]
[18 ]
[21 ]
[22 ]
[Table 1 ] summarizes studies using prehospital data for clinical decision-making in ED based
on their targets and modeling approaches. Benefits of using prehospital data to achieve
ED outcomes were noted in all of the studies included in the review.
All 12 ML articles used prehospital data to impact ED clinical decision-making and
ED outcomes. Accordingly, [Table 1 ] divides the data elements used for each study into pre–emergency department arrival
(pre-EDA) and post-EDA and provides further insights about the earliest time that
the prediction model can be run based on the predictors (referred in the table as
time to predict, or TTP) in each study. Pre-EDA includes data from emergency call
centers, dispatches stations and ambulance records collected prior to arrival to ED.
Post-EDA includes data collected during hospitalization starting from the arrival
to ED.
Use of Machine Learning–Related to Sepsis in ED
Early and accurate prediction of sepsis in ED can improve patient outcomes such as
decreased inpatient mortality rate, length of stay, and readmission rates.[26 ] Advantage of ML models includes the ability to predict patient outcomes hours prior
to onset of actual outcome such as diagnosis of sepsis and septic shock.[27 ]
[28 ]
[29 ]
[30 ]
[31 ] From the 12 studies included in this review, only 1 study used prospective research
design in a clinical setting.[26 ] All other studies used retrospective data for ML modeling. A meta-analysis by Fleuren
et al supported that ML models can accurately predict sepsis onset ahead of time in
ED, floor, and ICU, providing a novel approach to early identification of sepsis where
the biomarkers and screening tools, such as systematic inflammatory response syndrome
(SIRS) and SOFA criteria, fail to include all the clinical relevant information.[32 ] The meta-analysis also showed the impact of prediction hours before sepsis onset
on pooled area under receiving operating curve (AUC) for the ML models, further indicating
that the prediction hours before onset of sepsis with relatively high AUC through
ML model is plausible.[32 ]
Data Elements in Detection of Sepsis
The ML models used in the ED are evaluated based on their target, modeling approach,
data element characteristics, time to predict, and AUC. [Table 2 ] provides a list of studies found in the search and summarizes them based on their
target and modeling approach. It also includes a detailed list of variables used to
develop the ML models for each study. Furthermore, the table provides the AUC and
TTP which determine the earliest time that all required data elements for running
the model are collected and the performance of the model. Therefore, for each target,
by considering both TTP and prediction performance, this table helps the reader understand
the practicality of the developed models if applied within clinical workflow.
Table 2
Summary of studies with prediction/early identification of sepsis using machine learning
with their list of predictors in emergency department (ED)
Reference
Model
Target(s)
Predictors
Performance/result
[36 ]
Logistic regression with L1 regularization (LASSO)
30-day in-hospital mortality among septic patients
Mean RR; mean HR; NN50; TINN; power norm: very low, low, high frequency; LF/HF, ratio
of LF power to HF power; SBP; temp; GCS; age; approximate entropy; sample entropy;
detrended fluctuation analysis. TTP: 2 hours after EDA
12 selected HRVTS features outperformed model that only included patient demographics,
vital signs and one HRV measures taken at triage.
AUC: 0.82
[34 ]
Random forest, classification and regression tree (CART) model, logistic regression
model
28 days in-hospital mortality
SpO2; RR; BP; BUN; albumin; intubation in ED; need for vasopressors; age; RDW; potassium;
AST; HR; acuity level (triage), ED impression (Dx), CO2 (laboratory), ECG performed,
beta-blocker (home medication), cardiac dysrhythmia (primary medical history). TTP:
after ED stay
Machine learning outperformed statistical models.
AUC: 0.86
[27 ]
Support vector model
Dx of an infection
Age; gender; acuity; SBP; DBP; HR; pain scale, RR, SpO2; temperature, free text chief
complaint, free text nursing assessment. TTP: after ED stay
Best performing models included free text (NLP).
AUC: 0.89
[28 ]
Gradient tree boosting
Detect onset of sepsis, severe sepsis and septic shock
SBP; DBP; HR; RR; SpO2 and temperature. TTP: at the onset in ED, ICU and floor
Includes ED, ICU and floor. Early detection 4 hours before for septic shock. AUC:
0.92
[33 ]
Convolutional neural network plus softmax
Predict mortality over 72 hours and 28 days
5 BP; GCS; WBC; HgB; lymphocyte count; PT-INR; BUN; creatinine; bilirubin; AST; ALT;
troponin; pH; bicarbonate level; atypical lymphocyte; promyelocyte; metamyelocyte;
myelocyte; Sodium ion; potassium ion; albumin; BG; RDW-SD, MCV; RDW-CoV; Base excess;
MCH; MCHC; MAP; RR; temperature, HR, Age, Sex, qSOFA score, shock episode, liver cirrhosis,
DM; CRF; CHF; CVA; solid tumor, RI, UTI, STI; IAI; other infections, bacteremia, antibiotic
used within 24 hours. TTP: 24 hours after EDA
Significant comparisons between different models; outperformed all machine learning
and statistical models.
AUC: 0.94
[25 ]
Machine learning algodiagnostic (MLA)
In-hospital mortality
HR, RR, BP, SpO2, temperature. TTP: 1 hour after EDA
Prospective study noted decrease in sepsis-related in-hospital mortality rate, length
of stay and 30-day readmission rate 60.24%, 9.55% and 50.14%, respectively.
AUC: 0.91
[26 ]
Multivariate linear regression
Dx of sepsis
Age, HR, RR, temperature, SBP, DBP, MAP, SpO2, HR to SBP ratio. TTP: After EDA with
first vital signs.
HR: SBP ratio was higher in patients with sepsis than patients with trauma, stroke
and acute coronary syndrome. ML (74%) has higher accuracy than SIRS (34%).
[35 ]
Multivariate logistic regression, decision tree, and naïve Bayes' classifier
In-hospital mortality
SIRS criteria: temperature, HR, RR, WBC count. qSOFA criteria: SBP, GCS, RR
TTP: after EDA with first vital signs.
Low and similar accuracy between the three models with variables related to SIRS and
qSOFA. AUC: ranging from 0.622 to 0.696
[30 ]
SVM, gradient boosting machine with Bernoulli's loss, random forest, multivariate
adaptive regression splines, lasso and ridge regression
Septic shock within 24 hours of arrival
Sex; chief complaints; initial vital signs: SBP, DBP, HR, RR, SpO2 and temperature;
initial level of consciousness- AVPU scale; WBC, differential counts; RDW; platelet
count; PT; INR; fibrinogen; BUN; sodium; potassium; chloride, creatinine, AST, ALT,
ALP, total bilirubin; albumin; and C-reactive protein. TTP: 24 hours after arrival
All machine learning classifiers significantly outperformed statistically calculated
models. Embedding chief complaints was statistically significant in improving performance
AUC: ranging from 0.883 to 0.929.
[37 ]
eXtreme gradient boosting, light gradient boosting and random forest
3-day mortality among patients with suspected infection in the ED
The qSOFA criteria, including systolic blood pressure, respiratory rate, and mental
status.
TTP: After EDA with first vital signs
The machine learning models outperformed qSOFA score using the same variables in predicting
3-day mortality with suspected infections in ED
AUC: model 0.86.
[32 ]
Gradient-boosted tree models
Detection of sepsis
LA, shock index; WBS; neutrophils, change in LA; BG; BUN; RR; albumin; SBP; serum
creatinine; temperature.
TTP: after EDA after laboratories are resulted
Models' ROS was more sensitive and precise compared with qSOFA score at 1 hour and
at 24 hours.
AUC (95% CI): 0.93–0.97
[29 ]
Decision tree, discriminant analysis, logistic regression, KNN, neighbors, ensemble
classification, SVM, and neural network
Identification of sepsis
Age, dialysis, mobility, mentation, SBP, HR, RR, LA, temperature, and WBC.
TTP: first 6 hours EDA
Although the fine Gaussian SVM showed highest sensitivity (95.8%), NN model had the
highest accuracy (92.08%), high sensitivity (92.33), and high specificity (92.33%)
AUC: 0.92
Abbreviations: ALP, alkaline phosphatase; ALT, alanine aminotransferase; AST, aspartate
aminotransferase; AUC, area under receiving operating curve [with 95% confidence interval];
AVPU scale, alert, verbal, pain and unresponsive scale; BG, blood sugar; BP, blood
pressure; BUN, blood urea nitrogen; CHF, congestive heart failure; CoV, coefficient
of variation; CRF, chronic renal failure; CVA, cerebral vascular accident; DBP, diastolic
blood pressure; DM, diabetes mellitus; Dx, diagnosis; Dx, diagnosis; ECG, electrocardiogram;
EDA, ED arrival time; GCS, Glasgow coma scale; HgB, hemoglobin; HR, heart rate; IAI,
intra—abdominal infection; ICU, intensive care unit; INR, International Normalized
Ratio; KNN, k-nearest neighbors; LA, lactic acid; MAP, mean arterial pressure; MCH,
mean corpuscular hemoglobin; MCHC, mean corpuscular hemoglobin concentration; MCV,
mean corpuscular volume; NN50, RR number of consecutive RR triangular index (TINN);
pH, power of hydrogen (pH); PT, prothrombin time; qSOFA, quick sequential organ failure
assessment; RDW, red cell distribution width; RI, respiratory infection; RN, registered
Nurse; ROS, development of risk for sepsis; RR, respiratory rate; RT, respiratory
therapist; SBP, systolic blood pressure; SD, standard deviation; SpO2, oxygen saturation;
STI, soft tissue infection; SVM, support vector machine; TINN, baseline width of a
triangle fit into the RR interval histogram using a least squares; TTP, time to predict
(The earliest time that the prediction model can be run based on the predictors);
UTI, urinary tract infection; WBC, white blood count.
Although ML models have been used to detect early infection, sepsis, and septic shock,
research related to early identification of sepsis or septic shock in patients, specifically
on arrival to ED is still limited.[27 ]
[28 ]
[29 ]
[30 ]
[31 ]
[33 ] Common variables identified from the 12 articles in the review for detection of
infection, sepsis, or septic shock in all settings, such as ED, intensive care units,
and floors, include SBP, diastolic blood pressure (DBP), heart rate, RR, peripheral
capillary oxygen saturation, temperature, and chief complaints,[27 ]
[28 ]
[29 ]
[31 ] many of which often are not available in electronic health record (EHR) on patient
arrival to ED due to lack of interoperability. For instance, Mao et al developed a
gradient tree boosting model using data from only six vital signs: SBP, DBP, heart
rate, RR, peripheral capillary oxygen saturation, and temperature.[29 ] This model was able to predict sepsis at the onset with high AUC (0.92) and septic
shock 4 hours in advance with high AUC (0.96).[29 ] The model was also able to predict severe sepsis 4 hours in advance with higher
AUC (0.85) than the onset time for statistically calculated SIRS AUC (0.75).[29 ] This study used data from ED, critical care and regular floors and required 3 hours
of measurements of all six vital signs. Model performance might be different if limited
to data collected during the early ED visit which may not have continuous vital signs
recorded for three hours. Prehospital data can provide an opportunity to use a predictive
model on patient arrival that includes these additional data elements.
Only four studies used ML models to predict early onset of infection, sepsis, and
septic shock in patients in the ED.[28 ]
[30 ]
[31 ]
[33 ] In these studies, time needed for prediction of infection or sepsis ranged from
1 hour from the first vital signs on admission to ED to 24 hours after admission to
ED.[28 ]
[30 ]
[31 ]
[33 ] Horng et al found that diagnosis of infection in ED could be predicted more accurately
by using free-text chief complaints and nursing assessments along with structured
data, such as vital signs and demographics.[28 ] This study used a variation of NLP in ED to uncover latent data from chief complaints
and nursing assessment notes for early detection of infection. Mohamed et al used
neurological assessment in the form of mentation along with other common variables
to identify sepsis through multiple models where a neural network model performed
the best with greater than 90% accuracy.[30 ] Due to limitation in available data, different tools, such as NLP, and uncommonly
used data elements, such as neurological assessment in the form of mentation, may
be needed for early prediction of sepsis.
From the 12 studies, 10 defined their diverse methods of handling missing values.
Two studies included forward filling imputation[26 ]
[29 ]; two studies imputed by means, modes or median[31 ]
[34 ]; two studies converted missing values to categorical or nominal values[30 ]
[35 ]; one study used multivariate imputation by chained equations method[36 ]; one study used linear interpolation method[37 ]; one study excluded patients with missing data[38 ]; and one study automatically imputed physiologically normal values in the missing
vital signs data.[28 ] The diversity in handling of the missing values especially with vital signs raises
concerns related to functionality and ethics of ML model in clinical settings.
Discussion
The purpose of this integrative literature review was to discuss the importance of
utilizing prehospital data elements in ED, summarize their current use in developing
ML-based prediction models, and specifically identify those data elements that can
potentially contribute to early identification of sepsis in ED when used in ML-based
approaches. Prehospital data have been instrumental in the development and validation
of statistical and ML models with the purpose of improving early detection of time-sensitive
patient conditions, such as trauma, stroke, and shockable cardiac rhythms post–out-of-hospital
cardiac arrest; and patient outcomes such as inpatient mortality, survival rate, and
ROSC.[16 ]
[21 ]
[22 ] Currently, limited studies are available related to detection and prediction of
sepsis in ED, and there was no study found related to detection of sepsis in ED using
prehospital data. This gap in knowledge may be due to limitations in availability
of prehospital data in the EHR and difficulty in diagnosis of sepsis in ED. There
is an opportunity for researchers to develop and evaluate prediction models for early
detection of sepsis using prehospital data in the ED in the future.
ML has been found to be superior in performance in terms of accuracy, specificity,
and sensitivity compared with statistically calculated screening tools for sepsis
prediction.[16 ]
[20 ]
[21 ]
[24 ]
[32 ]
[38 ] Based on current and previous definitions of sepsis, the early detection of sepsis
has been linked with the following screening tools: SIRS, the modified Early Warning
Scores (MEWS), and SOFA and qSOFA scores.[6 ]
[39 ] However, SIRS, MEWS, SOFA, and qSOFA scores when compared with ML models have performed
poorly in identification of sepsis and in predicting inpatient mortality from sepsis
in ED.[27 ]
[33 ]
[35 ]
[38 ] In a meta-analysis study by Islam et al, ML models showed higher accuracy for sepsis
detection as evidenced by the AUC of 0.89 compared with sepsis screening tools such
as SIRS, MEWS, and SOFA score with AUC of 0.70, 0.50, and 0.78, respectively.[39 ] Contrarily, two sets of decision tree models using the same variables as qSOFA and
SIRS generated similar and low AUC and sensitivity.[36 ] Therefore, although the ML models in general outperform statistically calculated
sepsis screening tools, the performance of the prediction models depend highly on
the selection of the set of variables or data elements used to create the model.
As described in [Table 1 ], use of prehospital data could help expedite certain predictions in ED if the required
data elements were collected earlier during prehospital care. For instance, a promising
use of ML in ED using prehospital data could reduce overcrowding by managing the availability
of personnel.[14 ] However, use of prehospital data are limited by the quality and availability of
the data. Mashoufi et al conducted a survey among three groups of EMS stakeholders:
data producers, data collectors, and data consumers.[40 ] They concluded that the quality of EMS data with respect to their usability, completeness,
and compatibility are still low.[40 ] An additional caveat for developing an accurate prediction/early detection model
in ED is balancing TTP, and the helpfulness of the inputted variables. TTP will be
determined by the time that all data elements become available for analysis. For instance,
Duceau et al used 32 prehospital and postarrival variables that included electrocardiogram
(ECG), computed tomography (CT) scan, and transthoracic echocardiogram to predict
acute aortic syndromes (AAS) and assess the performance of undertriage and overtriage.[24 ] Use of data elements that take a longer time to result such as diagnostic tests
provides accuracy in performance but may delay the time to predict.[41 ] Contrarily, only using insufficient prehospital data, one model may be able to deliver
prediction at arrival time but with potential cost of reduced performance. Therefore,
balancing performance and time to predict is necessary in clinical setting with consideration
to quality and availability of data.
This literature review identifies data elements that may be used for early prediction
of sepsis in ED using ML models. ML models using prehospital data elements have been
applied in ED, although not for sepsis, and have reported high performance in improving
ED operations and predicting ED outcomes. Data elements found helpful in early prediction
or detection of sepsis in ED are also highlighted, although none were from prehospital
data. Therefore, the prehospital data elements, identified in [Table 2 ], and the data elements specific to sepsis, identified in [Table 2 ], may be combined to improve early identification of sepsis in ED. This integrative
literature review emphasizes the current gap in use of prehospital data and its potential
in supporting early identification of sepsis in ED using ML and provides data elements
that may facilitate further predictive analysis research.
Limitations
Some limitations should be noted. One limitation is related to the literature search
itself and its scope. The search for this literature review focused on the use of
prehospital data for ED decision-making and the ML models for sepsis in ED. Any other
studies, including interventional studies, not related to the two search strategies
and our search terms were not included. The limited search terms used in this review
excluded some of the studies that have been included in other reviews such as meta-analysis
by Fleuren et al and a systematic review by Kareemi et al.[32 ]
[42 ] The purpose of our review also differed from these systematic reviews as the results
focused on studies that used prehospital data for ED decision-making.
Another limitation is the use of retrospective research designs for most of the studies
included in this review and lack of implementation of ML models in a clinical setting.
Only one prospective study related to sepsis by McCoy and Das found reduction in in-hospital
mortality rate by 60.24% postimplementation of sepsis prediction score alert system
on floors, ICU, and ED.[26 ] That study also observed improvement in patient outcomes: a decrease in sepsis-related
hospital length of stay by 9.55%, a decrease in 30-day readmission rate by 50.14%,
and an increase in 3-hour sepsis bundle compliance by 72.7%.[26 ] However, the impact on patients specifically in ED and accuracy of the sepsis prediction
score in ED was not analyzed in the study. Therefore, more research with the target
outcome of early prediction or detection of sepsis focusing on ED patients using prehospital
data are necessary to impact real-time clinical decision-making in ED and to improve
patient outcomes.
There are also limitations in the use of prehospital data in practice, such as ML
models using prehospital data that only benefit patients who arrive by ambulance.
The lack of generalizability to all patients who arrive to ED limit the scope of the
use of prehospital data. Limited access to data elements further limits the use of
prehospital data. Despite the progress in recent years,[43 ]
[44 ] there is still a lack of interoperability that may limit the number and types of
the prehospital data available for ML models.[45 ] Moreover, assuming the health care system has interoperability with EMS data with
real-time access to the recorded data, availability of the included data elements
at the time of prediction is crucial in designing clinical decision-support systems.
Martin et al identified interoperability, accurate match algorithms, security, and
wireless connectivity as potential barriers to adoption of prehospital health information
exchanged.[13 ] Further development in ED–EMS interface is needed to enable prominent use of prehospital
data in ED decision-making and promote research in use of prehospital data for improved
patient outcomes. Additionally, incomplete or missing data are often a problem with
prehospital data. Using various imputing techniques would likely be fraught with measurement
error which would lead to inaccurate models. More research is needed in this area.
Conclusion
Sepsis has a significant impact on patient outcomes and HRQOL and financially impacts
patients, their families, hospitals, community, and national health care costs. Sepsis
is a time-sensitive condition that requires early detection and EGDT to improve patient
mortality and related patient outcomes. Prehospital data in partnership with NLP and
ML models can potentially improve clinical decision-making for sepsis in ED. Accessibility
and limitations of the prehospital data, appropriateness of the variable selection,
and the time it takes to generate a prediction have significant impacts on the performance
of the ML models to improve patient outcomes for time-sensitive conditions, such as
sepsis in ED.
Although ML models outperform sepsis severity scores and have the potential to predict
at the onset or prior to sepsis and septic shock, the performance of the model depends
heavily on selection of the variables or data elements. Future implications suggest
development, validation, and application of the ML combined with NLP models using
prehospital data in clinical settings to identify sepsis earlier and to support related
clinical decision making in ED to improve patient outcomes.
