Keywords
EHRs and systems - machine learning - workflows and human interactions - simulation
and modeling - clinical decision support system
Background and Significance
Background and Significance
Unplanned hospital readmissions refer to any admission postdischarge that are not
in the initial plan of care or treatment. Unplanned readmissions are associated with
adverse effects such as increased risk of mortality during the patient's hospital
stay or shortly after, negatively impacting the quality of care.[1] Thirty-day hospital readmission has long been established as an important and measurable
indicator of the quality of care.[2] Unplanned readmissions contribute significantly annually to the cost of the U.S.
health care system, costing over $52 billion for all unplanned readmissions and $3.8
million for 30-day readmissions.[3] Financial hospital reimbursement policies have made it critical to identify and
act on risk factors for unplanned readmissions.[4] In the fiscal year 2013, the Centers for Medicare and Medicaid Services initiated
the Hospital Readmission Reduction Program to encourage hospitals to reduce avoidable
readmissions by adjusting the payments based on their performance in reducing readmissions.
Hospital readmission can have a wide range of rates. A systematic review showed significantly
high 30-day readmission rates ranging from 6.9 to 23.89% for neurosurgery patients.[5] While there have been quite a few studies identifying risk factors and trends of
unplanned readmissions,[6]
[7]
[8]
[9]
[10] there are very few interventional studies in neurosurgery actively attempting to
reduce 30-day readmission rates.[11]
[12] While these interventions have displayed a decrease in the general rate of 30-day
unplanned readmissions, the interventions were applied to all participants in the
intervention group. The patients were not stratified based on their risk of readmissions,
which incurs substantial avoidable expenses and time inefficiencies. It is therefore
crucial to identify patients specifically at risk of unplanned readmissions for intervention
to better improve patient outcomes and reduce cost.
After identifying potential risk factors, advanced methods of prediction such as machine
learning and artificial intelligence (ML/AI) can be helpful to predict hospital readmissions.
Literature reports existing development and use of ML-based models in predicting neurosurgical
outcomes.[13]
[14] There are also developed models to predict 30-day unplanned readmissions in neurosurgery.
One study developed an ML model for stroke patients with an area under the receiver
operating characteristic curve (AUROC) of 0.74[15] and another study developed a model for patients after pituitary adenoma resection
with an AUROC of 0.76.[16] Although these models achieved relatively good performance, these studies have not
considered factors related to the clinical implementation through the clinical workflow
and decision point analysis. There is research suggesting the potential benefit of
clinical implementation of advanced algorithms in reducing unplanned hospital readmissions.[17]
[18] However, no such studies have been done in neurosurgery specifically to the best
of our knowledge. Moreover, the potential clinical and health impact of the models
were not demonstrated in a laboratory or field experiment. All of the above limit
the usefulness of the predictive models.
In this study, we reported the development and simulated evaluation of a novel Neurosurgical
Readmission Reduction Program (NSRRP) at an adult academic health center in the Midwestern
United States. The development of NSRRP utilized not only ML-based approaches and
electronic health records (EHRs), but also clinical workflow analysis to understand
the context of the decisions. The simulated evaluation utilizes simulation to demonstrate
potential clinical and health impact to get leadership support before the actual implementation.
Specifically, this study aimed to: (1) train individual predictive models on three
patient groups in neurosurgical intensive care unit (NSICU), spine surgery, and traumatic
brain injury (TBI) with good performance (AUROC > 0.8), (2) identify decision points
and potential interventions through semi-structured interviews, and (3) use agent-based
modeling (ABM) to demonstrate potential clinical and health impact.
Methods
Study Setting
This study involved a collaborative effort within the University of Cincinnati College
of Medicine (UCCoM) between the Department of Neurosurgery (Neuroinformatics Laboratory),
the Department of Biomedical Informatics (Center for Health Informatics), and the
University of Cincinnati Medical Center (UCMC). The Department of Neurosurgery provided
seed funds to establish the Neuroinformatics Laboratory and partnered with the Center
for Health Informatics to develop the data infrastructure. This study was one of the
pilot studies to demonstrate the efficacy of the data infrastructure and the synergy
that the Neuroinformatics Laboratory could offer. The study was also a part of the
2023 AI Evaluation Showcase, an initiative organized by the American Medical Informatics
Association (AMIA). The study followed the three-stage design of the AMIA AI Evaluation
Showcase, namely, technical performance, workflow study, and clinical impact. We formed
an interdisciplinary team including domain experts (clinicians), a data analyst, a
designer, and a statistician to conduct the study and discuss the findings. The protocols
for retrospective chart review and clinical workflow analysis were approved by our
institutional review board (IRB): #2019-1403 for the retrospective chart review and
#2022-0635 for the clinical workflow analysis. It is worth noting that in the workflow
study, the identity of the interview participants was removed, and no patient data
were collected.
Patient Population
The data were extracted from the institution's EHRs through the Center for Health
Informatics, an honest data broker between UCCoM and UCMC. Three patient cohorts,
including those from NSICU, spine surgery, and TBI, were identified for the study.
NSICU patients were selected from those admitted to the NSICU Department. The spine
surgery patients comprised patients who underwent cervical or lumbar surgery based
on the International Classification of Diseases procedure codes. Patients in the TBI
cohort were identified using neurotrauma flowsheet, diagnosis codes, Diagnostic Related
Group codes, and Current Procedural Terminology codes. Detailed inclusion criteria
were listed in [Supplementary Material S1] (available in the online version only). The readmissions were limited to inpatient-related
admission classes, and admissions marked as deceased in discharge disposition was
removed. A 30-day unplanned readmission was defined as an unexpected hospital encounter
occurring within 30 days after discharge, which follows a redefined rule in our hospital's
EHR system after consulted with our data analytics team.
Machine Learning Models
A three-stage analytic development process was applied to predict 30-day readmissions
in neurosurgical patients with disorders of the spine, TBI, and NSICU. Risk factors
were first identified from literature research and then expanded as well as categorized
by the domain experts in the research team. A subset of risk factors (variables of
interest) in each of the three patient cohorts were further determined by the domain
experts. Five classic ML classification algorithms including gradient boosting, decision
tree, random forest, ridge logistic regression, and linear support vector machine.
The selected ML models for comparison offer a diverse range of strengths. Gradient
boosting handles complex data and improves weak learners for accurate predictions.
Decision tree provides interpretability and capture nonlinear relationships, whereas
random forest provides robustness and feature importance estimates. Ridge logistic
regression offers simplicity and efficiency as a baseline model, and linear support
vector machine was included in finding optimal hyperplanes for robust generalization,
particularly in high-dimensional spaces. The dataset was randomly divided into 80%
for training and validation, with the remaining 20% reserved for testing. In the training
process, a 5-fold cross-validation with hyperparameter tuning was employed. The AUROC
was used to determine the model performance. On testing dataset, the model performance
was reported using sensitivity (recall), specificity (true negative rate), precision
(positive predictive value), and accuracy. To identify all patients who may face unplanned
readmission, we sought a threshold based on achieving a sensitivity of one. While
specificity may be lower and false positive rate can be higher, our approach is still
preferable to a blanket intervention for all identified patients.
Clinical Workflow Analysis
This study aimed to recruit a minimum of 10 participants from three teams: NSICU,
spine surgery, and TBI. The purpose of these interviews was to explore potential context
and use cases of the readmission predictive model in reducing readmission rates as
well as the decision points in the workflow. The interview questionnaires can be found
in [Supplementary Material S2] ([Supplementary Figs. S1] and [S2], available in the online version). The participants were asked to describe their
work processes, identify decision points, and provide potential use cases. Since the
interviews were semi structured, follow-up questions were asked based on the participants'
responses. The participants were recruited through convenience sampling, using the
research team's network, and snowball sampling, by asking participants to provide
names at the end of their interview. In addition to physicians, nurse practitioners
and navigators were also primary targets for interviews. At the beginning of each
interview, the participants were introduced to the project and verbally consented.
After this, each participant received information about the predictive model variables
and performance details. All interviews were conducted online; each lasted 30 to 45 minutes
and was recorded and transcribed automatically using Microsoft Teams. The transcripts
were reviewed and further analyzed to generate workflow diagrams and to summarize
possible interventions using the predictive models developed for each patient group.
Simulation-Based Evaluation
The clinical and health impact of the predictive models were estimated in terms of
the reduction of the 30-day readmission rate and financial costs using ABM, which
is a computational technique simulating the behaviors and interactions of individual
agents, or patients in this context, within a system to understand its overall dynamics.
In the health care setting, ABM has emerged as a powerful tool to assess clinical
impact. We assumed the readmission probability is equally likely within each agent,
with a mean calculated from the readmission rate and a standard deviation of 0.0005,
ensuring positive probabilities within 30 days after discharge. The study focused
on a series of interventions aimed at reducing readmission probabilities for each
of the three patient groups with 100 simulations on 30-day readmission. A prior study
estimated that completing a series of interventions would result in a reduction of
the readmission rate from 5.8 to 2.5% for 416 patients, equivalent to reducing the
readmission probability by 0.00149 for patients who received complete intervention
within 24 hours after discharge.[12] These interventions, including a preadmission overview, recorded anticipated discharge
date, daily communication with the transitional care team through the hospital stay,
advance notice before a day of discharge, prescription filling in the hospital, a
30-minute exit consultation and education with a skilled nurse at discharge, and a
follow-up telephone call within 2 days after discharge, were designed to address various
factors contributing to readmissions.
The simulation models were calibrated in two steps. First, a baseline model was developed
to match the readmission rates before and after the intervention in the previous study.
Specifically, the Wilcoxon signed-rank test was used to determine if the median simulated
readmission rate aligns with the actual readmission rates (5.8–2.5%).[12] Second, the baseline model was extended to simulate the readmission rates of our
three patient groups with adjusted parameters. The same test was used to determine
if the simulated readmission rates match the actual readmission rates without the
intervention. Once the calibration was done, each simulation models ran 100 times
to generate the simulated readmission rates after the intervention. The same test
was applied to determine significant median differences between the simulated readmission
rates before and after the intervention in each patient group.
The financial impact was quantified by calculating the reduction in readmission-related
costs used in the previous study. These costs encompass the expenses associated with
hospital stays minus the cost of training individuals to implement the interventions.
For NSICU, spine, and TBI, the mean costs of hospital stay expense and care transitions
coaches for the intervention were estimated to be $38,601.90,[19] $27,926.47,[20] and $20,030.54,[8] respectively. The intervention cost (transition of care) was estimated to be $264.46
in a TBI setting[11] and applied to other two settings since there was no available intervention cost
in the literature. The costs were adjusted for inflation from January 2019 to January
2020 to June 2023 in U.S. dollars using the U.S. Bureau of Labor Statistics Consumer
Price Index Calculator.[21] The cumulative reduction in readmission rates can directly translate to a decrease
in financial burden for the health care system. The estimated cost saving can increase
through the predictive models by removing patients who are unlikely readmitted.
Results
The data were extracted for the time span between July 2017 and November 2022. It
comprised a total of 12,334 admissions for 11,029 distinct patients in NSICU, 1,903
admissions for 1,641 patients undergoing spine surgery, and 2,208 admissions for 2,185
patients with TBI. The readmission rates for NSICU, spine surgery, and TBI were 13.13,
13.93, and 23.73%, respectively.
Model Performance
In total, 28 unique risk factors were selected from the literature by the domain experts
and put into six categories. Our developed model utilized a subset of risk factors
as follows: (1) patient demographics (N = 2); (2) medical history (N = 12); (3) diagnosis (N = 5); (4) admission Information (N = 3); (5) hospitalization outcome (N = 3); and (6) discharge disposition (N = 3). The detailed variables are available in [Supplementary Table S1] (available in the online version). Among the 28 unique risk factors, 16 of them
were selected by the domain experts and fed into the ML development process. As shown
in [Fig. 1A], the random forest model achieved the highest performance with an AUROC score of
0.89. This model demonstrated sensitivity of 1.0, specificity of 0.4, precision of
0.2, and accuracy of 0.48. Top features including discharge disposition, length of
stay, number of emergent admissions, age, and chronic obstructive pulmonary disease.
Next, or spine 30-day readmission prediction, the literature review identified a total
of 32 unique risk factors: (1) sociodemographic factors (N = 6); (2) medical history (N = 14); (3) surgical information (N = 7); (4) hospitalization outcomes (N = 4); (5) discharge disposition (N = 1). The detailed variables are located within [Supplementary Table S2] (available in the online version). Among the 32 risk factors, 17 of them were selected
and fed into the ML development process. The results revealed that the random forest
model had the highest performance with an AUROC of 0.84 in [Fig. 1B]. The model also achieved a sensitivity of 0.91, specificity of 0.62, precision of
0.28, and accuracy of 0.66. Additionally, we observed that with the same AUROC score,
the model demonstrated a sensitivity of 1.00, specificity of 0.28, precision of 0.18,
and accuracy of 0.38. The top five features of the predictive model are the discharge
disposition, has ICU stay, length of stay, body mass index, and age. Lastly, a total
of 24 unique predictors for TBI readmission were identified: (1) sociodemographic
factors (N = 7); (2) nature of TBI (N = 8); (3) non-TBI medical cause (N = 5); (4) discharge disposition (N = 4). The detailed variables are attached in [Supplementary Table S3] (available in the online version). Among the 24 risk factors, 13 of them were selected.
As presented in [Fig. 1C], the logistic regression model achieved the best performance at a sensitivity of
1.00 with specificity of 0.09, precision of 0.26, and accuracy of 0.31. The top features
in the predicting model included discharge disposition, hospital length of stay, age,
initial Glasgow Coma Scale score, and brain image results.
Fig. 1 The 30-day readmission prediction performance for NSICU (A), spine surgery (B), and TBI (C). AUC, area under the curve; NSICU, neurosurgical intensive care unit; ROC, receiver
operating characteristic; TBI, traumatic brain injury.
Workflow Diagrams and Decision Points
The workflow analysis involved interviews with a total of 12 participants from different
teams: three physicians from the spine surgery team; one physician, one nursing navigator,
and one medical assistant from the TBI team; and two physicians, two nurse practitioners,
one registered nurse, and one social worker from the NSICU team. The participants'
identity was removed, and no patient data were collected in this process. The feedback
collectively generated six areas for intervention that would benefit from utilization
of the predictive models as shown: (1) best practice alert (preoperative), (2) EHR
patient main page (inpatient stay), (3) discharge plan (inpatient stay), (4) checklist/reminder
(discharge, transfer, downgrade), (5) patient education (discharge), and (6) follow-up
plan. NSICU has more areas for intervention because acute care patients can either
be discharged directly or downgraded to a lower level of care. [Fig. 2] shows the general workflows of three teams with the potential interventions at the
specific point of care (represented by red dots) to reduce readmissions. [Table 1] presents the comprehensive list of potential interventions along with their corresponding
use cases at various stages, all of which were derived from the interview process.
Fig. 2 Workflows of NSICU (A), spine (B), and TBI (C) teams with six types of potential interventions: (1) best practice alert, (2) patient
main page, (3) discharge plan, (4) checklist reminder, (5) patient education, and
(6) follow-up. ICU, intensive care unit; NSICU, neurosurgical intensive care unit.
TBI, traumatic brain injury.
Table 1
The list of potential interventions and use cases from the interview
|
Using cases
|
Time point
|
Primary users
|
Scenarios
|
|
Best practice alert
|
Preoperative
|
Physicians nursing practitioner
|
A best practice alert in the form of a pop-up window appears before closing a patient's
chart, highlighting the patient's higher risk of readmissions due to specific reasons.
This prompt serves as a reminder for health care providers to consider the information
while making decisions for the patient
|
|
Patient main page
|
Inpatient stay
|
Nursing navigator
|
Nurse navigators routinely access patient accounts throughout the day, including responding
to patient inquiries, scheduling postoperative appointments, and monitoring patient
care across all stages in EPIC. The display of high readmission rates on the patient's
main page would assist nurse navigators in proactively reviewing patient conditions
and intervening as necessary
|
|
Discharge plan
|
Inpatient stay intensive care unit stay
(when making discharge plan)
|
All team members
|
Displaying the readmission rate on patients' dashboards during disposition or discharge
planning would be a valuable tool for health care providers. By considering the readmission
rates and other relevant variables, health care providers can more effectively determine
the appropriate disposition location and schedule the next appointment or extend the
days in the hospital for more observation or exams. This approach can optimize patient
care and reduce readmission
|
|
Checklist reminder
|
Discharging transferring (downgrading)
|
Nursing practitioner
|
A standardized checklist could ensure all the important variables are addressed before
discharging a patient to minimize the risk of readmission. This would help providers
feel confident and provide better patient outcomes. Furthermore, the checklist would
streamline the discharge process, making it more efficient, and improve patient satisfaction
|
|
Patient education
|
Discharging
|
Nursing navigator
|
Incorporating families in patient education by providing them with information on
potential readmission factors can enhance postdischarge care and possibly mitigate
the risk of readmission
|
|
Follow-up
|
Follow-up
|
Nursing navigator
|
With this model, the nurse navigator can proactively follow up on patients with a
high risk of readmission and implement necessary interventions. She reviews the list
of all patients every day, ensuring they have appropriate follow-up care. A flag or
alert indicating a patient's readmission risk would be helpful, allowing for timely
interventions such as a follow-up call
|
Simulation
The first step of the model calibration between the simulated and the actual readmission
rates before and after the intervention in the baseline model showed p-values of 0.35 and 0.63, indicating indifference (agreement) between the simulated
and the empirical scenarios. The second step of the calibration between the median
simulated readmission rates derived from three discrete ABMs representing the NSICU,
spine, and TBI showed p-values of 0.93, 0.83, and 0.64, respectively. This indicates that no substantial
disparities exist between the observed and simulated readmission rates. These results
warranted the efficacy of the models, calibrated with specific parameters and configurations,
in accurately replicating the observed readmission rates. By applying the calibrated
ABMs to the individual patient populations, the simulated median readmission rates
exhibit reductions ranging from 13.13 to 10.12% for NSICU, 13.90 to 10.98% for the
spine department, and 23.64 to 21.20% for the TBI department. These findings are demonstrated
through [Fig. 3], where the box plots graphically depict the outcomes of the simulations. Moreover,
the one-sided Wilcoxon signed rank test further strengthens the argument, providing
compelling evidence that the median simulated readmission rates following the interventions
are indeed significantly lower than those in the absence of intervention. This conclusion
is substantiated by the p-values less than 0.0001 for each respective case.
Fig. 3 Boxplots of readmission rate with/without intervention simulated on NSICU (A), spine (B), and TBI (C). NSICU, neurosurgical intensive care unit; TBI, traumatic brain injury.
The cost-saving were summarized in [Table 2]. The savings with intervention were calculated by the readmission cost per discrepant
readmitted patient. On the other hand, the intervention spending was calculated by
totaling the cost of the transition coach per readmission for intervention. The ML
predictive model removed any unlikely readmissions, leading to the adjusted intervention
spending, which was the monthly cost of the transition coach per readmission when
the sensitivity equals 1.0 in the predictive model. This adjustment narrowed the focus
to patients more inclined to face readmission. Finally, the overall savings were calculated
in the unadjusted versus adjusted scenario. By integrating the suggested intervention
into the operational workflow, a significant amount of roughly $12,548,618.71 was
effectively preserved across all three patient groups. Additionally, the use of ML
predictions can result in an incremental reduction of approximately $1,300,614.28.
Table 2
Cost analysis estimated from the result of agent-based model on neurosurgical intensive
care unit, spine, and traumatic brain injury
|
Title
|
NSICU
|
Spine
|
TBI
|
|
Without ML model
|
|
No. of readmissions intervened
|
12,344
|
1,903
|
2,208
|
|
No. of readmissions saved
|
1,250
|
209
|
468
|
|
Cost per readmission
|
$38,601.90
|
$27,926.47
|
$20,030.54
|
|
Saving with Intervention (S)
|
$14,282,703.00
|
$1,535,955.85
|
$1,081,649.16
|
|
Intervention spending (IS)
|
$3,264,494.24
|
$503,267.38
|
$583,927.68
|
|
Total saving (S)—(IS)
|
$11,018,208.76
|
$1,032,688.47
|
$497,721.48
|
|
With ML model
|
|
Readmission rate without intervention
|
13.13%
|
13.90%
|
23.64%
|
|
Readmission rate with intervention
|
10.12%
|
10.98%
|
21.20%
|
|
No. of of readmission intervened
|
7,906
|
1,583
|
2,048
|
|
No. of of readmission saved
|
1,250
|
209
|
468
|
|
False positive rate
|
0.600
|
0.811
|
0.908
|
|
Adjusted intervention spending (AIS)
|
$2,090,820.76
|
$418,640.18
|
$541,614.08
|
|
Adjusted total saving (S)—(AIS)
|
$12,191,882.24
|
$1,117,315.67
|
$540,035.08
|
Abbreviation: ML, machine learning; NSICU, neurosurgical intensive care unit; TBI,
traumatic brain injury.
Notes: The top table shows the cost-saving as a result of a hypothetical intervention
without using the best-performing ML model; the bottom table shows the cost-saving
with the ML model. The differences between IS and AIS come from the use of the ML
model.
Discussion
Key Findings
This study highlights the successful application of an ML-based approach in predicting
30-day hospital readmissions among patients with neurosurgical disorders, resulting
in satisfactory predictive performance. The study extended the predictive modeling
and introduced an analysis of clinical workflows, with the overarching goal of creating
decision support tools tailored to mitigate neurosurgical readmissions. Participants
of the study reveals a unanimous and robust endorsement of the NSRRP. Participants
believe that such an initiative holds the potential to empower health care providers
to concentrate their efforts on patients at elevated risk, thereby facilitating timely
interventions and ultimately enhancing patient outcomes. Moreover, the study incorporates
ABMs capable of reproducing readmission scenarios, enabling the evaluation of intervention
effectiveness and the estimation of potential cost reductions. These findings collectively
underscore the feasibility and potential impact of deploying advanced analytics in
the neurosurgical domain to address hospital readmissions and emphasize the significance
of such efforts in the realm of health care improvement.
Implications
The findings of this study showcase the considerable prospects associated with the
integration of ML methodologies to predict instances of hospital readmissions among
patients undergoing neurosurgical procedures. As reported in a few other studies,[17]
[18] the deployment of advanced predictive modeling techniques as decision support tools
hold promises in addressing the complex challenge of readmissions, thereby augmenting
the quality of patient care. The affirmative feedback received from study participants
resonates with the notion that the incorporation of such predictive tools within clinical
settings holds the potential for favorable reception among medical practitioners.
This receptiveness, in turn, could translate into tangible enhancements in patient
outcomes. Furthermore, the efficacy of the ABMs in accurately forecasting the reduction
in readmission rates subsequent to the implementation of interventions bears implications
for cost-effectiveness. By offering a means to quantitatively estimate the cost savings
resulting from intervention strategies, this model contributes to the strategic allocation
of resources and health care planning.
Limitations
This study has few limitations. First, the study only used EHR data but no other data
sources in the readmission prediction. This might lead to the omission of potential
variables that exert influence but are not documented clearly, such as social determinants
(or drivers) of health, which have been documented to be an important contributor
to hospital readmission.[22] These unrecorded factors could impact the model's accuracy by not being incorporated
into the predictive framework. Second, the predictive models were not retrained based
on the decision points in the workflow. Some information may not be available at a
specific point of care and can affect the overall model performance. Third, while
convenience sampling would allow quick access to potential participants, it may lead
to bias. We mitigated this bias by combining convenience sampling with snow sampling.
Given the small size of the clinical teams in this study setting, we believe that
this method can collect enough information to summarize workflow issues and bottlenecks.
Last but not least, the current intervention simulations entail uniform probability
reduction and intervention cost across all three patient groups. This approach could
introduce bias to the results, as each department possesses unique dynamics. Deviations
in intervention effectiveness and cost implications might lead to misinterpretations
of the model's impact on readmission reduction.
Future Work
Several avenues for future research warrant exploration. First, there is substantial
potential for enhancing model performance by integrating additional variables of interest
into the existing predictive framework. Incorporating these variables, such as genetic
markers or novel biomarkers, could lead to a more accurate and comprehensive prediction
model. Moreover, delving into the realm of model explainability is imperative for
building trust and understanding in predictive health care applications. The utilization
of tools like SHapley Additive exPlanations (SHAP) values can aid in identifying key
features and their corresponding weights, shedding light on the decision-making process
of the model. Transitioning from admission-based predictions to patient-based predictions
presents an intriguing avenue for research. The adoption of deep learning architectures,
such as Long Short-Term Memory, could offer improved sequential data analysis capabilities,
particularly when dealing with patient history and longitudinal data. Integrating
clinical notes extracted through natural language processing as model parameters presents
another exciting direction. This integration has the potential to enhance prediction
accuracy by harnessing unstructured data within EHRs. Expanding the predictive pipeline
to encompass patients with neurotrauma and those in neurocritical care settings would
extend the model's clinical applicability. Incorporating novel factors identified
through comprehensive literature searches can enrich the predictive capacity of the
model. Additionally, tailoring readmission probability adjustments based on demographic
variables acknowledges the heterogeneity of patient populations and contributes to
more personalized predictions. Lastly, the incorporation of environmental variables
within ABM could provide a holistic perspective on patient outcomes, considering the
broader context in which health care decisions are made. In conclusion, these proposed
avenues for future research collectively represent a substantial step forward in predictive
modeling for medical prognosis and offer promising contributions to both clinical
practice and research.
Conclusion
In this study, we successfully developed ML-based models for the prediction of 30-day
hospital readmissions for neurosurgery patients involved in NSICU, TBIs, and spine
surgeries with good predictive capabilities. For the desired goal of creating a clinical
support decision tool, we conducted clinical workflow analysis to better tailor the
model in specific decision points. Through the use of ABMs, we were able to successfully
reproduce readmission situations to evaluate the model's clinical effectiveness as
well as financial cost reductions, supporting the clinical efficacy of this intervention.
We will continue developing the NSRRP and pilot the decision support tools in clinical
routines to demonstrate clinical and health impact through field experiments.
Clinical Relevance Statement
Clinical Relevance Statement
Unplanned hospital readmissions in neurosurgery contributes significantly to suboptimal
patient health outcomes as well as increased financial burdens on both the patient
and hospital. While previous studies have focused on the sole development of models
to predict 30-day readmissions, an important indicator of quality of care, we focus
on not only the model development, but also the clinical workflow and potential clinical
and financial effect. With multiple steps taken, we believe that this intervention
would translate very well in mitigating and reducing neurosurgical hospital readmissions.
Multiple-Choice Questions
Multiple-Choice Questions
-
Which of the following was a finding of semi-structured interviews with providers
regarding the use of a machine learning interventional program for reducing hospital
readmissions?
-
This program enables efforts to be concentrated on patients with elevated risks
-
This program would help decrease provider burnout and work hours
-
Participants in this study did not endorse support for this type of intervention
-
Clinical workflow analysis did not identify any potential points of intervention
Correct Answer: The correct answer is option a. According to the results and discussion of this paper,
this is the only answer to correctly identify a finding of clinical workflow analysis
after performing semi-structured interviews. Providers did not mention any effect
on burnout or work hours. Participants did in fact provide unanimous support for this
intervention and did identify common points of interventions for our clinical workflow
analysis.
-
Which of the following methodologies was implemented in the development and evaluation
of our Neurosurgical Readmissions Reduction Program?
-
Machine learning model development
-
Clinical workflow analysis using semi-structured interviews
-
Simulation through agent-based modeling
-
Literature analysis for determining of variables predictive of neurosurgery readmission
-
All of the above
Correct Answer: The correct answer is option e. Machine learning, clinical workflow analysis, simulations,
and literature analysis were used in the development and evaluation of the Neurosurgical
Readmission Reduction Program.
