Comparison of illness severity scoring systems for mortality prediction in Neurointensive Care Unit in India

Abstract Background: Illness severity scoring systems (SSs) are increasingly being used to provide information about patients’ severity of illness and outcome in terms of mortality or length of Intensive care Unit (ICU) and hospital stay. In this retrospective study, we compared the predictive power of Acute Physiology and Chronic Health Evaluation (APACHE) II and IV, Simplified Acute Physiology Score (SAPS), Mortality Prediction Model at 24 h and Glasgow Coma Scale (GCS) with actual in-hospital 28 day mortality in patients admitted to neuro-ICU over a period of 6 months. Methods: The data required for calculation of above scores was retrieved from medical records. The 28-day post-admission outcome including in-hospital mortality was measured by Glasgow Outcome Scale (GOS). Logistic regression was used to determine the mortality prediction power of each SS. Results: A total of 197 adult patients with varied neurological diagnosis were included in this study. The in-hospital 28-day mortality rate was 19.8%, and the scores of all the SSs correlated significantly with GOS (P < 0.001). All the scores were significantly different between survivors and non-survivors. The accuracy of all the SSs to predict survival and non-survival was more than 80%. The highest accuracy rate was seen for GCS and SAPS (84.3% and 83.8%, respectively). Conclusions: The SSs used in this study had good predictive power, and they had good discriminative ability between survivors and non-survivors. GCS and SAPS have the highest predictive ability, GCS having added advantage of being simple and practical.


INTRODUCTION
Illness severity scoring systems (SSs) are used to provide information about the patients' severity of illness, to commonly used, include Acute Physiology and Chronic Health Evaluation (APACHE) II and IV, Mortality Prediction Model at 24 h (MPM 24 ) and Simplified Acute Physiology Score (SAPS). Furthermore, using these models, alternative models have been devised to cater to the specific patient population. [1] Glasgow Coma Scale (GCS) is commonly used in neuro-ICU (NICU) to describe the level of patient consciousness after brain injury. [2] It has been shown to have high inter-rater reliability even amongst registered nurses. [3] Although described initially for a head injury, it is generally applied to all types of neurosurgical population. Most SSs use GCS as one of the parameters in their model. Numerous studies have demonstrated a correlation between the GCS score on admission and long-term patient outcome. [4] In this study, we wanted to assess the predictive power of GCS when compared to other SSs. We routinely use only GCS for neurological assessment and prognostication of our patients in NICU as it is simple and easy to use in an ICU like ours, where there is a high turnover of patients. Moreover, neurological patients sometimes require augmentation of their systemic blood pressure with inotropes even if they are normotensive, in situations such as vasospasm after aneurysmal subarachnoid haemorrhage or in traumatic brain injury (TBI) to maintain their cerebral perfusion pressure. The use of SS in such scenarios may cause a bias towards the unfavourable outcome. In addition, respiratory insufficiency and tachypnoea can occur as a result of neurological insult, without any respiratory system pathology, thereby altering the information from SS. Moreover, we do not have electronic data entry in our NICU, which makes the calculation of APACHE or other SS tedious and time consuming. Therefore, we wanted to test whether GCS alone can be used for predicting patients' prognosis in our NICU and other similar centres in our country or whether we need to apply other SS to evaluate patient prognosis. This retrospective study was designed to compare the predictive power of APACHE II and IV, SAPS, MPM 24 and GCS with the actual in-hospital 28-day mortality of neurologically ill patients (both surgical and non-surgical) admitted to our NICU over a period of 6 months. In addition, we compared these SSs between patients who had good outcome and poor outcome based on Glasgow Outcome Scale (GOS). We hypothesised that predictive power of GCS is comparable to other SSs, in spite of the fact that it accounts for only neurological status and does not take into consideration any systemic illnesses.

SUBJECTS AND METHODS
The study was approved by the Institute Ethics Committee, and waiver of patient consent was obtained.
Medical records of all patients ≥18 years of age admitted to NICU from April 2012 to September 2012 were used for data collection. Exclusion criteria included pregnant patients and any patient who got discharged from ICU within 6 h of admission. The yearly turnover is around 500 patients. We mainly use GCS in our ICU for prognostication. Our nurse to patient ratio is usually 3:1 and functions as a closed unit under the full medical responsibility of the ICU staff in close concert with the referring medical specialists.
The following data were collected for each patient: 1. Characteristics: Age, sex, elective or emergency admission, any surgery done before ICU admission, any surgical service provided within first 24 h of ICU admission 2. Chronic health status: The Presence or absence of chronic renal failure, liver insufficiency, underlying malignancy, immunosuppression, any admission to ICU in the previous 6 months (defined in APACHE II SS). Patients with brain oedema are usually treated with steroids (4 mg 6 th hourly) as an anti-oedema measure and hence, such patients were considered as immune-compromised (dose >0.2 mg/kg/day) 3. Physiologic measurements (values recorded in the first 24 h of admission to ICU): The lowest and the highest values of the following: GCS, systolic blood pressure (SBP), diastolic blood pressure (DBP), heart rate, respiratory rate including both spontaneous and controlled breaths in intubated patients, temperature, urine output 4. Laboratory investigations (within first 24 h): Serum sodium, potassium, blood urea, (blood urea nitrogen was calculated as blood urea/3), serum creatinine, blood glucose, haematocrit, total leucocyte count, platelet count, arterial pH, partial pressure of oxygen in arterial blood , fraction of inspired oxygen, partial pressure of carbon dioxide in arterial blood, blood bicarbonate levels and prothrombin time (PT) 5. Intensity of treatment: Number of invasive lines in place (endotracheal tube, arterial line, urinary catheter, central venous catheter, peripheral intravenous access, external ventricular drain, lumbar drain), cardiopulmonary resuscitation (CPR) before admission to ICU, any evidence of infection at admission or subsequently proven infection in 24 h, shock in the first 24 h and total number of hours of mechanical ventilation 6. Outcome of patients: As in-hospital mortality within 28 days of hospital admission or as GOS at the end of 28 days from admission or at discharge whichever is earlier.

Statistical analysis
Continuous variables are described in mean ± standard deviation (SD) and categorical variables are described in numbers and percentages. Non-parametric (Spearman) correlation was used to find out the relation between the scores of different SSs and GOS. A logistic regression was used for each SS to determine if the SS could be used to predict survival or death. Mann-Whitney test was used to compare the illness severity SSs between survivors and non-survivors and between those with favourable and unfavourable outcome. Model discrimination was analysed using the area under the curve (AUC). Discrimination was classified as perfect, excellent, very good, good, moderate and poor if the AUCs, were 1.0, 0.9-0.99, 0.8-0.89, 0.7-0.79, 0.6-0.69 or 0.6, respectively. [5] Calibration for each model was assessed using the Hosmer-Lemeshow goodness-of-fit C statistic. A non-significant P value was considered evidence of good calibration. Individually, each variable was again compared between survivors and non-survivors using Chi-square test and the variables which were significant were entered into multivariate analysis. Data analyses were performed using SPSS software for windows (SPSS Inc. Version 15.0., Chicago, IL, USA).

DISCUSSION
In this single centre retrospective study, we found that the mortality rate predicted by different illness severity SSs correlated negatively with GOS, that is, as the score value of APACHE II, APACHE IV, MPM and SAPS increased, patient outcome worsened and vice-versa. On the other hand, for GCS, the higher the GCS, the better was the outcome. The correlation between the scores of different SSs and mortality was significant. The AUC for the most of the SSs was more than 0.7 indicating good discriminative ability. The ability of all the SSs to predict survival and non-survival was more than 80%. The highest predictability was seen for GCS and SAPS (84.3% and 83.8%, respectively).
the illness severity scores such as GCS, APACHE II, APACHE II predicted mortality rate (PMR), APACHE IV, APACHE IV PMR, MPM PMR, SAPS II and SAPS II PMR, were significantly different between the two groups as shown in Table 4. Similarly, patients with unfavourable outcome (GOS 1-2; n = 58) had higher APACHE II, APACHE IV, MPM and SAPS score and PMR compared to patients with favourable outcome (GOS 3-5; n = 139) [ Table 5]. Figure 1 depicts the receiver operating characteristic curves which were constructed to determine the discriminative power of the SSs and Table 6 Table 7 was >0.05 indicating that the models fit the data.  A number of studies have been done in the past comparing the various SSs to predict mortality in the ICU. Cho and Wang, [6] compared APACHE III, APACHE II and GCS in patients with acute head injury for prediction of mortality and functional outcome. In their study, early hospital mortality prediction was similar between APACHE and GCS SSs. However, for late mortality prediction, APACHE II and III scored better than GCS. Though GCS was initially described to assess patients' level of consciousness and not to predict mortality, in our study, we found that GCS had very good predictability for 28-day in-hospital mortality rate. Unlike the Cho's study, which included only head injured patients, our study included not only patients with neurosurgical pathology (both traumatic and non-traumatic), but also patients with non-surgical diagnoses like GBS, and myasthenia gravis. In spite of this wide variety of neurological population, GCS turned out to be a significant predictor of mortality. Ting et al., [7] compared APACHE II and SAPS II with multiple linear regression model of GCS (GCS-mr), to predict mortality in 154 neurosurgical patients. They demonstrated that the power of GCS-mr was similar to APACHE II and SAPS II in predicting mortality. The advantage of APACHE II over SAPS II is that it has special disease calibration and hence better mortality prediction with different disease groups. A retrospective study by Nejmi et al., [8] compared APACHE II and SAPS II in 225 patients with TBI. The end point for prognostic analysis was in-hospital mortality. Although both APACHE II and SAPS II were accurate in predicting mortality, the authors observed better discriminative power with APACHE II as compared to SAPS II. However, in the current study, SAPS II had better discrimination ability. One possibility could be that in SAPS II, emergency admission is given more weightage. This is important as the target population in this study were patients with head injuries or other neurological/neurosurgical pathology who require emergency ICU care. This may be contributing to slightly better discriminative ability of SAPS II as compared to APACHE II in our study.   Zali et al. [9] compared APACHE II and GCS in 93 TBI patients with multiple trauma and found APACHE II to be superior to GCS. In this study, all the trauma patients studied had isolated head injury. Hence, we observed better discriminative power with GCS. In patients with neurological illness, GCS performed the best, probably because of its diagnostic specificity.
All the SSs studied in the present data include GCS as one of the components, and the contribution of GCS varies in each SS. In APACHE II SS, the contribution of GCS is 16.9%, and in APACHE III, it is 19%. [7] Although many parameters were significantly different between survivors and non-survivors, in the multivariate analysis, only CPR within the first 24 h of admission turned out to be a significant and independent predictor of mortality. In addition, in a study conducted by Cuthbertson et al., [10] the authors found that requirement of CPR in the 24 h preceding ICU admission was predictive of outcome in their ICU and it is a late sign of deterioration. All the previously published studies, which compared the SSs in patients with neurological diagnosis, have included a very limited variety of neurological diseases such as either only head injury or elective neurosurgical population. We, in our study, included a diverse neurological population such as elective spine and brain tumours, patients with neurotrauma, and patients with myasthenia, GBS and cortical venous thrombosis. We do not follow the policy of withdrawal of care.
In a study by Brandner et al., [11] the authors introduced a new score called Clinical Course Score (CCS) defined as the difference between the GOS grade (assessed at 6 months after discharge) and the reduced GCS grade (five-grade scale) and applied it to 248 patients with neurotrauma and SAH. CCS ranges from −4 to +4. A positive value would represent an improving clinical course of the patient during the post-traumatic phase, whereas a negative CCS value would represent a deteriorating clinical course. It can be easily calculated retrospectively, can be evaluated at various points in time following neurotrauma, and provides better insight into the assessment of the beneficial effects of various therapeutic procedures in neurotrauma.
In a study similar to ours, by Handschu et al., [12] the authors tested the performance of SAPS I/II and GCS in ninety patients with acute stroke to predict mortality at 10 days, 90 days and 1 year. The authors found that all the three scores had good predictive value and were independent predictors for predicting short-term and long-term mortality in stroke patients. They also concluded that GCS and even its motor subscore provided the same prognostic information as the other complex severity of illness scores and therefore recommended its routine use in critically ill stroke patients.
In this study, we found that GCS and SAPS II have the highest predictive ability amongst all the scores, although all the other models also performed fairly well. One of the advantages of GCS over other SSs is that it is simple, practical and most importantly does not require enormous effort for data collection and calculation.
Our study has some limitations. First, as the study design was retrospective, a 3.5% of observations were missing, and the missing values were substituted with normal values for the calculator to calculate the scores. As the missing values were <10%, we assume that the results would not have been different but needs validation. Second, as this was a single centre study, the applicability of results to a larger population needs to be tested. Third, it has been shown that SAPS II model fail for patients who stay in the ICU for more than a week. As we did not record ICU or hospital LOS, high predictive power of SAPS II results should be interpreted with caution.

CONCLUSIONS
We found that the GCS and SAPS II PMR SSs had higher chances of predicting mortality in patients with neurological illnesses. The use of GCS can be continued in view of its simplicity.