Forecasting Survival in the Medical Intensive Care Unit: A Comparison of Clinical Prognoses With Formal Estimates

Abstract

Physicians often need to make prognostic judgments. In the present study, the accuracy was explored of survival estimates for patients in the Medical Intensive Care Unit (MICU). Estimates were made by physicians and nurses several times during each patient’s stay in the MICU and were compared to those of the APACHE II scale, a widely used quantitative index for critically ill patients. ROC curve and calibration curve analyses were performed to assess the accuracy of these estimates. Results revealed that MICU personnel were fairly accurate discriminators of patients who survived vs. who died, although there was a consistent tendency to underestimate survival. In addition, there was some relationship between the level of physician training and forecasting accuracy, but only within the patient’s first 24 hours in the MICU. Finally, the estimates of physicians did not differ significantly from those of the APACHE II scale. Physicians tended to be better calibrated in their predictions, while the APACHE II scale was slightly superior in terms of discrimination.

Key-Words

Forecasting Survival - Calibration - Intensive Care Survival - APACHE

• REFERENCES

• 1 Chassin MR. Costs and outcomes of medical intensive care. Med Care 1982; 20: 165-79.
• 2 Engelhardt HT, Rei MA. Intensive care units, scarce resources, and conflicting principles of justice. JAMA 1986; 255: 1159-64.
• 3 Knaus WA, Draper EA, Wagner DP, Zimmerman JE. An evaluation of outcome from intensive care in major medical centers. Ann Int Med 1986; 104: 410-18.
• 4 Wagner DP, Knaus WA, Draper EA. Physiologic abnormalities and outcome from acute disease. Arch Int Med 1986; 146: 1389-96.
• 5 Poses RM, Bekes C, Copare FJ, Scott WE. The answer to “What are my chances, doc?” depends on whom you ask: Prognostic disagreement and inaccuracy for critically ill patients. Crit Care Med 1989; 17: 827-33.
• 6 Poses RM, Bekes C, Copare FJ, Scott WE. What difference do two days make? The inertia of physicians’ sequential prognostic judgments for critically ill patients. Med Dec Making 1990; 10: 6-15.
• 7 McCIish DK, Powell SK. How well can physicians estimate mortality in the medical intensive care unit?. Med Decis Making 1989; 09: 125-32.
• 8 Brannen AL, Godfrey U, Goetter WE. Prediction of outcome from critical illness: A comparison of clinical judgment with a prediction rule. Arch Int Med 1989; 149: 1083-6.
• 9 Kruse JA, Thill-Baharozian MC, Carlson RW. Comparison of clinical assessment with APACHE 11 for predicting mortality risk in patients admitted to a medical intensive care unit. JAMA 1988; 260: 1739-42.
• 10 Brier GW. Verification of forecasts in terms of probability. Monthly Weather Rev 1950; 75: 1-3.
• 11 Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29-36.
• 12 Metz CE. Basic principles of ROC analysis. Sem Nuc Medicine 1978; 08: 283-9.
• 13 McNeil BJ, Adelstein SJ. Determining the value of diagnostic and screening tests. J Nuc Med 1976; 17: 439-44.
• 14 Griner PF, Mayewski RJ, Mushlin AI, Greenland P. Selection and interpretation of diagnostic tests and procedures: principles and application. Ann Int Med 1981; 94: 557-92.
• 15 Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic (ROC) curves derived from the same cases. Radiology 1983; 148: 839-44.
• 16 Kahneman D, Tversky A. On the psychology of prediction. Psych Rev 1973; 80: 237-51.
• 17 Christensen-Szalanski JJ, Bushyhead JB. Physician’s use of probabilistic information in a real clinical setting. J Exper Psych Human Percep Perf 1981; 04: 928-35.
• 18 Poses RM, Cebul RD, Collins M, Fager SS. The accuracy of experienced physicians probability estimates for patients with sore throats. JAMA 1985; 254: 925-9.
• 19 Berwick DM, Thibodeau LA. Receiver operating characteristic analysis of diagnostic skill. Med Care 1983; 21: 876-85.
• 20 Tierney WM, Fitzgerald J, McHenry R. et al. Physicians’ estimates of the probability of myocardial infarction in emergency room patients with chest pain. Med Dec Making 1986; 06: 12-7.