The Measurement of Performance in Probabilistic Diagnosis - Hilden, J.; Habbema, J. D. F.; Bjerregaard, B. (Methods of Information in Medicine, 1978)
Attention is focused on one important aspect of good performance in probabilistic diagnosis, the »reliability« (external validity) of probabilistic assertions: a diagnostic alternative claimed to be 90% certain, say, must occur neither more nor less than nine times out of ten on the average. Statistical measures are offered by which departures from such perfect reliability can be estimated, and statistical tests are developed in order to test the hypothesis of perfect reliability. The specific reliability defects looked for include overconfident diagnoses and so-called size bias (common diseases being over diagnosed). Reliability is contrasted with discriminatory power and other performance aspects. The illustrative data derive from a study of computer-aided diagnosis of the acute abdomen.
Keywords: Computer-aided Diagnosis; External Validity; Reliability; Overconfidence; Size Bias; Graphical Reliability Analysis