Download PDF
Methods Inf Med 1994; 33(04): 402-416
DOI: 10.1055/s-0038-1635048
Original Article
Schattauer GmbH

The Relative Accuracy of a Variety of Medical Diagnostic Programs

B. S. Todd
1   The Programming Research Group, Oxford University Computing Laboratory, UK
,
R. Stamper
1   The Programming Research Group, Oxford University Computing Laboratory, UK
› Author Affiliations
Further Information

Publication History

Publication Date:
08 February 2018 (online)

    Permissions and Reprints

Abstract:

Acute abdominal pain is one of the most widely studied applications of computer-aided diagnosis. The usual approach is to apply Bayes’ theorem with the assumption of conditional independence (“independence Bayes”). We compared various approaches to designing diagnostic programs for abdominal pain of suspected gynaecological origin. The methods range from statistical to knowledge-based. All programs were evaluated using a database of 1,270 cases collected retrospectively. Our results suggest that in this application no significant improvement in accuracy can be made by taking interactions into account, either by statistical or by knowledge-based means; independence Bayes is near-optimal. As far as accuracy is concerned, there appears to be little point in pursuing knowledge-based approaches. However, the “nearest neighbours” method using a new metric appears to be at least as accurate as independence Bayes. We argue that the nearest neighbours method is more suitable than independence Bayes for clinical use because of greater accountability.

Keywords

Computer-aided Diagnosis - Nearest Neighbours - Abdominal Pain
 

  • REFERENCES

  • 1 De Dombal FT. The diagnosis of acute abdominal pain with computer assistance: worldwide perspective. Ann Chir 1991; 45: 273-7.
    PubMedGoogle Scholar
  • 2 Adams ID, Chan M, Clifford PC. et al. Computer-aided diagnosis of acute abdominal pain: a multicentre study. BMJ 1986; 293: 800-4.
    CrossrefPubMedGoogle Scholar
  • 3 De Dombal FT, Leaper DJ, Horrocks JC, Staniland JR, McCann AP. Computer-aided diagnosis of acute abdominal pain. BMJ 1972; ii: 9-13.
    PubMedGoogle Scholar
  • 4 Wellwood JM, Spiegelhalter DJ. Computers and the diagnosis of acute abdominal pain. Br J Hosp Med 1989; 41: 564-7.
    PubMedGoogle Scholar
  • 5 Gunn AA. The acute abdomen: the role of computer-assisted diagnosis. Baillieres Clin Gastroenterol 1991; 05: 639-65.
    CrossrefPubMedGoogle Scholar
  • 6 Wellwood JM, Johannessen S, Spiegelhalter DJ. How does computer-aided diagnosis improve the management of acute abdominal pain?. Ann R Coll Surg Engl 1992; 74: 40-6.
    PubMedGoogle Scholar
  • 7 Paterson-Brown S. Strategies for reducing inappropriate laparotomy rate in the acute abdomen. BMJ 1991; 303: 1115-8.
    CrossrefPubMedGoogle Scholar
  • 8 Paterson-Brown S, Vipond MN, Simms K, Gatzen C, Thompson JN, Dudley HAF. Clinical decision making and laparoscopy versus computer prediction in the management of the acute abdomen. Br J Surg 1989; 76: 1011-3.
    CrossrefPubMedGoogle Scholar
  • 9 Stonebridge PA, Freeland P, Rainey JB, Macleod DAD. Audit of computer-aided diagnosis of abdominal pain in accident and emergency departments. Arch Emerg Med 1992; 09: 271-3.
    CrossrefPubMedGoogle Scholar
  • 10 Sutton GC. Computer-aided diagnosis: a review. Br J Surg 1989; 76: 82-5.
    CrossrefPubMedGoogle Scholar
  • 11 Dixon JM, Elton RA, Rainey JB, Macleod DAD. Rectal examination in patients with pain in the right lower quadrant of the abdomen. BMJ 1991; 302: 386-8.
    CrossrefPubMedGoogle Scholar
  • 12 Norusis MJ, Jacquez JA. Diagnosis. II. Diagnostic models based on attribute clusters: a proposal and comparisons. Comput Biomed Res 1975; 08: 173-88.
    CrossrefPubMedGoogle Scholar
  • 13 Fryback DG. Bayes’s theorem and conditional nonindependence of data in medical diagnosis. Comput Biomed Res 1978; 11: 423-34.
    CrossrefPubMedGoogle Scholar
  • 14 Todd BS, Stamper R. Limits to diagnostic accuracy. Med Inf (Lond) 1993; 18: 255-70.
    PubMedGoogle Scholar
  • 15 Hilden J. Statistical diagnosis based on conditional independence does not require it. Comput Biol Med 1984; 14: 429-35.
    CrossrefPubMedGoogle Scholar
  • 16 Russek E, Kronmal RA, Fisher LD. The effect of assuming independence in applying Bayes’ theorem to risk estimation and classification in diagnosis. Comput Biomed Res 1983; 16: 537-52.
    CrossrefPubMedGoogle Scholar
  • 17 Guppy KH, Detrano R, Abbassi N, Janosi A, Sandhu S, Froelicher V. The reliability of probability analysis in the prediction of coronary artery disease in two hospitals. Med Decis Making 1989; 09: 181-9.
    CrossrefPubMedGoogle Scholar
  • 18 Norusis MJ, Jacquez JA. Diagnosis. I. Symptom nonindependence in mathematical models for diagnosis. Comput Biomed Res 1975; 08: 156-72.
    CrossrefPubMedGoogle Scholar
  • 19 Nordyke R, Kulikowski CA, Kulikowski CW. A comparison of methods for the automated diagnosis of thyroid dysfunction. Comput Biomed Res 1971; 04: 374-89.
    CrossrefPubMedGoogle Scholar
  • 20 Croft DJ, Machol RE. Mathematical methods in medical diagnosis. Ann Biomed Eng 1987; 02: 69-89.
    PubMedGoogle Scholar
  • 21 Titterington DM, Murray GD, Murray LS. et al. Comparison of discrimination techniques applied to a complex data set of head-injured patients. J R Statist Soc A 1981; 144: 145-75.
    PubMedGoogle Scholar
  • 22 Orient JM. Evaluation of abdominal pain: clinicians’ performance compared with three protocols. South Med J 1986; 79: 793-9.
    CrossrefPubMedGoogle Scholar
  • 23 Gammerman A, Thatcher AR. Bayesian diagnostic probabilities without assuming independence of symptoms. Meth Inform Med 1991; 30: 15-22.
    PubMedGoogle Scholar
  • 24 Fox J, Barber D, Bardhan KD. A quantitative comparison with rule-based diagnostic inference. Meth Inform Med 1980; 19: 210-5.
    PubMedGoogle Scholar
  • 25 Heckerman DE, Horvitz EJ, Nathwani BN. Toward normative expert systems: Part I. The Pathfinder project. Meth Inform Med 1992; 31: 90-105.
    PubMedGoogle Scholar
  • 26 De Dombal FT. Computer-aided decision support in clinical medicine. Int J Biomed Comput 1989; 24: 9-16.
    CrossrefPubMedGoogle Scholar
  • 27 Heckerman DE, Nathwani BN. An evaluation of the diagnostic accuracy of Pathfinder. Comput Biomed Res 1992; 25: 56-74.
    CrossrefPubMedGoogle Scholar
  • 28 Edwards FH, Davies RS. Use of a Bayesian algorithm in the computer-assisted diagnosis of appendicitis. Surg Gynecol Obstet 1984; 158: 219-22.
    PubMedGoogle Scholar
  • 29 Cooper GF. A diagnostic method that uses causal knowledge and linear programming in the application of Bayes’ formula. Comput Meth Progr Biomed 1986; 22: 223-37.
    PubMedGoogle Scholar
  • 30 Harris N, Spiegelhalter DJ, Bull K, Franklin RCG. Criticizing conditional probabilities in belief networks. In: Proceedings of the Fourteenth Annual Symposium on Computer Applications in Medical Care. Los Alamitos, CA: IEEE Computer Society Press; 1990: 805-9.
    Google Scholar
  • 31 Seroussi B. The ARC & AURC Cooperative Group. Computer-aided diagnosis of acute abdominal pain when taking into account interactions. Meth Inform Med 1986; 25: 194-8.
    PubMedGoogle Scholar
  • 32 Ebehart RC, Dobbins RW, Hutton LV. Neural network paradigm comparisons for appendicitis diagnoses. In: Proceedings of Fourth Annual IEEE Symposium on Com-puter-Based Medical Systems. Los Alamitos, CA: IEEE Computer Society Press; 1991: 298-304.
    Google Scholar
  • 33 Emparanza JI, Aldamiz-Echevarria L, Perez-Yarza EG, Aldazabal P, Tovar J. Statistical approach to the acute abdomen diagnosis in children. In: Expert Systems and Decision Support in Medicine. Thirty-third Annual Meeting of the GMDS EFMI Special Topic Meeting. Peter L. Reichertz Memorial Conference. Berlin: Springer-Verlag; 1988: 131-5.
    Google Scholar
  • 34 Todd BS, Stamper R. The formal design and evaluation of a variety of medical diagnostic programs. Technical Monograph PRG-109. Oxford University Computing Laboratory; 1993
    Google Scholar
  • 35 Dickson JAS, Jones A, Telfer S, de Dombal FT. Acute abdominal pain in children. Scand J Gastroenterol Suppl 1988; 144: 43-6.
    PubMedGoogle Scholar
  • 36 Telfer S, Fenyo G, Holt PR, de Dombal FT. Acute abdominal pain in patients over 50 years of age. Scand J Gastroenterol Suppl 1988; 144: 47-50.
    PubMedGoogle Scholar
  • 37 Neapolitan RE. Probabilistic Reasoning in Expert Systems. New York: John Wiley; 1989
    Google Scholar
  • 38 Lauritzen SL, Spiegelhalter DJ. Local computation with probabilities on graphical structures and their application to expert systems. J R Statist Soc B 1988; 50: 157-224.
    PubMedGoogle Scholar
  • 39 Pauerstein CJ. Gynecological Disorders. Differential Diagnosis and Therapy. New York: Grune and Stratton; 1982
    Google Scholar
  • 40 Chamberlain G. Contemporary Gynaecology. London: Butterworths; 1984
    Google Scholar
  • 41 Whitfield CR. Dewhurst’s Textbook of Obstetrics and Gynaecology for Postgraduates, 4th ed. Oxford: Blackwell Scientific Publications; 1986
    Google Scholar
  • 42 Cestnik B. Estimating probabilities: a crucial task in machine learning. In: Proceedings of the Ninth European Conference on Artificial Intelligence. (ECAI90). London: Pitman; 1990: 147-9.
    Google Scholar
  • 43 Shortliffe EH. Computer-based Medical Consultations: MYCIN. New York: American Elsevier; 1976
    Google Scholar
  • 44 Duda R, Gaschnig J, Hart P. Model design in the PROSPECTOR consultant system for mineral exploration. In: Michie D. ed. Expert Systems in the Micro-Electronic Age. Edinburgh: Edinburgh University Press; 1979: 153-67.
    Google Scholar
  • 45 Felix WSWong, Leung KS, So YT. The recent development and evaluation of a medical expert system (ABVAB). Int J Biomed Comput 1990; 25: 223-9.
    CrossrefPubMedGoogle Scholar
  • 46 Spiegelhalter DJ, Knill-Jones RP. Statistical and knowledge-based approaches to clinical decision-support systems, with an application in gastroenterology. J R Statist Soc A 1984; 147: 35-77.
    PubMedGoogle Scholar
  • 47 Heckerman DE, Horvitz EJ. The myth of modularity in rule-based systems. In: Lem-mer JF, Kanal LN. eds. Uncertainty in Artificial Intelligence 2. Amsterdam: North-Holland Publ Comp; 1988: 23-34.
    Google Scholar
  • 48 Dan Q, Dudeck J. Certainty factor theory: its probabilistic interpretations and problems. Artif Intell Med 1992; 04: 21-34.
    CrossrefPubMedGoogle Scholar
  • 49 Todd BS, Stamper R, Macpherson PM. A probabilistic rule-based expert system. Int J Biomed Comput 1993; 33: 129-48.
    CrossrefPubMedGoogle Scholar
  • 50 Anderson JA. Logistic discrimination. In: Krishnaiah R, Kanal LN. eds. Handbook of Statistics 2. Amsterdam: North-Holland Publ Comp; 1982: 169-91.
    Google Scholar
  • 51 Todd BS, Stamper R, Macpherson PM. The design and construction of a medical simulation model. Comput Meth Progr Biomed 1994; 42: 77-91.
    CrossrefPubMedGoogle Scholar
  • 52 Shortliffe EH, Buchanan B, Feigenbaum EA. Knowledge engineering for medical decision making: a review of computer-based clinical decision aids. Proc IEEE 1979; 67: 1207-24.
    CrossrefPubMedGoogle Scholar
  • 53 Macartney FJ. Diagnostic logic. BMJ 1987; 295: 1325-31.
    CrossrefPubMedGoogle Scholar
  • 54 Franklin RCG, Spiegelhalter DJ, Macartney FJ, Bull K. Evaluation of a diagnostic algorithm for heart disease in neonates. BMJ 1991; 302: 935-9.
    CrossrefPubMedGoogle Scholar
  • 55 Stamper R, Todd BS, Macpherson PM. A software engineering approach to the design of a medical expert system. In: Proceedings of the Fourth International Conference on Software Engineering and Knowledge Engineering (SEKE 92). Los Alamitos, CA: IEEE Computer Society Press; 1992: 341-8.
    Google Scholar
  • 56 Breiman L, Friedman JH, Olshen RA, Stone CJ. Classification and Regression Trees. Monterey: Wadsworth and Brooks/Cole; 1984
    Google Scholar
  • 57 Cover TM, Hart PE. Nearest neighbour pattern classification. IEEE Trans Inf Theory 1967; 13: 21-7.
    CrossrefPubMedGoogle Scholar
  • 58 Todeschini T. k-nearest neighbour method: the influence of data transformations and metrics. Chemometrics Intell Lab Syst 1989; 06: 213-20.
    CrossrefPubMedGoogle Scholar
  • 59 Salzberg S. Distance metrics for instance-based learning. Lecture Notes Artif Intell 1991; 542: 399-408.
    PubMedGoogle Scholar
  • 60 Bounds DG, Lloyd PJ, Mathew BG. A comparison of neural network and other pattern recognition approaches to the diagnosis of low back pain. Neural Networks 1990; 03: 583-91.
    CrossrefPubMedGoogle Scholar
  • 61 Baxt WG. Use of an artificial neural network for the diagnosis of myocardial infarction. Ann Intern Med 1991; 115: 843-8.
    CrossrefPubMedGoogle Scholar
  • 62 Maclin PS, Dempsey J, Brooks J, Rand J. Using neural networks to diagnose cancer. J Med Syst 1991; 15: 11-21.
    CrossrefPubMedGoogle Scholar
  • 63 Akay M. Noninvasive diagnosis of coronary artery disease using a neural network algorithm. Biol Cybern 1992; 67: 361-7.
    CrossrefPubMedGoogle Scholar
  • 64 Rumelhart DE, Hinton GE, Williams RJ. Learning representations by back-propagating errors. Nature 1986; 323: 533-6.
    CrossrefPubMedGoogle Scholar
  • 65 Crichton NJ, Fryer JC, Spicer CC. Some points on the use of “independent Bayes” to diagnose acute abdominal pain. Stat Med 1987; 06: 945-59.
    CrossrefPubMedGoogle Scholar
  • 66 Crichton NJ, Hinde JP. Correspondence analysis as a screening method for indicants for clinical diagnosis. Stat Med 1989; 08: 1351-62.
    CrossrefPubMedGoogle Scholar
  • 67 Ohmann C, Kunneke M, Zaczyk R, Thon K, Lorenz W. Selection of variables using “independence Bayes” in computer-aided diagnosis of upper gastrointestinal bleeding. Stat Med 1986; 05: 503-15.
    CrossrefPubMedGoogle Scholar
  • 68 Chard T, Rubenstein EM. A model-based system to determine the relative value of different variables in a diagnostic system using Bayes theorem. Int J Biomed Comput 1989; 24: 133-42.
    CrossrefPubMedGoogle Scholar
  • 69 Mosteller F. Some statistical problems in measuring the subjective response to drugs. Biometrics 1952; 08: 220-6.
    CrossrefPubMedGoogle Scholar
  • 70 Hart A, Wyatt J. Connectionist models in medicine: an investigation of their potential. In: Proceedings of the Second European Conference on Artificial Intelligence in Medicine (AIME 89). Berlin: Springer-Verlag; 1989: 115-24.
    Google Scholar
  • 71 Phillips S, Wiles J, Schwartz S. A comparison of three classification algorithms on the diagnosis of abdominal pains. In: Proceedings of the Second Australian Conference on Neural Networks (ACNN 91). Sydney University Press; 1991: 283-7.
    Google Scholar
  • 72 Clark LA, Pregibon D. Tree-based models. In: Chambers JM, Hastie TJ. eds. Statistical models in S. Pacific Grove: Wadsworth and Brooks/Cole; 1992
    Google Scholar
  • 73 Teach R, Shortliffe E. An analysis of physician attitudes regarding computer-based clinical consultation systems. Comput Biomed Res 1981; 14: 542-58.
    CrossrefPubMedGoogle Scholar
  • 74 Young D, Chapman T, Poile C. Physician, reveal thyself. Br J Healthcare Comput 1990; 07: 16-21.
    PubMedGoogle Scholar
  • 75 Stamper T, Todd BS, Macpherson PM. Case-based explanation for medical diagnostic programs, with an example in gynaecology. Meth Inform Med 1994; 33: 205-13.
    PubMedGoogle Scholar
  • 76 Seroussi B. The ARC & AURC Cooperative Group. Comparison of several discrimination methods. Application to the acute abdomen. Lecture Notes Med Inf 1985; 28: 12-S.
    PubMedGoogle Scholar
  • 77 Reggia JA, Perricone BT. Answer justification in medical decision support systems based on Bayesian classification. Comput Biol Med 1985; 15: 161-7.
    CrossrefPubMedGoogle Scholar
  • 78 De Dombal FT, Barnes S, Dallos V. et al. How should computer-aided decision-support systems present their predictions to the practising surgeon?. Theor Surg 1992; 07: 111-6.
    PubMedGoogle Scholar