Download PDF
Methods Inf Med 1979; 18(04): 222-227
DOI: 10.1055/s-0038-1636474
Original Article
Schattauer GmbH

On the Optimum Choice of Categories for the Classification of Biomedical Data Patterns

ZUR OPTIMALEN WAHL VON KLASSENEINTEILUNGEN ZUR KLASSIFIZIERUNG BIOMEDIZINISCHER DATEN
C. F. Hess
1   From the Institute of Physiology, Johannes Gutenberg-University Mainz, Fed. Rep. of Germany
,
K. Brodda
1   From the Institute of Physiology, Johannes Gutenberg-University Mainz, Fed. Rep. of Germany
› Author Affiliations The work has been supported by Deutsche Forschungsgemeinschaft, Project Me 274/2.
Further Information

Publication History

Publication Date:
19 February 2018 (online)

    Permissions and Reprints

The optimum choice of categories for pattern recognition problems is on the one hand determined by the requirement of a low rate of misclassification. On the other hand, the classification of a pattern should result in an information gain as high as possible. A criterion for an optimum choice of categories which is the best compromise between the demands mentioned above is worked out. The Bayes rule is used as a decision function. The alteration of the Bayes risk as indicator for the rate of malrecognition is examined for different choice of categories concerning the very same classification problem. As the calculation of the Bayes risk is commonly difficult, an estimation using the Bhattacharyya coefficient is given. The information content of a choice of categories is defined using Shannon’s information measure. The alteration of the information contents is analyzed by putting together certain categories, i.e. a coarser choice of categories. With the aid of the relative information loss and the relative reduction of the Bayes risk coefficient, a criterion on the goodness of a choice of categories can be given. The criterion also serves as an optimum choice of classes. The extension of the latter criterion to a generalized decision rule is possible.

Die optimale Wahl einer Klasseneinteilung für Probleme der Mustererkennung mit mehreren Entscheidungsmöglichkeiten wird einerseits durch die Forderung nach einer Ideinen Fehlerrate bei der Klassifikation bestimmt; andererseits soll die Einordnung in eine Klasse zu einem möglichst großen Informationsgewinn führen. Es wird ein Kriterium für eine optimale Wahl der Klassen erarbeitet, das einen Kompromiß zwischen diesen beiden, z.T. einander widersprechenden Forderungen darstellt. Als Entscheidungsfunktion wird die Bayes-Regel benutzt. Die Veränderung des Bayes-Risikos bei anderer Wahl der Klassen für dasselbe Klassifikationsproblem wird, da diese Größe schwer zu berechnen ist, mit Hilfe des Bhattacharyya-Koeffizienten abgeschätzt. Der Informationsgehalt einer Klasseneinteilung wird mit Hilfe des Shannonschen Informationsmaßes definiert und die Änderung des Informationsgehalts bei Zusammenlegung von Klassen, d. h. bei Wahl einer gröberen Klasseneinteilung berechnet. Mit Hilfe der relativen Veränderungen von Informationsgehalt und Risiko läßt sich ein Kriterium für die Güte und daraus abgeleitet ein solches für die optimale Wahl einer Klasseneinteilung angeben. Das Kriterium läßt sich zu einer verallgemeinerten Entscheidungsregel erweitern.

Key-Words

Decision Theory - Classification - Bayes Rule - Information Content of Classifications

Schlüssel-Wörter

Entscheidungstheorie - Klassifikation - Bayes-Regel - Informationsgehalt einer Klasseneinteilung
 

  • References

  • 1 ANDEBSON T. W.. An Introduction to Multivariate Statistical Analysis. ( Wiley; New York: 1958. ).
    Google Scholar
  • 2 ASH R.. Information Theory. ( Wiley; New York: 1965. ).
    Google Scholar
  • 3 BLACKWELL D., GIBSHICK M. A.. Theory of Games and Statistical Decisions. ( Wiley; New York: 1954. ).
    Google Scholar
  • 4 CATTANEO A. D., LUCCELLI P. E., ROCCA E.. Decision theory on surgical risk. Meth. Inform. Med 13 ( 1974; ) 238-241.
    PubMedGoogle Scholar
  • 5 CHOW C. K.. On optimum recognition error and reject tradeoff. IEEE Trans. Inf. Theory 16 ( 1970; ) 41-46.
    CrossrefPubMedGoogle Scholar
  • 6 CHU J. T.. Optimal decision functions for computer character recognition. J. Ass. Comput. Mach 12 ( 1965; ) 213-226.
    CrossrefPubMedGoogle Scholar
  • 7 CHU J. T., CHUEH J. C.. Error probability in decision functions for computer character recognition. J. Ass. Comput. Mach 14 ( 1967; ) 273-280.
    CrossrefPubMedGoogle Scholar
  • 8 COVEB T. M., HABT P. E.. Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13 ( 1967; ) 21-27.
    CrossrefPubMedGoogle Scholar
  • 9 Fu K. S.. Sequential Methods in Pattern Recognition and Machine Learning. ( Academic Press; New York: 1968. ).
    Google Scholar
  • 10 FUKUNAGA K., KBILE T. F.. Calculation of Bayes' recognition error for two multivariate Gaussian distributions. IEEE Trans. Comput. IS ( 1969; ) 220-220.
    PubMedGoogle Scholar
  • 11 HEBMANS J., EGGEBMONT J. J., HAGEDOOBEN J., ODENTHAL D. W.. Probabilistic differential diagnosis of auditory dysfunction on the basis of electrocochleography. Meth. Inform. Med 14 ( 1975; ) 87-96.
    PubMedGoogle Scholar
  • 12 HEYDORN R. P.. An upper bound estimate on classification error. IEEE Trans. Inf. Theory 14 ( 1968; ) 783-784.
    CrossrefPubMedGoogle Scholar
  • 13 KAILATH T.. The divergence and Bhattacharyya distance measures in signal selection. IEEE Trans. Comm. Technol 15 ( 1967; ) 52-60.
    CrossrefPubMedGoogle Scholar
  • 14 KLINGEMAN J., PIPBEBGEB H. V.. Computer classifications of electrocardiograms. Comput. biomed. Res 1 ( 1967; ) 1-17.
    CrossrefPubMedGoogle Scholar
  • 15 KULLBACK S.. Information Theory and Statistics. ( Wiley; New York: 1959. ).
    Google Scholar
  • 16 LAINIOTIS D. G.. A class of upper bounds on probability of error for multihypotheses pattern recognition. IEEE Trans. Inf. Theory 15 ( 1969; ) 730-731.
    CrossrefPubMedGoogle Scholar
  • 17 Low D. A., SELLES W. D., NEUBATH P. W.. Bayes' Theorem applied to chromosome classification. Comput. biomed. Res 4 ( 1971; ) 561-570.
    CrossrefPubMedGoogle Scholar
  • 18 MICHAELIS J.. Zur Anwendung der Diskriminanzanalyse für die medizinische Diagnostik. Habilitationsschrift, Mainz. 1972
    PubMedGoogle Scholar
  • 19 MUCCIARDI A. N., GOSE E. E.. A comparison of seven techniques for choosing subsets of pattern recognition properties. IEEE Trans. Comput 20 ( 1971; ) 1023-1031.
    PubMedGoogle Scholar
  • 20 OICAJIMA M., STABK L., WHIPPLE G., JASTTI S.. Computer pattern recognition techniques: Some results with real electrocardiographic data. IEEE Trans, bio-med. Electron 10 ( 1963; ) 106-114.
    PubMedGoogle Scholar
  • 21 PIPBEBGEB H. V., KLINGEMAN J. D., COSMA J.. Computer evaluation of statistical properties of clinical information in the differential diagnosis of chest pain. Meth. Inform. Med 7 ( 1968; ) 79-92.
    PubMedGoogle Scholar
  • 22 STABK L., OKAJIMA M., WHIPPLE G. H.. Computer pattern recognition techniques: electrocardiographic diagnosis. Comm. ACM 5 ( 1962; ) 537-542.
    PubMedGoogle Scholar
  • 23 STAKMEE C. F., LEE K. L.. A mathematical approach to medical decisions: Application of Bayes' rule to a mixture of continuous and discrete clinical variables. Comput. biomed. Res 9 ( 1976; ) 531-541.
    CrossrefPubMedGoogle Scholar
  • 24 YOUNG T. Y., CALVEBT T. W.. Classification, Estimation and Pattern Recognition. ( Elsevier; New York: 1974. ).
    Google Scholar
  • 25 YOUNG T. Y., HUGGINS W. H.. Computer analysis of electrocardiograms using a linear regression technique. IEEE Trans, bio-med. Engin 11 ( 1964; ) 60-67.
    PubMedGoogle Scholar