Use of a Hospital Database to Determine the Characteristics of Diagnostic Tests

Abstract

Test indices are often determined by comparing test results of healthy persons with test results of patients known to have the disease. However, the patient population for which the test is ordered in clinical practice often differs from the study population on which the test indices are based. Hence, these indices are not applicable to clinical practice and should be recalculated using data from daily clinical practice. Two major problems of using routinely collected data are discussed: the assessment of the final health status and tracing the reason for ordering the test. Prior considerations are given to the use of hospital information systems (HIS) to sample the patient population that is desired and to collect the necessary data for calculating test indices. We investigated whether the HIS of Leiden University Hospital (which is presented as an example) can be used to calculate the indices of clinical laboratory tests, histopathologic examinations and radiodiagnostic investigations. The results indicate that the registration of diagnoses must be improved and that a way must be found to capture the implicit reasoning for ordering diagnostic tests.

Key-Words

Hospital Information Systems - Diagnoses Registration - Diagnostic Test Indices - Systematic Errors

¹ Until February 1988 the author was employed at BAZIS (Central Development and Support Group Hospital Information Systems), Leiden, where the study was initiated.

• REFERENCES

• 1 Fletcher RH, Fletcher SW, Wagner EH. Clinical Epidemiology. Baltimore: Williams & Wilkins; 1982
• 2 Galen RS, Gambino SR. Beyond Normality: The Predictive Value and Efficiency of Medical Diagnoses. New York: Wiley; 1975
• 3 Ingelfinger JA, Mosteller F, Thibodeau LA, Ware JH. Biostatistics in Clinical Medicine. New York: Macmillan Publishing Co; 1983
• 4 Weiss NS. Clinical Epidemiology. Oxford: Oxford University Press; 1986
• 5 Zwetsloot-Schonk JHM, Snitker P, Vandenbroucke JP, Bakker AR. Using hospital information systems for clinical epidemiological research. Med Inform 1989; 14: 53-62.
• 6 Hlatky MA, Mark DB, Harrell FE, Lee KL, Califf RM, Pryor DB. Rethinking sensitivity and specificity. Am J Cardiol 1987; 59: 1195-8.
• 7 Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med 1978; 299: 926-30.
• 8 Foti AG, Cooper JF, Herschman H, Malvaez RR. Detection of prostatic cancer by solid-phase radioimmunoassay of serum prostatic acid phosphatase. N Engl J Med 1977; 297: 1357-61.
• 9 Gittes R. Acid phosphatase reappraised. N Engl J Med 1977; 297: 1398-9.
• 10 Klein LA, Shapiro P. Role of acid phosphatase measurement in management of prostatic cancer. Urology 1981; 17: 550-3.
• 11 Libshitz HI. Thermography of the breast: current status and future expectations. JAMA 1977; 238: 1953-4.
• 12 Meeker WR, Kashmiri R, Hunter L. Clinical evaluation of carcinoembryonic antigen in clinical disorders. Gut 1973; 14: 794-9.
• 13 Pontes JE, Choe BK, Rose NR, Ercole C, Pierce JM. Clinical evaluation of immunological methods for detection of serum prostatic acid phosphatase. J Urol 1981; 126: 363-5.
• 14 Segal AW, Trustey SF, Levi AJ. Re-evaluation of nitroblue-tetrazolium test. Lancet 1973; 02: 879-83.
• 15 Cain KC, Greenes RA, Begg CE. Patient-oriented performance measures of diagnostic tests: 3. U-factor. Med Decis Making 1984; 04: 33-46.
• 16 Doubilet P. A mathematical approach to interpretation and selection of diagnostic tests. Med Decis Making 1983; 03: 177-95.
• 17 Greenes RA, Cain KC, Begg CB. Patient-oriented performance measures of diagnostic tests: 1. Tools for prospective evaluation of test order decisions. Med Decis Making 1984; 04: 7-15.
• 18 Greenes RA, Begg CB, Cain KC, Swets J A, Feehrer CE, McNeil JB. Patient-oriented performance measures of diagnostic tests: 2. Assignment potential and assignment strength. Med Decis Making 1984; 04: 17-31.
• 19 Fineberg HV. Clinical chemistries: the high cost of low cost diagnostic tests. Med Tech 1979; 144-56.
• 20 Wong ET. Cost-effective use of laboratory tests: a joint responsibility of clinicians and Iaboratorians. Clin Lab Med 1984; 04: 665-72.
• 21 Deneef P. Evaluating rapid tests for streptococcal pharyngitis: The apparent accuracy of a diagnostic test when there are errors in the standard of comparison. Med Deris Making 1987; 07: 92-6.
• 22 Staquet M, Rozencweig M, Lee YJ, Mug-gia FM. Methodology for the assessment of new dichotomous tests. J Chron Dis 1981; 34: 599-610.
• 23 Thibodeau LA. Evaluating diagnostic tests. Biometrics 1981; 37: 801-4.
• 24 Panzer RJ, Suchman AL, Griner PE. Work-up bias in prediction research. Med Deris Making 1987; 07: 115-9.
• 25 Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics 1983; 39: 207-16.
• 26 Gray R, Begg CB, Greenes RA. Construction of receiver operating characteristic curves when disease verification is subject to selection bias. Med Decis Making 1984; 04: 151-64.
• 27 Bakker AR. Implementation, approach and evaluation of the use of the Leiden University Hospital Information System. In: Shires DH, Wolf H. eds. MEDINFO 77. Amsterdam: North Holland Publ Comp; 1977: 943-7.
• 28 Bakker AR, Mol JL. Hospital information systems. Effective Health Care 1984; 01: 169-80.
• 29 Bakker AR. The development of an integrated and co-operative hospital information system. Med Inform 1984; 09: 135-42.
• 30 Kouwenberg JM, Bongers A, Bakker AR. Storage structure in a large database. In: Barber BF, Gremy F, Uberla K, Wagner G. eds. WllE 79. Berlin: Springer Verlag; 1979: 590-601.
• 31 Leguit FA. The Case: Leiden University Hospital Information System. In: Bakker AR, Ball MJ, Scherrer JR, Willems JL. eds. Towards New Hospital Information Systems. Amsterdam: North-Holland Publ Comp; 1988: 59-65.
• 32 Komaroff AL. The variability and inaccuracy of medical data. Proc IEEE 1979; 67: 1196-207.
• 33 Albert A. On the use and computation of likelihood ratios in clinical chemistry. Clin Chem 1982; 28: 1113-9.
• 34 Linnet K. A review on the methodology for assessing diagnostic tests. Clin Chem 1988; 34: 1379-86.
• 35 Polister PE. A structural method to guide test evaluation. Med Deris Making 1985; J: 417-36.
• 36 Rifkin RD. Maximum Shannon information content of diagnostic medical testing: including application to multiple non-independent tests. Med Decis Making 1985; 05: 179-90.
• 37 Wennberg JE. Non-experimental methods in the assessment of efficacy. Med Decis Making 1988; 08: 175-6.
• 38 Hsia DC, Krushat WM, Fagan RRA, Teb-butt JA, Kusserow RP. Accuracy of diagnostic coding for medicare patients under the prospective-payment system. N Engl J Med 1988; 318: 352-5.
• 39 Safran C, Porter D, Lightfoot J, Rury CD, Underhill LH, Bleich HL, Slack WV. ClinQuery: a system for online searching of data in a teaching hospital. Ann Intern Med 1989; 111: 751-6.
• 40 Yerushalmy J. Statistical problems in assessing methods of medical diagnosis, with special reference to X-ray techniques. Pub Health Reps 1947; 62: 1432-49.
• 41 Lilienfeld AM, Kordan B. A study of variability in the interpretation of chest X-rays in the detection of lung cancer. Cancer Res 1966; 26: 2145-7.
• 42 Weinstein MC, Fineberg HV. (eds). Clinical Decision Analysis. Philadelphia: W. B. Saunders Comp; 1980