Untersuchungen und Empfehlungen zum Design von ROC-Studien in der Nuklearmedizin

Zusammenfassung

Die ROC-Analyse ist die Methode der Wahl zur objektiven Bewertung diagnostischer Verfahren, erfordert aber große Stichprobenumfänge. Wir untersuchten den Einfluß von Studiendesign und Datenanalyse auf den erforderlichen Stichprobenumfang. Der Nachweis eines Sensitivitätsunterschieds von 75% gegenüber 90% (Spezifität 90%, Teststärke 0,8) erfordert Stichprobenumfänge von jeweils 123 für das Patienten- und Kontrollkollektiv. Die Anwendung des korrelierten bivariaten binormalen ROC-Modells erlaubt bei Vergleichsstudien am selben Patientenkollektiv eine Reduktion des Stichprobenumfangs von über 35%. Wenn ein Kollektiv zahlenmäßig überwiegt, dann kann jeweils das andere Kollektiv verkleinert werden; auch hier ermöglicht das korrelierte ROC-Modell eine substantielle Verkleinerung des Stichprobenumfangs. Ein problemorientiert angepaßtes Studiendesign, möglichst mit Paralleluntersuchungen am selben Patienten- und Kontrollkollektiv, und die Berücksichtigung der resultierenden Datenstrukturen durch das korrelierte ROC-Modell erlauben eine statistisch abgesicherte objektive Bewertung diagnostischer Verfahren mit der ROC-Analyse an vergleichsweise kleinen Kollektiven.

Summary

ROC analysis is the method of choice for an objective assessment of diagnostic tests; however, it requires large sample sizes. We investigated the influence of study design and data analysis on sampling requirements. A sample size of 123 for the patient as well as the control group, is required to prove a difference in sensitivity of 75% vs 90% (specificity 90%, statistical power 0.8). Analysis with the binormal bivariate ROC model allows >35% reduction in sample size. If the patient group is increased the control group can be smaller, and vice versa; here the correlated ROC model also allows substantial decreases in sample size. If both diagnostic tests are performed in the same patient and control group and evaluated with the correlated ROC model, an objective, statistically sound assessment of diagnostic performance is possible with relatively small samples.

Schlüsselwörter

ROC-Analyse - korreliertes ROC-Modell - Studiendesign - Teststärke - Stichprobenumfänge

Keywords

ROC analysis - correlated ROC model - study design - statistical power - sample size

• LITERATUR

• 1 Begg CB, McNeil BJ. Assessment of radiological tests: Control of bias and other design considerations. Radiology 1988; 167: 565-9.
• 2 Bronskill MJ, Henkelman RM, Poon PY. et al. Magnetic resonance imaging, computed tomography, and radionuclide scintigraphy in detection of liver metastases. J Can Assoc Radiol 1988; 39: 3-9.
• 3 Centor RM. Signal detectability: The use of ROC curves and their analyses. Med Decis Making 1991; 11 (Suppl. 02) 102-6.
• 4 DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 1988; 44: 837-45.
• 5 Dorfman DD, Alf E. Maximum-likelihood estimation of parameters of signal detection theory and determination of confidence-interval rating method data. J Math Psych 1969; 6: 487-96.
• 6 Egan JP. Signal detection theory and ROC analysis. New York: Academic Press; 1975
• 7 Freedman LS. Evaluating and comparing imaging techniques: a review and classification of study designs. Br J Radiol 1987; 60: 1071-81.
• 8 Gray R, Begg CB, Greenes RA. Construction of receiver operating characteristic curves when disease verification is subject to bias. Med Decis Making 1984; 4: 151-64.
• 9 Green DM, Swets JA. Signal detection theory and psychophysics. New York: Krieger; 1966. 74
• 10 Hanley JA. Receiver operating characteristic (ROC) methodology: the state of the art. Crit Rev Diagn Imaging 1989; 29 (Suppl. 03) 307-35.
• 11 Hanley JA. The robustness of the »binormal« assumptions used in fitting ROC curves. Med Decis Making 1989; 8: 197-203.
• 12 Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29-36.
• 13 Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983; 148: 839-43.
• 14 Hunink MG, Richardson DK, Doubilet P, Begg CB. Testing for fetal pulmonary maturity: ROC analysis involving covariates, verification bias, and combination testing. Med Decis Making 1990; 10: 201-11.
• 15 Judy PF, Swensson RG. Detectability of equally visible disks at unknown locations. Radiology 1988; 169: 239.
• 16 Kijewski MF, Swenson RG, Judy PF. Analysis of rating data from multiple-alternative tasks. J Math Psych 1989; 33: 428-51.
• 17 Lusted LB. Logical analysis in Roentgen diagnosis. Radiology 1960; 74: 178-93.
• 18 McClish DK. Analyzing a portion of the ROC curve. Med Decis Making 1989; 9: 190-5.
• 19 McClish DK. Determining a range of false-positive rates for which ROC curves differ. Med Decis Making 1990; 10 (Suppl. 04) 283-7.
• 20 McNeil BJ, Hanley JA. Statistical approaches to the analysis of receiver operating characteristic (ROC) curves. Med Decis Making 1984; 17: 439-48.
• 21 Metz CE. Basic principles of ROC analysis. Sem Nucl Med 1978; 8: 283-98.
• 22 Metz CE. ROC methodology in radiological imaging. Invest Radiol 1986; 21: 720-33.
• 23 Metz CE. Statistical analysis of ROC data in evaluating diagnostic performance. In: Multiple regression analysis: applications in the health sciences. Myers HD. (ed). New York: American Institute of Physics; 1986: 365-84.
• 24 Metz CE. Some practical issues of experimental design and data analysis in radiological ROC studies. Invest Radiol 1989; 24: 234-45.
• 25 Metz CE, Kronman HB. Statistical significance tests for binormal ROC curves. J Math Psych 1980; 22: 218-43.
• 26 Metz CE, Wang P-L, Kronman HB. A new approach for testing the significance of differences between ROC curves measured from correlated data. In: Information processing in medical imaging. Deconninck F. (ed). The Hague: Nijhoff; 1984: 432-45.
• 27 Müller SP, Paas M, Reiners C. ROC-Analyse der Tumorerkennbarkeit mit ^20lTl(I)Chlorid und ^99mTc(I)Hexakis (2-Methoxy-2-Isobuty-lisonitril). Nucl-Med 1994; 33: 15-23.
• 28 Swensson RG, Theodore GH. Search and nonsearch protocols for radiographic consultation. Radiology 1990; 177 (Suppl. 03) 851-6.
• 29 Swets JA. Form of empirical ROCs in discrimination and diagnostic tasks: Implications for theory and measurement of performance. Psych Bull 1986; 99 (Suppl. 02) 181-98.
• 30 Swets JA, Pickett RM. Evaluation of diagnostic systems: Methods from signal detection theory. New York: Academic Press; 1982
• 31 Swets JA, Tanner WP, Birdsall TG. Decision processes in perception. Psychol Rev 1961; 68: 301-40.
• 32 Tosteson ANA, Begg CB. A general regression methodology for ROC curve estimation. Med Decis Making 1988; 8: 204-15.
• 33 Wickens TD. Maximum-likelihood estimation of a multivariate Gaussian rating model with excluded data. J Math Psych 1992; 36: 213-34.