On the Use of Pooled Receiver Operating Characteristic Curves in the Evaluation of Scintigrams^[*]

Es wird die Auswertung simulierter Szintigraphiebilder durch Kurven beschrieben, die auf binären Entscheidungen einzelner Beobachter basieren und als “pooled ROC”-Kurven oder “pooled L-ROC”-Kurven bezeichnet werden. Die Methode der Herstellung solcher Kurven wird beschrieben. Beispiele verschiedener Verarbeitungsmethoden und Befunde verschiedener Beobachtergruppen werden verglichen und diskutiert. Zur Auswertung wurden Bilder zweier simulierter Phantome der IAEO verwendet. Die Ergebnisse zeigen, daß die “pooled L-ROC”-Kurven als Maß für die Qualität des Bildes dienen können.

The assessment of simulated scintigraphic pictures by means of curves based on binary decisions of individual observers called pooled ROC-curves or pooled L-ROC-curves is presented. The technique of creating the curve is described. The examples of various processing techniques and the findings of various groups of observers are compared and discussed. The images of two IAEA simulated phantoms were used to perform the evaluation. The results indicate that the pooled L-ROC-curves may serve as a measure of image quality.

^* A portion of this work was done in partial fulfilment of the Research Agreement Nr. 1091/R4/CF of the IAEA

• References

• 1 Goodenough D. J. The use of ROC curves in testing the proficiency of individuals in classifying pneumoconiosis. Radiology 1975; 114: 472.
• 2 Goodenough D. J, Metz C. E, Lusted L. B. Caveat on use of the parameter d’ for evaluation of observer performance. Radiology 1973; 106: 565.
• 3 Goodenough D. J, Rossmann K, Lusted L. B. Radiographic applications of signals detection theory. Radiology 1972; 105: 199.
• 4 Gwiazdowska B. A, Hahn L, Skrzypczak E, Tarkowski W, Tolwiński J. Zastosowanie charakterystyki skuteczności per-cepcji do oceny wykrywalności ognisk w obrazach scyntygra-ficznych. Postępy Fizyki Medycznej 1976; 11: 161.
• 5 Herath K. B, Sharp P. F. Effects of “matched filter” smoothing as measured by Receiver Operating Characteristic curve. Phys. Med. Biol 1976; 21: 442.
• 6 IAEA co-ordinated research programme on the intercompar-ison of computer assisted scintigraphic tecnniques. Medical Radioisotope Scintigraphy 1972. Vienna, 1973; 1: 727.
• 7 IAEA co-ordinated research programme on the intercompar-ison of computer-assisted scintigraphic techniques. Second Progress Report, Medical Radionuclide Imaging 1976, IAEA, Vienna, 1977; 1: 571.
• 8 Kuhl D. E, Sanders T. D, Edwards R. Q, Makler Jr. P. T.. Failure to improve observer performance with scan smoothing. J. nucl. Med 1972; 13: 752.
• 9 Lusted L. B. Decision-making studies in patient management. New Engl. J. Med 1971; 284: 416.
• 10 Lusted L. B. Signal detectability and medical decision making. Science 1971; 171: 1217.
• 11 Metz C. E, Goodenough D. J. On failure to improve observer performance with scan smoothing: a rebuttal. J. nucl. Med 1973; 14: 873.
• 12 Metz C. E, Goodenough D. J, Rossmann K. Evaluation of Receiver Operating Characteristic curve data in terms of information theory, with applications in radiography. Radiology 1973; 109: 297.
• 13 Metz C. E, Starr S. J, Lusted L. B. Quantitative evaluation of visual detection performance in medicine: ROC analysis and determination of diagnostic benefit. Medical Images (Ed. G. A. Day) 220 1977
• 14 Metz C. E, Starr S. J, Lusted L. B, Rossmann K. Progressin evaluation of human observer visual detection performance using ROC curve approach. Information Processing in Scintigraphy. Proceedings of the IVth International Conference, Orsay 1975. Eds Raynaud C., Todd-Pokropek A.. 1975: 420
• 15 Morgan R. H, Donner M. W, Gayler B. W, Margulies S. I, Rao P. S, Weeler P. S. Decision progress and observer error in the diagnosis of pneumoconiosis by chest roentgenography. Amer. J. Roentgenol 1973; 177: 757.
• 16 Starr S. J, Metz C. E, Lusted L. B, Goodenough D. J. Visual detection and localization of radiographic images. Radiology 1975; 116: 533.