Rofo 2013; 185(7): 635-643
DOI: 10.1055/s-0033-1335220
Qualität/Qualitätssicherung
© Georg Thieme Verlag KG Stuttgart · New York

Investigation of the Performance of Digital Mammographic X-Ray Equipment: Determination of Noise Equivalent Quanta (NEQQC) and Detective Quantum Efficiency (DQEQC) Compared with the Automated Analysis of CDMAM Test Images with CDCOM and CDIC Programs

Untersuchung zur Leistungsfähigkeit digitaler Mammografie-Einrichtungen: Bestimmung der rauschäquivalenten Quantenzahl (NEQQC) und der detektiven Quanteneffizienz (DQEQC) im Vergleich zu automatisierten Auswertungen von CDMAM-Prüfkörperaufnahmen mit den Programmen CDCOM und CDIC
C. Loos
,
H. Buhr
,
C. Blendl
Further Information

Correspondence

Clemens Loos
Institut für Medien- und Phototechnik, Fachhochschule Köln
Betzdorfer Str. 2
50679 Köln
Germany   
Phone: +49/1 79/9 02 62 36   
Fax: +49/2 21/82 75 25 11   

Publication History

25 August 2011

14 February 2013

Publication Date:
25 June 2013 (online)

 

Abstract

Purpose: The purpose of this study was to determine the values for noise equivalent quanta, detective quantum efficiency, modulation transfer function, noise power spectrum, and the values for the parameters for automated CDMAM test phantom analyses required to achieve satisfactory quality of digital mammograms.

Materials and Methods: During the course of tests according to PAS 1054 (8 CR and 12 DR systems), test images were made with a test phantom insertion plate containing two lead edges in nearly horizontal and vertical directions. Only original data were processed with a program that was developed at the Cologne University of Applied Sciences (FH-Köln). All equipment systems complied with the requirements regarding visual recognition of gold-plated mammo detail test objects. CDMAM test images were also evaluated using the CDIC (CUAS) and CDCOM (EUREF) programs.

Results: CDMAM test images show comparable values for the parameters, precision, sensitivity and specificity. DR systems require about half the dose used for CR systems for similar results. The NEQ values achieved with the dose used for the CDMAM test images show larger scatter ranges. The MTF of the different equipment system types differ significantly from each other.

Conclusion: Visual evaluation of CDMAM test images can be replaced by automated evaluation. Limiting values were determined for each parameter. Automated evaluation of CDMAM test phantom images should be used to determine the physical parameter NEQQC. This method is much more sensitive to noise and sharpness influences and has a higher validity than diagnostic methods. Automated evaluation objectivizes testing.


#

Zusammenfassung

Ziel: Es sollte ermittelt werden, welche Werte für die Rauschäquivalente Quantenzahl, die detektive Quanteneffizienz, die Modulationsübertragungsfunktion, das Rauschleistungsspektrum und für die Kenngrößen der automatischen CDMAM Prüfkörperanalysen einzuhalten sind, damit eine ausreichende Qualität von digitalen Mammogrammen erreicht werden kann.

Material und Methoden: Im Rahmen von Prüfungen nach PAS 1054 (8 CR- und 12 DR-Systeme) wurden Aufnahmen mit einem Prüfkörpereinsatz gemacht, der zwei Bleikanten in nahezu horizontaler und vertikaler Richtung enthält. Es wurden nur Originaldaten mit einem an der FH-Köln entwickelten Programm verarbeitet. Alle Einrichtungen erfüllten die Anforderungen an die visuelle Erkennbarkeit von Goldplättchen. Die CDMAM Aufnahmen wurden zudem mit dem Programm CDIC (FH-Köln) und CDCOM (EUREF) ausgewertet.

Ergebnisse: Die CDMAM Aufnahmen zeigen vergleichbare Werte für die Kenngrößen Genauigkeit, Sensitivität und Spezifität. Für vergleichbare Ergebnisse benötigen DR-Systeme etwa halb so viel Dosis wie CR-Systeme. Die Werte der NEQ bei der Dosis, mit der die CDMAM-Aufnahmen angefertigt wurden zeigen größere Streubreiten. Die unterschiedlichen Systemtypen haben eine MÜF, die sich signifikant voneinander unterscheiden.

Schlussfolgerungen: Die visuelle Auswertung von CDMAM Aufnahmen lässt sich durch eine maschinelle ersetzen. Grenzwerte für die einzelnen Kenngrößen wurden ermittelt. Die Bestimmung der physikalischen Kenngröße NEQQC ist der automatischen Auswertung von CDMAM Prüfkörperaufnahmen vorzuziehen. Sie ist weit sensitiver gegenüber Rausch- und Schärfe-Einflüssen und besitzt eine höhere Validität als diagnostische Verfahren. Prüfungen werden durch eine automatische Auswertung objektiviert.


#

Introduction

Up to now, automated methods may not be used to evaluate mammographic test phantom images according to the regulations for acceptance and constancy tests [1] [2] [3] [4]. The MammoControl DIANA software developed by the Reference Center for Mammography in Muenster for mammographic screening already offers a platform for automated, centralized quality control [5] [6]. This software enables, in principle, both the transmission of measurement values and test phantom images as well as automated evaluation. Various analysis algorithms can be implemented due to the modular structure of the software. The software gives immediate feedback to the operator. Preparations for automated evaluation of test phantom images in mammography screening are far advanced [7].

The constancy test for screening including a visual evaluation of CDMAM test phantom images must be conducted on an annual basis (in the field of curative mammography every two years). This image quality test that is decisive in obtaining/extending approval to conduct patient examinations has the shortcoming that the foreseen analysis by three viewers contradicts ICRU recommendations [8] due to the lack of randomization and learning effects as well as the switching back and forth between recognizing circular test objects and noise analysis. The test objects themselves can cause significant variations in the detection rate. There is reasonable doubt whether this method offers adequate accuracy and whether it is suitable for this application. A significant change in the detection rate can only be observed if the dose is increased by a factor of 2 [9]. The CDCOM program [10] offered by EUREF [11] does not meet the requirements for a diagnostic method as it does not provide values for specificity and detection accuracy nor has the detection process been clearly described. It is a noise analysis-based method that enables further processing of its primary detection rates in various ways. An alternative method, the freely available CDIC [12], is a diagnostic method, and its detection method has been clearly described [13]. No limiting values that could be used for image quality testing are available for either of these methods. Nevertheless, the phantom was used in other studies to compare image quality [14].

Therefore, comparative examinations must be made with equipment systems that comply with the existing PAS 1054 and statutory requirements (CDMAM) to determine and compare values in order to extrapolate limiting values that were obtained with different methods. Physical methods provide values based on the IEC Standard [15] for the modulation transfer function (MTF), noise power spectrum (NPS), noise equivalent quanta (NEQ) and detective quantum efficiency (DQE). Diagnostic methods provide parameters such as sensitivity, specificity and accuracy [16] [17].


#

Materials and Methods

Additional test phantom images were made during the course of the annual constancy tests of digital mammographic X-ray equipment according to PAS 1054 (with 8 CR and 12 DR detector systems; the tests with the Slanted Edge insert started later. This resulted in a weak data basis for the physical methods in respect to the CDMAM analysis). The tests according to PAS 1054 and the visual evaluation of the CDMAM test phantom images were conducted by employees of the Prüfstelle für Strahlenschutz (pfs) (a private German Test Center for Radiation Protection). A test plate for the PAS 1054 test phantom was developed for these additional exposures. It contains two lead edges that are imaged with a slight tilt toward the detector matrix (ROI – Region of Interest, of 10 × 30 mm). The test phantom used is described in detail in [9]. When the test phantom is correctly positioned, the angle to the edge of the Bucky table on the chest wall side equals about 3°. To determine the characteristic curve, a series of exposures was made with different tube loads [mAs] and the target-filter combination normally used for patient exposures together with a “manual exposure technique” with each of the tested digital mammographic systems. The field that does not attenuate the primary radiation was analyzed. The estimated detector dose was used as the corresponding dose; the entrance surface air kerma (ESAK; without backscattering) on the test phantom was corrected by the extended distance to the detector as well as with a Bucky factor of 2. Depending on the detector type, a curve was fitted through the data points with a linear, logarithmic or potential function. Prior to further evaluation, all images were transformed into a “dose image,” that is, linearized.

In addition to [18], a window function (Hanning Window [19]) was applied to the central ROI (see below). The MTF-INDEX is determined as the average value of the frequency range from 0 lp/mm to the limiting frequency of the detector. The center of the test plate, 6 cm away from the chest wall with the test phantom positioned in the middle of the Bucky table, is exposed by the central X-ray beam. This area of 30 × 30 [mm] is used to determine the NPSQC. A flat field correction (polynomial fit) is applied to the NPSQC ROI while maintaining the average value. The 2-dimensional noise spectrum is calculated according to [20]. To create a 1-dimensional display, the NPSQC values of the same frequency (with the same radius in the 2-D spectrum, center = Frequency 0) are averaged. The median of all NPSQC values was used for the NPS-INDEX to dampen the influence of fixed pattern noise (FPN) that primarily occurs in the lower frequency ranges.

The terms and definitions of IEC, like DQE, NPS, MTF and NEQ are indexed with QC as “Quality Control” to stress on the one hand the methodical closeness and on the other hand the existence of some minor, experimentally based differences to establish these values.

Based on the DQE measurement, a modified formula according to [20] was used to determine the detective quanta efficiency [DQEQC]: []

Zoom Image

with the MTF-Index of the respective better MTF curve (usually horizontal to the chest wall side), ESAKPMMA is the dose calculated using the characteristic curve from the average pixel value in the ROI to determine the NPS, and Φin is the interpolated quanta fluence ([Table 1]) and the NPS-Index is the median of the NPSQC values across the frequency range under review.

Table 1

The values for the quanta flux, which are presented in the table, are calculated with an open accessible program of Siemens [25] based on the procedure of J. M. Boone [23]. The used coefficients of interaction are sourced by NIST [24]. The calculations consider the tube filtering, 550 mm air in the course of beam, 1 mm PMMA for the paddle, 46 mm PMMA for the test device, 1 mm for the carbon fiber laminate of the bucky table and 1 mm Al for the influence of the grid on the beam quality.
Tab. 1 Die in der Tabelle angegebenen Werte für die Quantenfluenz am Detektoreingang wurden mit einem öffentlich zugänglichen Computerprogramm von SIEMENS [25] berechnet, das auf dem Verfahren von J. M. Boone [23] beruht. Die verwendeten Wechselwirkungskoeffizienten stammen von NIST [24]. Die Berechnungen berücksichtigen die Zusatzfilterung, 550 mm Luft im Strahlengang, 1 mm PMMA für das Paddel, 46 mm PMMA für den Prüfkörper, 1 mm CFK für den Buckytisch (Auflageplatte) und 1 mm Al für den Einfluss auf die Strahlenqualität durch das Raster.

quanta fluence [1/(mm2*mGy)]

tube voltage [kV]

target

filter

filter [µm]

26

28

30

32

34

36

Mo

Mo

32

10,624,633

11,654,676

13,072,705

14,639,916

16,372,435

18,093,313

Mo

Rh

25

11,700,324

12,240,600

13,053,155

14,083,138

15,365,921

16,766,246

Rh

Rh

25

12,405,710

12,945,572

13,557,945

14,313,211

15,157,267

16,056,749

W

Rh

50

12,440,765

12,846,260

13,468,413

14,491,935

15,909,125

17,689,924

W

Ag

50

*

*

*

*

*

*

W

Al

250

13,043,945

14,431,527

15,825,607

17,303,299

18,769,454

20,217,760

W

Al

500

13,210,110

14,632,300

16,068,760

17,595,597

19,106,050

20,594,917

* incalculable by the used program, but required in the future

* mit dem verwendeten Programm nicht berechenbar, aber zukünftig benötigt.

In preparing the reference test image, the dose was measured with a calibrated dosimeter at the position in the phantom provided for this purpose.

The evaluation included only systems that provided access to original data (according to IEC Definition [15], or to DICOM images “for processing” [21], or to raw data). In the case of some CR systems, DICOM images “for presentation” were saved in the constancy testing mode (linear or related designations) without recognizable image processing. Some of the tested CR units deliver original data, but nevertheless they were processed in such a way that establishing the characteristic curve was not possible (Cropping) but the evaluation of the CDMAM images was. The requirements for visual recognition of gold-plated mammo detail test objects in the CDMAM test images had to be fulfilled, and the responsible radiologist had to agree with the preparation of the test images.

Automated Evaluation Methods

All test images were evaluated with the help of an internally developed batch processing program in which the three methods were integrated: CDCOM (Version 1.5.2), CDIC (Version 3.10) and SE (Slanted Edge). Certain tags from the DICOM File Meta Information (header) had to be used during processing. The headers of the individual test images were supplemented as needed whenever tags had not been automatically filled in. The characteristic curve, pixel value versus detector dose, was automatically determined from the “mAs series.” The parameters of the linear smoothing functions (linear or logarithmic equations and/or potential function (y = a*xb)) were recorded in a database. These values are needed for the dose linearization of the original data.

The evaluation method described in [9] for the SE method was modified. Instead of using the average glandular dose (AGD) to replace the quanta fluence (Φin) in the detector surface, the quanta fluence at the detector was estimated. Normally, Φin is determined using a dose and an aluminum half value layer (Al HVL) measurement at the detector. However, such measurements cannot be made on site due to a lack of access, at least with DR systems. As a result, Φin was estimated as follows: The entrance surface air kerma (ESAK) was corrected by the extended distance to the detector (inverse square law, specification of the focal spot-detector distance in the manufacturer’s documentation, or, in the case of a lack of access, a fixed value of 65 cm) and an attenuation factor of two was assumed for the Bucky table (see [22]) and grid attenuation. This factor represents an upper limit and most mammographic X-ray equipment exhibits an effective value that is very close to this value. The error or measurement uncertainty that occurs due to a possibly overestimated attenuation or variation between different systems is probably only in the range of a few percent and is small compared with the uncertainty of the dose measurement. A pixel value is determined in the open area of the PAS 1054 test phantom that is correlated with a detector dose ESAKDet.. According to Boone [23], the quanta fluence can be calculated from the detector dose and the manufacturer’s specifications regarding target-filter combination and tube voltage. That means that by determining the MTFQC, the NPSQC and the Φin, a value can be calculated that roughly approximates the DQE. The value determined in this way is designated as system power (DQEQC) to differentiate it from the DQE according to IEC [15]. All physical parameters are marked with the index QC, for Quality Control, to differentiate them from the IEC parameters: DQEQC, NEQQC etc. A correction of the fixed pattern noise to compensate for low frequency interference as required by IEC cannot be made due to the limited number of images made and the corresponding area.

The system power at discrete frequencies (1, 2, 3, 4, 5, … lp/mm) according to IEC [15] was also calculated.


#
#

Results

Diagnostic Methods

All important data are summarized in [Table 2]. As shown in [Fig. 1a], the values for sensitivity, specificity and accuracy that were determined with the CDIC program are all at about the same level regardless of the equipment system used. DR systems exhibit the trend that a higher dose leads to an increase in the 3 parameters. The average values for the three parameters cannot be statistically separated between the DR and CR systems as the uncertainty ranges overlap. It is striking that the standard deviation for the sensitivity, the suitability of the method to detect circular-shaped structures, is significantly greater than for the other two parameters: 13 to about 2 %. With about 10 analyzed CDMAM test images, the uncertainty of the average value of the specificity and accuracy parameters is so small that the plateau value is almost reached. However, the uncertainty of the average value of the sensitivity parameter remains at a high level regardless of the number of averaged test images. The average values reduced by the standard deviation (with a confidence interval of 1σ) may be used as limiting values for acceptance tests. Based on the above, the following values apply when using the CDIC program: sensitivity 24, specificity 93, and accuracy 66. These values can be achieved if the dose used with CR systems is approximately double the dose used with DR systems: ca. 6:12 and if the entrance surface air kerma (ESAK) of DR systems amounts to at least 3 mGy.

Table 2

Results of the analysis with physical and diagnostic methods of the systems under investigation.
Tab. 2 Ergebnisse der Analyse mit physikalischen und diagnostischen Verfahren bei den untersuchten Systemen.

system

MTFvertical

MTFhoriz.

DQEQC

NEQQC

diagnostic methods

reader

generator

type

U [kV]

target

filter

MTF-Index

σ %

MTF-Index

σ %

SL

σ (%)

intercept

slope

ESAKDet. [mGy]

for NEQQC

370 000

NEQQC for

CDMAM

expos.

ESAKDet. [mGy]

for CDMAM expos.

sensitivity

specificity

accuracy

S CDCOM

PMS: PCR Eleva

PMS System

CR

26

Mo

Rh

0.22

0.0

0.26

0.0

0.09

0.0

 4,288

4,4450

8.23

251,874

5.57

Agfa CR 85

Planmed Sophie Nuance

CR

28

Mo

Rh

0.22

0.7

0.26

0.6

0.12

9.3

69,564

76,757

3.91

391,650

4.20

28.6

94.9

68.4

68.1

Agfa CR 85

No data

CR

28

Mo

Mo

0.23

1.8

0.28

2.3

0.14

1.4

959

106,156

3.48

Agfa CR 35

GE Senographe DR

CR

28

Mo

Rh

0.24

0.5

0.28

0.4

0.12

8.5

61,485

63,664

4.85

367,560

4.81

31.7

95.0

69.6

73.0

Agfa CR 85

SAG Mammomat 3000

CR

27

Mo

Rh

0.25

1.0

0.26

0.8

0.11

13.9

91,905

49,610

5.61

344,918

5.10

31.0

95.4

69.6

73.6

Agfa CR 85

No data

CR

28

Mo

Rh

0.25

1.2

0.27

0.8

0.10

8.0

25,017

48,474

7.12

Agfa DX M

SAG Mammomat 3000

CR

27

Mo

Rh

0.27

0.4

0.29

0.3

0.29

3.2

36,134

132,880

2.51

616,716

4.37

42.8

94.8

74.0

71.4

Agfa DX M

GE DMR

CR

26

Mo

Rh

0.27

0.6

0.34

0.4

0.33

2.1

49,723

158,743

2.02

average CR Systems

0.24

0.9

0.27

0.8

0.19

5.8

4.30

394,543

4.92

30.42

95.09

69.22

71.56

standard deviation

0.03

0.03

0.11

standard Dev. (%)

39

34

12

5

0

1

4

modality

Planmed SophieNuance

DR

28

Mo

Mo

0.66

0.2

0.70

0.4

0.17

10.7

–57,317

76,330

5.60

SAG Mammomat Novation DR

DR

28

Mo

Mo

0.70

0.3

0.70

1.1

0.10

41.9

–92,006

50,093

9.22

Planmed SophieNuance

DR

28

Mo

Mo

0.66

0.3

0.70

0.4

0.17

7.3

–57,317

76,330

5.60

SAG Mammomat Inspiration

DR

26

Mo

Mo

0.68

0.1

0.74

0.2

0.11

5.4

–202,015

98,991

5.78

SAG Mammomat Inspiration

DR

28

Mo

Mo

0.69

0.3

0.74

0.2

0.11

8.1

–190,001

116,416

4.81

SAG Mammomat Inspiration

DR

30

Mo

Mo

0.69

0.6

0.74

0.2

0.09

3.7

–221,958

237,248

2.50

GE Senographe 2000

DR

28

Rh

Rh

0.57

4.2

0.60

2.5

0.13

43.0

–198,254

128,994

4.41

117,782

2.45

40.1

95.3

73.3

76.9

GE Senographe 2000 D

DR

28

Rh

Rh

0.57

4.2

0.57

0.2

0.16

18.9

–232,713

179,099

 3.37

GE Senographe Essential

DR

26

Rh

Rh

0.60

0.60

0.12

*

GE Senographe Essential

DR

28

Rh

Rh

0.65

0.0

0.65

7.2

0.17

29.1

–235,693

138,048

 4.39

GE Senographe Essential

DR

30

Rh

Rh

0.64

0.63

n. b.

0.19

*

GE Senographe Essential

DR

32

Rh

Rh

0.67

0.65

n. b.

0.18

*

Sectra Imtec AB

DR

35

W

Al

0.39

2.1

0.49

0.3

0.13

 0.005

161,845

579,948

 0.36

1,381,074

2.10

38.9

97.0

73.8

80.2

SAG Mammomat Novation DR

DR

28

W

Rh

0.70

0.2

0.73

0.5

0.22

16.4

30,878

82,154

 4.13

252,063

2.69

35.2

96.7

72.1

72.7

Hologic Selenia Dimensions

DR

28

W

Rh

0.61

0.4

0.60

0.8

0.10

31.2

–108,991

113,216

 4.23

SAG Mammomat Inspiration

DR

29

W

Rh

0.66

1.3

0.72

6.6

0.19

31.9

–159,409

226,354

 2.34

317,675

2.11

29.9

96.8

70.1

77.6

PMS Mammodiagnost DR

DR

26

W

Rh

0.68

0.4

0.74

0.2

0.31

 3.4

–78,637

287,033

 1.56

PMS Mammodiagnost DR

DR

28

W

Rh

0.68

0.0

0.74

0.0

0.30

 4.8

–53,542

303,226

 1.40

774,498

2.73

35.5

95.5

71.5

78.2

PMS Mammodiagnost DR

DR

30

W

Rh

0.68

0.0

0.74

0.0

0.29

 2.9

–91,140

398,704

 1.16

average DR Systems*

0.66

0.9

0.67

1.3

0.18

16.4

 3.64

365,504*

2.50

35.9

96.3

72.1

77.1

standard deviation

0.04

0.07

0.07

standard Dev. (%)

59

      78

11.5

3.98

0.81

1.47

2.75

*without Sectra

*without Sectra

* Due to preprocessing not determinable

* Aufgrund von Vorverarbeitung nicht bestimmbar

Zoom Image
Fig. 1a Comparison of the results of an automatic evaluation of CDMAM test images of different DR and CR systems, which are yielded with the program CDIC (CUAS), b Comparison of the results of an automatic evaluation of CDMAM test images of different DR and CR systems, which are yielded with the program CDCOM (EUREF).

Abb. 1a Vergleich der Ergebnisse einer automatischen Auswertung von CDMAM Prüfkörperaufnahmen mit dem Programm CDIC (FHK) von unterschiedlichen DR- und CR-Systemen, b Vergleich der Ergebnisse einer automatischen Auswertung von CDMAM Prüfkörperaufnahmen mit dem Programm CDCOM (EUREF) von unterschiedlichen DR- und CR-Systemen.

In evaluations performed with the CDCOM program ([Table 2, ] [Fig. 1b]), DR systems do not exhibit a recognizable dose dependence. The distribution around the average value equals ca. 6 %. A value of 70 should be used as the lower limiting value for sensitivity SCDCOM. There is no need to mathematically adjust the results to visual evaluation methods as such adjustments significantly depend upon the visual evaluation method selected.


#

Physical Methods

Values for NEQQC and DQEQC can be determined with the values measured for MTF-INDEX, NPS-INDEX and dose as well as by using the values for quanta fluence ([Table 1]). The uncertainty in determining the MTF is very small (standard deviation of 6 test images of about 1 %), and the values determined are independent of dose, target-filter combination (TFC) and tube voltage ([Table 3]). Nearly identical values are achieved for the respective detector types of the different equipment systems ([Table 3]). The idea of horizontal and vertical edges that was originally developed for testing CR systems also proved effective in testing the Sectra System in which the MTF characteristic curves differ significantly from one another ([Fig. 2]). The uncertainty in determining the NPSQC was estimated using the standard deviation of the values of a potential fit function over the upper half of the NPSQC spectrum. Depending on the system, relative fluctuations of 2 – 5 % of the polynomial fit can be observed. The uncertainty of the value for NPS-INDEX can be reduced to under 1 % by using the quantity of up to one hundred interpolation points. The greatest uncertainty in determining detective quanta efficiency DQEQC results from the dose measurement. The uncertainty from the calibration and the determination of corrective factors for different radiation qualities (TFC, Al HVL, X-ray tube voltage U) dominate the overall uncertainty of the method. The estimation of input fluence is subject to slighter uncertainties but is primarily dominated by PMMA filtration. Once NEQ and dose are linked linearly with each other, the sensitivity to changes in dose or detector sensitivity lies in a range of just a few percentage points.

Table 3

Results of the determination of the MTF-Index with respect to different tube loadings and voltages on the system: MammoDiagnostDR (manufacturer: Philips Medical Systems).
Tab. 3 Darstellung der Unabhängigkeit der Werte für den MÜFvert.-Index von unterschiedlichen Spannungen und Ladungen am Beispiel des MammoDiagnostDR (Hersteller: Phillips Medical Systems).

Tube loading Q

Tube voltage U

ESAK

MTFvert.-Index

[mAs]

[kVp]

[mGy]

 

71

28

2.16

0.744

80

28

2.43

0.740

90

28

2.74

0.737

100

28

3.04

0.739

110

28

3.34

0.737

125

28

3.8

0.737

125

26

3.07

0.738

140

26

3.44

0.742

160

26

3.93

0.744

80

30

2.61

0.738

90

30

2.93

0.734

100

30

3.26

0.737

 

 

average

0.739

 

 

standard dev.

0.003

 

 

standard dev.%

0.41

 

 

uncertainty U

0.001

 

 

U%

0.12

Zoom Image
Fig. 2 Vertical and horizontal MTF of the Sectra System as an example for an asymmetric MTF of a DR system.

Abb. 2 Vertikale und horizontale MTF des Sectra Systems als ein Beispiel für eine asymetrische MTF eines DR Systems.

The values of the physical parameters of the CR systems exhibit very similar values, especially the NEQQC value. The values of the Agfa DM X System with “Needle Phosphor” technology are also very close to the other CR results when one considers that the system has a significantly better DQE and is operated with entrance doses that are used with “Powder Phosphor” systems. The DR systems exhibit wider ranges; the Sectra System should be retested due to the suspicion of image processing (see also asymmetric MTF results); the GE Senographe 2000 seems to be an example of an underexposed DR System.

The minimum limiting value for NEQQC that must be achieved is approximately 370,000. This value is not an averaged value from measured data, but rather it’s a preliminary fixed one under special respect of the values of CR systems. Some DR systems showed some irregularities which could not be explained without new test series and a broader data base. If, in addition, the detector-specific values for MTF-INDEX are achieved, one can be sure that the equipment systems will produce diagnostic-level image quality regardless of the respective detector technology.


#
#

Discussion

Automated analysis of test phantom images must fulfill three criteria: objectivity, reliability and validity. Automated analysis provides objectivity for all physical methods. Reliability is also ensured as uniform threshold values, algorithms, etc. are used. Validity is achieved by determining the dose (in line with IEC written as d), the MTFQC and the NPSQC as they are clearly linked with the DQEQC and the NEQQC:[]

Zoom Image

with d2 = Signal (mGy)

The first two criteria are fulfilled by diagnostic methods. The third criteria can only be determined by comparison with physical methods.

In performing automated evaluation, the tags in the DICOM header should be used to accelerate the method. Missing or incorrect tags significantly disrupt automated evaluation. We determined during this study that one cannot assume that tags are complete and correct and that possible tag entries in certain fields according to DICOM do not always lead to a logically clear classification of the systems: e. g. tag 0008;0060 – Modality. In this case, one cannot clearly determine whether a DR or a CR system is involved.

The manufacturers’ claim of “DICOM conformity” for all of the equipment systems does not ensure that automated evaluation is possible. We also determined at the same time that the term “original images” definitely has different meanings. For instance, operations such as histogram cropping are performed making the plotting of a characteristic curve impossible and preventing further physical analyses.

To enable testing of the physical quality of the detector, all manufacturers must comply with the IEC requirements for original images and the DICOM standard for header tags. The EUREF Group will include the requirement in the new supplement (to be published probably by year 2011) that the original images must be made available.

Diagnostic Methods

Automated diagnostic methods exclude the influence of the viewer and have high reproducibility. However, the values for standard deviation in evaluating different images produced with the same exposure parameters and slight shifts in phantom position are finite: for sensitivity with CDIC about 10 % and for accuracy about 1 – 2 % and about 5 % with CDCOM. This requires that a sufficiently large number of CDMAM test images must be made to ensure that the uncertainty of the result remains at a reasonable level: ca. 10 test images also for automated evaluation. All three parameters, the dose measurement, the phantom and the method influence the accuracy, that is, influence the cross-comparability of the results. It should be noted that there are no systematic studies about how diagnostic methods using different detection methods react to interference (FPN and/or TN, MTF changes, etc.), i. e., about the validity of these methods. See [Fig. 3] for two sample results of CDIC in which the results of an analysis are presented: gray shaded circles represent the true positions of gold discs (type 3.14). Colored dots – blue in the center and red in corners – represent findings (TP and FP).

Zoom Image
Fig. 3 Results of CDIC, a Agfa CR 85 Planmed Sophie Nuance, b Siemens Mammomat Novation DR.

Abb. 3 Ergebnisse von CDIC, a Agfa CR 85 Planmed Sophie Nuance, b Siemens Mammomat Novation DR.

The EPQC method uses two different limiting values. The ratings “acceptable” and “achievable” should be used with diagnostic methods. “Achievable” should correspond with the average value and “acceptable” with the average value reduced by the standard deviation.


#

Physical Methods

Characteristic curve: We determined that when using standard exposure parameters for the CDMAM test image, the open area in the aluminum step wedge region was frequently overexposed and could not be used for the evaluation. Therefore, the exposure parameters for the CDMAM test image should be used as maximum values, and subsequent test images be made with a lower dose.

Determination of the MTFQC or the NPSQC does not dominate the uncertainty of NEQ QC, but rather the uncertainty of the dose measurement is the limiting factor. The value of NEQQC is a required parameter to describe detector-related image quality, but has to be completed by an additional value which gives information regarding the imaged dynamic range of glandular tissue respective to the contrast in the image. A specific test procedure is under development.

Test equipment: The edge can be produced with an accuracy that fulfills the IEC requirements (5 – 8 µm according to manufacturer's data from PEHA med. Geräte GmbH, Sulzbach). That means that the edge testing device does not notably influence the test result.

This means in practice that as a rule, only a few test images that can be used for the evaluation must be made and that they are very dose-sensitive. As modern generators and dosimeters have very high reproducibility, a deviation of just a few percentage points in NEQQC can indicate that the ROI was not correctly positioned when evaluating the image. In such cases, additional test images must be made to exclude incorrect measurements.


#

Cross-Comparability

Theoretically, one would expect that systems that fulfill the requirements for image quality and dose would exhibit a close correlation between the dose required to produce CDMAM test images and the NEQQC calculated using the same dose. However, as the CDMAM test is not particularly dose-sensitive, one can only expect a rough correlation between the dose required for a certain NEQQC and the dose with which the CDMAM test images were made. This expectation was confirmed ([Fig. 4]). One may assume that a minimum NEQQC value of 370,000 must be achieved.

Zoom Image
Fig. 4  Comparison of NEQQC yielded with the dose of CDMAM exposures with the results of diagnostic methods (Sectra results included).

Abb. 4 Vergleich der NEQQC die mit der Dosis erzielt wird, mit der die CDMAM Aufnahmen angefertigt wurden mit den Ergebnissen der diagnostischen Verfahren (Sectra Ergebnisse eingeschlossen).

#
#
#
  • References

  • 1 Blendl C, Mertelmeier T, Hermann KP. PAS 1054: Anforderungen und Prüfverfahren für die digitale Mammographie. Berlin: Beuth; 2005
  • 2 Blendl C, Mertelmeier T, Hermann KP. PAS 1054: Anforderungen und Prüfverfahren für die digitale Mammographie. Berlin: Beuth; 2005
  • 3 Bundesministerium für Umwelt, Naturschutz und Reaktorsicherheit. Richtlinie zur Durchführung der Qualitätssicherung bei Röntgeneinrichtungen zur Untersuchung oder Behandlung von Menschen nach den §§ 16 und 17 der Röntgenverordnung – Qualitätssicherungs-Richtlinie (QS-RL). Berlin: 2011
  • 4 Bundesministerium für Umwelt, Naturschutz und Reaktorsicherheit. Richtlinie für die technische Prüfung von Röntgeneinrichtungen und genehmigungsbedürftigen Störstrahlern – Sachverständigen-Prüfrichtlinie (SV-RL) –. Berlin: 2011
  • 5 Sommer A, Girnus R, Wendt B et al. Physical and technical quality assurance in German breast cancer screening: progress report of the Reference Center Muenster after three years. Fortschr Röntgenstr 2009; 181: 447-453
  • 6 Sommer A, Girnus R, Wendt B et al. Development of an objective rating system for the annual physical quality control for digital mammography systems. Fortschr Röntgenstr 2010; 182: 788-792
  • 7 Czwoydzinski J, Girnus R, Sommer A et al. Zentrale Online-Qualitätssicherung in der Radiologie: eine IT-Lösung am Beispiel des deutschen Mammografie-Screening-Programms. Fortschr Röntgenstr 2011; 183: 849-854
  • 8 ICRU Report 54. Medical Imaging – The Assessment of Image Quality. Bethesda: (USA); 1996
  • 9 Blendl C, Loos C, Eiben B. Vergleich zweier automatischer Verfahren zur Auswertung von CDMAM-Prüfkörperaufnahmen. Fortschr Röntgenstr 2009; 181: 637-643
  • 10 Karssemeijer N, Thijssen MA O. Determination of contrast-detail curves of mammography systems by automated image analysis. Digital Mammography 1996; 96: 155-160
  • 11 EUREF/ EPQC. European protocol for the quality control of the physical and technical aspects of mammography screening. Addendum Digital Mammography (ADM). Brussels: European commission; 2006
  • 12 Clemens Loos. CDIC: CDMAM Image Checker – CDIC (Version 3.10). als Download mit vollständigen Javadoc und Bedienungsanleitung verfügbar auf: http://cdic.clemensloos.de (Stand April 2011)
  • 13 Blendl C, Schreiber AC, Buhr H. Ergebnisse einer automatischen Auswertung von Prüfkörperaufnahmen nach PAS 1054 und IEC 62220-1-2 an unterschiedlichen Typen digitaler Mammografie-Röntgeneinrichtungen. Fortschr Röntgenstr 2009; 181: 979-988
  • 14 Schreiter NF, Steffen IG, Miller J et al. Qualitative JPEG 2000 compression in digital mammography – evaluation using 480 mammograms of the CDMAM phantom. Fortschr Röntgstr 2011; 183: 650-657
  • 15 International Electrotechnical Commission. Determination of the detective quantum efficiency–detectors used in mammography. IEC 62220–1–2, 2007
  • 16 Himme A. Gütekriterien der Messung: Reliabilität, Validität und Generalisierbarkeit. In: Albers S, Klapper D, Konradt U, et al. (eds). Methodik der empirischen Forschung. Wiesbaden: Gabler; 2007: 375-379
  • 17 Bortz J, Döring N. Forschungsmethoden und Evaluation. Berlin: Springer; 2006: 326-327
  • 18 Cunningham IA, Fenster A. A method for modulation transfer function determination from edge profiles with correction for finite-element differentiation. Med Phys 1987; 14: 533-537
  • 19 Burger W, Burge MJ. Digitale Bildverarbeitung: Eine Einführung mit Java und ImageJ. Berlin: Springer; 2006: 342-343
  • 20 Beutel J, Kundel HL, Van Metter RL (eds). Handbook of Medical Imaging. Bellingham: SPIE Press; 2000: 110-112
  • 21 National Electrical Manufacturers Association. Digital Imaging and Communications in Medicine (DICOM). Rosslyn, Virginia (USA): 2009
  • 22 International ElectrotechnicalCommission. Particular requirements for basic safety and essential performance of mammographic X-ray equipment and mammographic stereotactic devices. IEC 60601-2-45 2011
  • 23 Boone JM, Fewell TR, Jennings RJ. Molybdenum, rhodium, and tungsten anode spectral models using interpolating polynomials with application to mammography. Med Phys 1997; 24: 1863-1874
  • 24 National Institute of Standards and Technology. Physical Measurement Laboratory. http://www.nist.gov/physlab (Stand April 2011)
  • 25 Siemens. X-ray Toolbox. https://w9.siemens.com/cms/oemproducts/Home/X-rayToolbox/spektrum/Pages/MamIn.aspx (Stand April 2011)

Correspondence

Clemens Loos
Institut für Medien- und Phototechnik, Fachhochschule Köln
Betzdorfer Str. 2
50679 Köln
Germany   
Phone: +49/1 79/9 02 62 36   
Fax: +49/2 21/82 75 25 11   

  • References

  • 1 Blendl C, Mertelmeier T, Hermann KP. PAS 1054: Anforderungen und Prüfverfahren für die digitale Mammographie. Berlin: Beuth; 2005
  • 2 Blendl C, Mertelmeier T, Hermann KP. PAS 1054: Anforderungen und Prüfverfahren für die digitale Mammographie. Berlin: Beuth; 2005
  • 3 Bundesministerium für Umwelt, Naturschutz und Reaktorsicherheit. Richtlinie zur Durchführung der Qualitätssicherung bei Röntgeneinrichtungen zur Untersuchung oder Behandlung von Menschen nach den §§ 16 und 17 der Röntgenverordnung – Qualitätssicherungs-Richtlinie (QS-RL). Berlin: 2011
  • 4 Bundesministerium für Umwelt, Naturschutz und Reaktorsicherheit. Richtlinie für die technische Prüfung von Röntgeneinrichtungen und genehmigungsbedürftigen Störstrahlern – Sachverständigen-Prüfrichtlinie (SV-RL) –. Berlin: 2011
  • 5 Sommer A, Girnus R, Wendt B et al. Physical and technical quality assurance in German breast cancer screening: progress report of the Reference Center Muenster after three years. Fortschr Röntgenstr 2009; 181: 447-453
  • 6 Sommer A, Girnus R, Wendt B et al. Development of an objective rating system for the annual physical quality control for digital mammography systems. Fortschr Röntgenstr 2010; 182: 788-792
  • 7 Czwoydzinski J, Girnus R, Sommer A et al. Zentrale Online-Qualitätssicherung in der Radiologie: eine IT-Lösung am Beispiel des deutschen Mammografie-Screening-Programms. Fortschr Röntgenstr 2011; 183: 849-854
  • 8 ICRU Report 54. Medical Imaging – The Assessment of Image Quality. Bethesda: (USA); 1996
  • 9 Blendl C, Loos C, Eiben B. Vergleich zweier automatischer Verfahren zur Auswertung von CDMAM-Prüfkörperaufnahmen. Fortschr Röntgenstr 2009; 181: 637-643
  • 10 Karssemeijer N, Thijssen MA O. Determination of contrast-detail curves of mammography systems by automated image analysis. Digital Mammography 1996; 96: 155-160
  • 11 EUREF/ EPQC. European protocol for the quality control of the physical and technical aspects of mammography screening. Addendum Digital Mammography (ADM). Brussels: European commission; 2006
  • 12 Clemens Loos. CDIC: CDMAM Image Checker – CDIC (Version 3.10). als Download mit vollständigen Javadoc und Bedienungsanleitung verfügbar auf: http://cdic.clemensloos.de (Stand April 2011)
  • 13 Blendl C, Schreiber AC, Buhr H. Ergebnisse einer automatischen Auswertung von Prüfkörperaufnahmen nach PAS 1054 und IEC 62220-1-2 an unterschiedlichen Typen digitaler Mammografie-Röntgeneinrichtungen. Fortschr Röntgenstr 2009; 181: 979-988
  • 14 Schreiter NF, Steffen IG, Miller J et al. Qualitative JPEG 2000 compression in digital mammography – evaluation using 480 mammograms of the CDMAM phantom. Fortschr Röntgstr 2011; 183: 650-657
  • 15 International Electrotechnical Commission. Determination of the detective quantum efficiency–detectors used in mammography. IEC 62220–1–2, 2007
  • 16 Himme A. Gütekriterien der Messung: Reliabilität, Validität und Generalisierbarkeit. In: Albers S, Klapper D, Konradt U, et al. (eds). Methodik der empirischen Forschung. Wiesbaden: Gabler; 2007: 375-379
  • 17 Bortz J, Döring N. Forschungsmethoden und Evaluation. Berlin: Springer; 2006: 326-327
  • 18 Cunningham IA, Fenster A. A method for modulation transfer function determination from edge profiles with correction for finite-element differentiation. Med Phys 1987; 14: 533-537
  • 19 Burger W, Burge MJ. Digitale Bildverarbeitung: Eine Einführung mit Java und ImageJ. Berlin: Springer; 2006: 342-343
  • 20 Beutel J, Kundel HL, Van Metter RL (eds). Handbook of Medical Imaging. Bellingham: SPIE Press; 2000: 110-112
  • 21 National Electrical Manufacturers Association. Digital Imaging and Communications in Medicine (DICOM). Rosslyn, Virginia (USA): 2009
  • 22 International ElectrotechnicalCommission. Particular requirements for basic safety and essential performance of mammographic X-ray equipment and mammographic stereotactic devices. IEC 60601-2-45 2011
  • 23 Boone JM, Fewell TR, Jennings RJ. Molybdenum, rhodium, and tungsten anode spectral models using interpolating polynomials with application to mammography. Med Phys 1997; 24: 1863-1874
  • 24 National Institute of Standards and Technology. Physical Measurement Laboratory. http://www.nist.gov/physlab (Stand April 2011)
  • 25 Siemens. X-ray Toolbox. https://w9.siemens.com/cms/oemproducts/Home/X-rayToolbox/spektrum/Pages/MamIn.aspx (Stand April 2011)

Zoom Image
Zoom Image
Fig. 1a Comparison of the results of an automatic evaluation of CDMAM test images of different DR and CR systems, which are yielded with the program CDIC (CUAS), b Comparison of the results of an automatic evaluation of CDMAM test images of different DR and CR systems, which are yielded with the program CDCOM (EUREF).

Abb. 1a Vergleich der Ergebnisse einer automatischen Auswertung von CDMAM Prüfkörperaufnahmen mit dem Programm CDIC (FHK) von unterschiedlichen DR- und CR-Systemen, b Vergleich der Ergebnisse einer automatischen Auswertung von CDMAM Prüfkörperaufnahmen mit dem Programm CDCOM (EUREF) von unterschiedlichen DR- und CR-Systemen.
Zoom Image
Fig. 2 Vertical and horizontal MTF of the Sectra System as an example for an asymmetric MTF of a DR system.

Abb. 2 Vertikale und horizontale MTF des Sectra Systems als ein Beispiel für eine asymetrische MTF eines DR Systems.
Zoom Image
Zoom Image
Fig. 3 Results of CDIC, a Agfa CR 85 Planmed Sophie Nuance, b Siemens Mammomat Novation DR.

Abb. 3 Ergebnisse von CDIC, a Agfa CR 85 Planmed Sophie Nuance, b Siemens Mammomat Novation DR.
Zoom Image
Fig. 4  Comparison of NEQQC yielded with the dose of CDMAM exposures with the results of diagnostic methods (Sectra results included).

Abb. 4 Vergleich der NEQQC die mit der Dosis erzielt wird, mit der die CDMAM Aufnahmen angefertigt wurden mit den Ergebnissen der diagnostischen Verfahren (Sectra Ergebnisse eingeschlossen).