Accuracy of Coronary Plaque Detection and Assessment of Interobserver Agreement for Plaque Quantification Using Automatic Coronary Plaque Analysis Software on Coronary CT Angiography

Abstract

Purpose: To evaluate the accuracy of automatic plaque detection and the interobserver agreement of automatic versus manually adjusted quantification of coronary plaques on coronary CT angiography (cCTA) using commercially available software.

Materials and Methods: 10 cCTA datasets were evaluated using plaque software. First, the automatically detected plaques were verified. Second, two observers independently performed plaque quantification without revising the automatically constructed plaque contours (automatic approach). Then, each observer adjusted the plaque contours according to plaque delineation (adjusted approach). The interobserver agreement of both approaches was analyzed.

Results: 32 of 114 automatically identified findings were true-positive plaques, while 82 (72 %) were false-positive. 20 of 52 plaques (38 %) were missed by the software (false-negative). The automatic approach provided good interobserver agreement with relative differences of 0.9 ± 16.0 % for plaque area and –3.3 ± 33.8 % for plaque volume. Both observers independently adjusted all contours because they did not represent the plaque delineation. Interobserver agreement decreased for the adjusted approach with relative differences of 25.0 ± 24.8 % for plaque area and 20.0 ± 40.4 % for plaque volume.

Conclusion: The automatic plaque analysis software is of limited value due to high numbers of false-positive and false-negative plaque findings. The automatic approach was reproducible but it necessitated adjustment of all constructed plaque contours resulting in deterioration of the interobserver agreement.

Key points:

• Automatic plaque detection is limited due to high false-positive and false-negative findings.

• Automatic plaque quantification was reproducible in the few accurately detected plaques.

• The automatically constructed contours did not represent the plaque delineation.

• Both observers independently adjusted the plaque contours.

• Manual adjustment of plaque contours reduced the interobserver agreement.

Citation Format:

• Laqmani A, Klink T, Quitzke M et al. Accuracy of Coronary Plaque Detection and Assessment of Interobserver Agreement for Plaque Quantification Using Automatic Coronary Plaque Analysis Software on Coronary CT Angiography. Fortschr Röntgenstr 2016; 188: 933 – 939

Zusammenfassung

Ziel: Evaluation der Detektionsgenauigkeit und der Interobserver-Übereinstimmung von automatischer gegenüber manuell adjustierter Quantifizierung von Koronarplaques mit der Koronar-CT-Angiografie (cCTA) unter Verwendung einer kommerziell erhältlichen Software.

Material und Methoden: 10 cCTA-Datensätze wurden unter Verwendung einer Plaqueanalyse-Software evaluiert. Als erstes wurden die automatisch detektierten Plaques verifiziert. Danach führten 2 Untersucher unabhängig voneinander die automatische Plaque-Quantifizierung durch ohne die automatisch konstruierten Plaquekonturen zu revidieren (automatische Vorgehensweise). Anschließend konnten die Untersucher die Konturen gemäß der Plaquegrenzen anpassen (adjustierte Vorgehensweise). Die Interobserver-Übereinstimmung beider Vorgehensweisen wurde überprüft.

Ergebnisse: 32 der 114 automatisch identifizierten Befunde waren richtig-positiv, während 82 (72 %) der Befunde falsch-positiv waren. 20 der 52 (38 %) Plaques wurden nicht von der Software detektiert (falsch-negativ). Die automatische Vorgehensweise bot eine gute Interobserver-Übereinstimmung mit relativen Unterschieden von 0,9 ± 16,0 % für Plaquefläche und –3,3 ± 33,8 % für Plaquevolumen. Beide Untersucher passten unabhängig von einander alle Plaquekonturen an, da die automatisch konstruierten Konturen nicht den Plaquegrenzen entsprachen. Die anschließende adjustierte Vorgehensweise verschlechterte die Interobserver-Übereinstimmung mit relativen Unterschieden von 25.0 ± 24.8 % für Plaquefläche und 20,0 ± 40,4 % für Plaquevolumen.

Schlussfolgerung: Die untersuchte Software ist wegen vieler falsch-positiven und falsch-negativen Plaque-Detektionen von begrenztem Nutzen. Die automatische Vorgehensweise lieferte reproduzierbare Ergebnisse, benötigte jedoch eine Adjustierung sämtlicher Plaquekonturen, die zu einer Verschlechterung der Interobserver-Übereinstimmung führte.

Kernaussagen:

• Automatische Plaquedetektion ist durch hohe Raten an falsch-positiven und falsch-negativen Befunden limitiert.

• Bei den wenigen korrekt detektierten Plaques ist die automatische Plaquequantifizierung reproduzierbar.

• Die automatisch konstruierten Konturen entsprachen nicht den Plaquegrenzen.

• Folglich passten beide Untersucher unabhängig von einander die Plaquekonturen an.

• Diese Anpassung der Plaquekonturen führte zu einer Herabsetzung der Interobserver-Übereinstimmung.

Key words

atherosclerotic plaque - coronary computed tomography angiography - coronary artery disease - software tools - automatic quantification

• References

• 1 Taylor AJ, Cerqueira M, Hodgson JM et al. ACCF/SCCT/ACR/AHA/ASE/ASNC/NASCI/SCAI/SCMR 2010 Appropriate Use Criteria for Cardiac Computed Tomography. A Report of the American College of Cardiology Foundation Appropriate Use Criteria Task Force, the Society of Cardiovascular Computed Tomography, the American College of Radiology, the American Heart Association, the American Society of Echocardiography, the American Society of Nuclear Cardiology, the North American Society for Cardiovascular Imaging, the Society for Cardiovascular Angiography and Interventions, and the Society for Cardiovas- cular Magnetic Resonance. Circulation 2010; 122: e525-e555
• 2 Andreini D, Pontone G, Mushtaq S et al. A long-term prognostic value of coronary CT angiography in suspected coronary artery disease. JACC Cardiovasc Imaging 2012; 5: 690-701
• 3 Korosoglou G, Mueller D, Lehrke S et al. Quantitative assessment of stenosis severity and atherosclerotic plaque composition using 256-slice computed tomography. Eur Radiol 2010; 20: 1841-1850
• 4 Hoffmann H, Frieler K, Hamm B et al. Intra- and interobserver variability in detection and assessment of calcified and noncalcified coronary artery plaques using 64-slice computed tomography: variability in coronary plaque measurement using MSCT. Int J Cardiovasc Imaging 2008; 24: 735-742
• 5 Knollmann F, Ducke F, Krist L et al. Quantification of atherosclerotic coronary plaque components by submillimeter computed tomography. Int J Cardiovasc Imaging 2008; 24: 301-310
• 6 Pundziute G, Schuijf JD, Jukema JW et al. Head-to-head comparison of coronary plaque evaluation between multislice computed tomography and intravascular ultrasound radiofrequency data analysis. JACC Cardiovasc Interv 2008; 1: 176-182
• 7 Burgstahler C, Reimann A, Beck T et al. Influence of a lipid-lowering therapy on calcified and noncalcified coronary plaques monitored by multislice detector computed tomography: results of the New Age II Pilot Study. Invest Radiol 2007; 42: 189-195
• 8 Dey D, Schuhbaeck A, Min JK et al. Non-invasive measurement of coronary plaque from coronary CT angiography and its clinical implications. Expert Rev Cardiovasc Ther 2013; 11: 1067-1077
• 9 Lee MS, Chun EJ, Kim KJ et al. Reproducibility in the assessment of noncalcified coronary plaque with 256-slice multi-detector CT and automated plaque analysis software. Int J Cardiovasc Imaging 2010; 26: 237-244
• 10 Busch S, Johnson TR, Nikolaou K et al. Visual and automatic grading of coronary artery stenoses with 64-slice CT angiography in reference to invasive angiography. Eur Radiol 2007; 17: 1445-1451
• 11 Oberoi S, Meinel FG, Schoepf UJ et al. Reproducibility of noncalcified coronary artery plaque burden quantification from coronary CT angiography across different image analysis platforms. Am J Roentgenol 2014; 202: W43-W49
• 12 Versteylen MO, Kietselaer BL, Dagnelie PC et al. Additive value of semiautomated quantification of coronary artery disease using cardiac computed tomographic angiography to predict future acute coronary syndrome. J Am Coll Cardiol 2013; 61: 2296-2305
• 13 Dey D, Schepis T, Marwan M et al. Automated three-dimensional quantification of noncalcified coronary plaque from coronary CT angiography: comparison with intravascular US. Radiology 2010; 257: 516-522
• 14 Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: 307-310
• 15 Groth M, Muellerleile K, Klink T et al. Improved agreement between experienced and inexperienced observers using a standardized evaluation protocol for cardiac volumetry and infarct size measurement. Rofo 2012; 184: 1131-1137
• 16 Karamitsos TD, Hudsmith LE, Selvanayagam JB et al. Operator induced variability in left ventricular measurements with cardiovascular magnetic resonance is improved after training. J Cardiovasc Magn Reson 2007; 9: 777-783
• 17 Klass O, Kleinhans S, Walker MJ et al. Coronary plaque imaging with 256-slice multidetector computed tomography: interobserver variability of volumetric lesion parameters with semiautomatic plaque analysis software. Int J Cardiovasc Imaging 2010; 26: 711-720