Rofo 2021; 193(12): 1436-1444
DOI: 10.1055/a-1515-2923
Chest

Present Limitations of Artificial Intelligence in the Emergency Setting – Performance Study of a Commercial, Computer-Aided Detection Algorithm for Pulmonary Embolism

Article in several languages: English | deutsch
1  Klinik für Diagnostische und Interventionelle Radiologie, Albert-Ludwigs-Universität Freiburg Medizinische Fakultät, Freiburg, Deutschland
,
Lena Kretzschmar
2  Klinik und Poliklinik für Radiologie, Ludwig-Maximilians-Universität, München, Deutschland
,
Giovanna Negrão de Figueiredo
2  Klinik und Poliklinik für Radiologie, Ludwig-Maximilians-Universität, München, Deutschland
,
Alexander Crispin
3  Institut für Medizinische Informationsverarbeitung, Biometrie und Epidemiologie, Klinikum der Universität München-Großhadern, München, Deutschland
,
Robert Stahl
4  Institut für Diagnostische und Interventionelle Neuroradiologie, Klinikum der Universität München-Großhadern, München, Deutschland
,
Fabian Bamberg
1  Klinik für Diagnostische und Interventionelle Radiologie, Albert-Ludwigs-Universität Freiburg Medizinische Fakultät, Freiburg, Deutschland
,
Christoph Gregor Trumm
4  Institut für Diagnostische und Interventionelle Neuroradiologie, Klinikum der Universität München-Großhadern, München, Deutschland
› Author Affiliations

Abstract

Purpose Since artificial intelligence is transitioning from an experimental stage to clinical implementation, the aim of our study was to evaluate the performance of a commercial, computer-aided detection algorithm of computed tomography pulmonary angiograms regarding the presence of pulmonary embolism in the emergency room.

Materials and Methods This retrospective study includes all pulmonary computed tomography angiogram studies performed in a large emergency department over a period of 36 months that were analyzed by two radiologists experienced in emergency radiology to set a reference standard. Original reports and computer-aided detection results were compared regarding the detection of lobar, segmental, and subsegmental pulmonary embolism. All computer-aided detection findings were analyzed concerning the underlying pathology. False-positive findings were correlated to the contrast-to-noise ratio.

Results Expert reading revealed pulmonary embolism in 182 of 1229 patients (49 % men, 10–97 years) with a total of 504 emboli. The computer-aided detection algorithm reported 3331 findings, including 258 (8 %) true-positive findings and 3073 (92 %) false-positive findings. Computer-aided detection analysis showed a sensitivity of 47 % (95 %CI: 33–61 %) on the lobar level and 50 % (95 %CI 43–56 %) on the subsegmental level. On average, there were 2.25 false-positive findings per study (median 2, range 0–25). There was no significant correlation between the number of false-positive findings and the contrast-to-noise ratio (Spearman’s Rank Correlation Coefficient = 0.09). Soft tissue (61.0 %) and pulmonary veins (24.1 %) were the most common underlying reasons for false-positive findings.

Conclusion Applied to a population at a large emergency room, the tested commercial computer-aided detection algorithm faced relevant performance challenges that need to be addressed in future development projects.

Key Points:

  • Computed tomography pulmonary angiograms are frequently acquired in emergency radiology.

  • Computer-aided detection algorithms (CADs) can support image analysis.

  • CADs face challenges regarding false-positive and false-negative findings.

  • Radiologists using CADs need to be aware of these limitations.

  • Further software improvements are necessary ahead of implementation in the daily routine.

Citation Format

  • Müller-Peltzer K, Kretzschmar L, Negrão de Figueiredo G et al. Present Limitations of Artificial Intelligence in the Emergency Setting – Performance Study of a Commercial, Computer-Aided Detection Algorithm for Pulmonary Embolism. Fortschr Röntgenstr 2021; 193: 1436 – 1444



Publication History

Received: 31 December 2020

Accepted: 04 May 2021

Publication Date:
05 August 2021 (online)

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany