Abstract
Background: Whether the sensitivity of Deep Learning (DL) models to screen chest radiographs
(CXR) for CoVID-19 can approximate that of radiologists, so that they can be adopted
and used if real-time review of CXRs by radiologists is not possible, has not been
explored before. Objective: To evaluate the diagnostic performance of a doctor-trained DL model (Svita_DL8)
to screen for COVID-19 on CXR, and to compare the performance of the DL model with
that of expert radiologists. Materials and Methods: We used a pre-trained convolutional neural network to develop a publicly available
online DL model to evaluate CXR examinations saved in .jpeg or .png format. The initial
model was subsequently curated and trained by an internist and a radiologist using
1062 chest radiographs to classify a submitted CXR as either normal, COVID-19, or
a non-COVID-19 abnormal. For validation, we collected a separate set of 430 CXR examinations
from numerous publicly available datasets from 10 different countries, case presentations,
and two hospital repositories. These examinations were assessed for COVID-19 by the
DL model and by two independent radiologists. Diagnostic performance was compared
between the model and the radiologists and the correlation coefficient calculated.
Results: For detecting COVID-19 on CXR, our DL model demonstrated sensitivity of 91.5%, specificity
of 55.3%, PPV 60.9%, NPV 77.9%, accuracy 70.1%, and AUC 0.73 (95% CI: 0.86, 0.95).
There was a significant correlation (r = 0.617, P = 0.000) between the results of the DL model and the radiologists’ interpretations.
The sensitivity of the radiologists is 96% and their overall diagnostic accuracy is
90% in this study. Conclusions: The DL model demonstrated high sensitivity for detecting COVID-19 on CXR. Clinical Impact: The doctor trained DL tool Svita_DL8 can be used in resource-constrained settings
to quickly triage patients with suspected COVID-19 for further in-depth review and
testing.
Keywords
Artificial intelligence - COVID 19 - CXR - deep learning
