Keywords artificial intelligence - machine learning - deep learning - radiology - informatics
It is unlikely that, in 2018, a practicing radiologist has managed to escape the constant
hype of artificial intelligence (AI). Whether it is a prominent AI researcher advocating
for the cessation of radiologist training,[1 ] a president citing radiologist displacement as an example of AI's impact on the
economy,[2 ] or the constant articles in the literature and lay press,[3 ] the prominence of AI is everywhere. So, it may surprise some readers to learn that
references to AI in radiology date back to at least 1994.[4 ] Despite the seemingly daily news regarding the promise of AI and its impending impact
on radiology, true disruption to the practice of radiology has not occurred. Nevertheless,
these technologies have the potential to elicit a paradigm shift within radiology
and medicine as a whole. Unlike many prior technologies that have impacted radiology,
AI may not only alter the interpretation of images but also affect every element of
the clinical workflow of a radiology practice ([Fig. 1 ]).
Fig. 1 Standard radiology workflow and opportunities for AI-powered disruption within each
step. Unlike most new technologies in radiology, AI can potentially impact every step
of a radiologist's value-adding activities.
What Is AI? A Brief Primer
What Is AI? A Brief Primer
Although many definitions for AI exist, the term is generally used in the medical
context to refer to devices or systems that can perceive some element of their environment
and use this information to achieve a predefined goal.[5 ] Most medical use cases comprise what is known as weak or narrow AI, indicating the
goal is the completion of a single task or set of tasks. Machine learning specifically
is a subset of AI ([Fig. 2 ]) that focuses on the development of computer algorithms without the explicit encoding
of decision-making rules.[6 ] Machine learning is often subdivided further into supervised and unsupervised learning.
In supervised learning, the algorithm is given annotated data (“ground truth” data)
that is used in the development of the algorithm. In unsupervised learning, the system
is supplied with unlabeled data that it must classify itself.[6 ] Deep learning and specifically deep convolutional neural networks (also known as
DCNNs or CNNs) represent a subset of supervised machine learning that has garnered
the greatest amount of enthusiasm in recent years. DCNNs are a type of supervised
learning that uses an algorithmic structure based on neural networks with many layers,
that is, deep neural networks.[7 ] The power of this technique is in its scalability and the ability of the neural
network architecture to extract its own relevant features from data without any direction
other than labeled input data.
Fig. 2 Relationship of artificial intelligence, machine learning, supervised learning, and
deep learning. Recent advances in machine learning in radiology have primarily come
using deep learning techniques.
AI in Scheduling and Protocoling
AI in Scheduling and Protocoling
As mentioned earlier, AI is unique from many prior technological breakthroughs in
radiology such as voice recognition or development of new modalities because it is
capable of affecting every portion of the radiologic workflow. For example, AI algorithms
can be used at the point of care for medical decision support in requests for imaging
by not only analyzing a patient's medical record and determining the appropriateness
of imaging, but also in providing guidance for which imaging exam may be most appropriate.[8 ] The American College of Radiology (ACR) has developed rule-based appropriateness
criteria to assist in this task, and deep learning algorithms can be applied to further
tailor exams.[9 ] An example would be providing guidance for utilizing computed tomography (CT) or
magnetic resonance imaging (MRI) rather than ultrasound in a patient with a body habitus
that may preclude accurate sonographic diagnosis.
AI can also be used at the time of a patient's scheduling. Curtis et al demonstrated
an AI model that could accurately predict wait times or appointment delays for CT,
MRI, radiography, and ultrasound.[10 ] The ability to communicate these times to patients can result in improved patient
satisfaction. Additionally, collection of data regarding wait times and appointment
delays can be used to actively identify process improvement opportunities, which can
increase the throughput of patient scanning and therefore increase the number of exams
performed by any given resource.
Optimization of image acquisition using AI is also being targeted by industry and
research parties alike. For example, some systems have shown promise in de-noising
data acquired at significantly lower radiation levels than previously required, resulting
in diagnostic imaging with dramatically low radiation exposure.[11 ] Similarly, AI-powered image processing can be applied during or after the reconstruction
steps to minimize artifacts introduced during the acquisition process.[12 ] This includes a decreased radiation dose, particularly in the case of positron emission
tomography (PET) and CT, and decreased scanning time for MRI.[13 ] The decrease in radiation dose offered by such approaches could serve to popularize
exams such as PET/CT or multiphasic CT, which are often deferred or avoided due to
concerns about radiation. Decreased radiation dose may also lower the threshold for
imaging and increase the frequency of imaging, resulting in greater imaging volume.
This may prove to be particularly useful in cancer screening programs. From an MRI
perspective, deep learning algorithms have shown the ability to reduce scan time by
improving reconstruction efficiency as well as improving scan quality.[14 ] Scanning protocols can also be dynamically adjusted before or during scanning, based
on patient factors as well as imaging content from early sequences, to optimize sequence
selection and decrease scanning time.[15 ] This decrease in necessary MRI scanner time to arrive at an accurate diagnosis could
enable increased patient throughput.
AI as a Tool to Optimize Radiology Workflow
AI as a Tool to Optimize Radiology Workflow
Perhaps the greatest potential area of immediate disruption is in AI work list management.
Classifiers have been developed to identify abnormal chest radiographs with the intent
of expediting interpretation of an abnormal exam.[16 ] Similarly, classifiers have been developed to detect intracranial hemorrhage and
stroke on noncontrast head CT and acute stroke on diffusion-weighted MRI.[17 ]
[18 ] Again, these tools could be incorporated into the picture archiving and communication
system (PACS) to develop a “smart work list” that flags the radiologist to interpret
abnormal exams first, thereby reducing the time to diagnosis and treatment.[17 ] Similar techniques could also be used to flag studies that may potentially be nondiagnostic,
such as improperly positioned radiographs or motion-degraded cross-sectional exams.
By identifying these issues at the time of scanning, the technologist would be able
to repeat the exam or confer with the radiologist, thereby minimizing issues in patient
callbacks and delays in patient care.
Intuitive and useful hanging protocols are another pre-interpretation step that AI
can aid. Hanging protocols refer to the details of how a study, and relevant prior
studies, are displayed when opened in a PACS. Among surveyed radiologists, automated
hanging protocols was considered the biggest factor in improving efficiency.[19 ] Currently, hanging protocols rely on Digital Imaging and Communications in Medicine
(DICOM) data generated by scanners that is inherently inhomogeneous and therefore
introduces variability when images from multiple scanners are being reviewed at a
single workstation. Ongoing research for this purpose leverages the ability of AI
to identify structures within the image and integrate that information with image
metadata to display images in a way that can minimize the time from exam loading to
interpretation.
Image Interpretation with AI: What Can It Really Do?
Image Interpretation with AI: What Can It Really Do?
Some of the most excitement regarding deep learning techniques in radiology focuses
on automated image interpretation. Rajpurkar et al demonstrated a model with similar
diagnostic accuracy to radiologists for pneumonia on chest radiography in a public
data set,[20 ] and similar studies were performed for fractures,[21 ] tuberculosis detection,[22 ] and bone age determination.[23 ] Despite these studies, there is currently no commercially available solution that
will interpret images and generate a report. Thus this research really serves to demonstrate
the potential of AI to augment the radiologist work flow by alerting the radiologist
to possible findings. Advances in natural language processing (NLP) may also soon
mean that a report can be auto-populated with the imaging findings that the machine
sees, thereby reducing the time the radiologist spends in generating a report.[23 ] Another developing application for AI in image interpretation lies in the detection
of incidental findings. It was shown that simple classifiers can be trained to detect
shoulder dislocations on chest radiographs with a sensitivity of at least 70%. With
AI helping to alert radiologists to incidental findings, greater time and cognitive
effort can be dedicated to answering relevant clinical questions while the machine
assistant can assess the remainder of the scan. As AI systems become more advanced,
their role in assisting diagnosis will grow. It is relatively easy to envision a narrow
AI engine tailored to bone tumor evaluation, for example, that can supply a differential
diagnosis for an imaging finding and therefore assist the radiologist in interpretation.
Intelligent Reporting Systems
Intelligent Reporting Systems
Many additional opportunities within the realm of image interpretation exist that
do not necessarily entail an automated diagnosis. For example, automated lesion measurements
with volumetric calculations can be performed with advances in image segmentation
driven by deep learning.[24 ] More precise and reproducible measurements could be used to provide better longitudinal
data when following tumors while also significantly reducing imaging interpretation
time by removing these tasks from the radiologist. This could apply to anatomical
measurements such as the tibial tuberosity-trochlear groove measurement or calculation
of the α angle in the hip. In fact, any task that is repeated with mild variation
can likely be automated by AI. Simple versions of AI facilitating reporting can already
be seen in applications such as Nuance PowerScribe that can be trained to recognize
certain key phrases and utilize rule-based systems to automatically organize an unstructured
dictation (Nuance, Burlington, MA, USA). As NLP techniques advance, it is likely that
the process of reporting an exam can also be significantly automated without the need
for specifying explicit rule sets beforehand.
NLP can also be used to extract relevant information from an electronic medical record
(EMR).[25 ] An AI system could be trained to extract cancer history from the EMR if an osseous
lesion is recognized in imaging. Again, providing the radiologist with the relevant
information at the time of diagnosis would lead to improved workflow efficiency.
Radiology as Big Data
Much of the excitement surrounding deep learning in radiology depends on the fact
that radiology is a data-driven specialty. Given this reality, it is also quite likely
that there is additional data within our imaging that is clinically relevant, although
it may not be apparent at the time of interpretation. Radiomic analysis, which can
be described as the quantitative analysis of images into mineable data that can be
analyzed mathematically, will also become more commonplace as AI tools become more
accessible.[26 ] Recently, Rajkomar et al demonstrated a deep learning model that could predict in-hospital
mortality, readmission, length of stay, and discharge diagnoses better than currently
used clinical models.[27 ] Similarly, deep learning may enable large-scale radiomic analysis with the potential
to identify disease characteristics based on imaging patterns that may not be readily
apparent to the human eye.[28 ] “Big data” techniques, consisting of both deep learning and other machine learning
strategies, may also allow for better prediction of response to therapy and provide
information on which patients may benefit from percutaneous intervention as opposed
to surgery.
Real Communication with Artificial Intelligence
Real Communication with Artificial Intelligence
AI can also be used after the conclusion of imaging interpretation in communication.
Google (Alphabet Inc., Mountain View, CA) recently demonstrated its Duplex digital
assistant that can converse with humans in real time to make a reservation at a hair
salon or restaurant.[29 ]
[30 ] Although calling in incidental and routine imaging findings may be more complicated
than booking a table for six, it is apparent that such technology could at least be
partially adapted to a radiology practice to expedite and perhaps automate the reporting
of nonemergent findings. Similarly, an AI chat-bot has demonstrated the ability to
advise referrers to select appropriate imaging studies without human intervention.
The rise of personal digital assistants such as Siri (Apple, Cupertino, CA) and Alexa
(Amazon.com, Inc., Seattle, WA), although currently predominantly for consumer use,
hold some promise for facilitating various communication steps within the radiology
workflow. As more of these tasks, which often do not require the extensive training
that radiologists undergo, become automated, radiologists will again have increased
time to dedicate to the remainder of their activities.
Business Aspects of AI
Billing and coding, a complex but fundamentally necessary task in any radiology practice,
can also be optimized using AI. Systems that evaluate both the images and incorporate
information from reports using NLP can ensure that imaging exams are billed and coded
appropriately. It was estimated that > 100 variables may contribute to denied claims,
a number large enough to be tedious and expensive for humans but still easily handled
by modern AI systems.[31 ] Thus machine learning may be able not only to identify rejected claims but also
assess what could be added to resubmit the claim and reduce similar future denials.
What is a radiologist to do with the extra time and efficiency afforded by these possible
AI solutions? The natural answer is to increase throughput, which of course would
also serve to diagnose more patients, reduce delays in treatment, and augment the
bottom line. But the AI-augmented radiologist will also have opportunities that radiologists
lost several years ago: time to consult with referrers and patients. The ACR in recent
years has pushed for greater visibility of radiologists with patients and patient-facing
providers.[32 ] Perhaps AI will increase our efficiency to a point where it is cost effective for
radiologists to provide in-person consultation to referrers and patients, increasing
the visibility and health of our profession as a whole.
AI Products: What Is the Fine Print?
AI Products: What Is the Fine Print?
Although the promise of AI in radiology has captured the imagination of the research
community and many potential applications have been outlined here, finding truly disruptive
AI that is currently commercially available yields startling few results ([Fig. 3 ]). It is true that some software packages may automatically incorporate measurements
made into a report (iMorgon, San Mateo, CA) or that dictation software may automatically
structure a dictation using rule-based methods (Nuance Communications, Burlington,
MA), and that these applications could be considered very narrow AI. However, much
of the groundbreaking tools that AI promises are not without their own issues. In
the case of image interpretation and diagnostic support specifically, deep learning
techniques notoriously suffer from the so-called black box problem: Many of their
decisions cannot be explained. There has been some work in this space, particularly
through the development of saliency and attention maps that can demonstrate what portions
of an image contribute to neural network activation.[33 ] But these explanations do not come near the robustness of radiologists providing
an explanation of their conclusions using a combination of anatomical knowledge and
imaging physics. In fact, some authors debate whether deep learning systems will ever
be capable of this level of explainability.[34 ]
Fig. 3 Consideration of currently available AI-powered products and summary of future directions.
Short term opportunities are predominantly comprised of non-image interpretation tasks.
Decisions that directly affect patient care, particularly without the possible intervention
of a radiologist, have a longer time horizon as prospective studies proving safety
will be necessary.
In 2018, the U.S. Food and Drug Administration proposed classifying AI-powered computer-aided
diagnosis software for breast imaging as class II devices, making it easier for similar
AI products to come to market.[35 ] The coming wave of AI-driven systems will bring their evaluation metrics that have
been borrowed from the computer science and statistics fields. Radiologists need to
educate themselves on the evaluation of AI systems because they will be the initial
targets of these vendors. Additionally, gaining knowledge in the evaluation of AI
systems within health care can also give radiologists a unique niche in larger health
care systems where decision making regarding AI systems even outside of the radiology
department will need to be made. For example, radiologists commonly discuss diagnostic
tests using the metrics of sensitivity and specificity. In machine learning, however,
the metrics often quoted are precision (a synonym for positive predictive value) and
recall (synonymous with sensitivity).[36 ]
Although there has been much speculation regarding the replacement of radiologists
by AI, the fact is that most applications of AI in radiology are best served as adjuncts
to a radiologist. The entire lifecycle of a radiology exam, from order placement to
result communication, can be made more efficient by the incorporation of AI techniques.
Advanced information synthesis, the combination of multiple findings, patient history,
and clinical data to arrive at a diagnosis, remains the holy grail of AI but is still
demonstrably far in the future. Radiologists are in a unique position to welcome the
AI revolution in health care by virtue of their close relationship with an extraordinary
amount of data. In fact, AI will grant radiologists new opportunities to participate
in patient care both via increased time for consultation but also through improvements
in imaging and extraction of useful data from those images. Radiologists who position
themselves to use this technology to their benefit, rather than to avoid it out of
fear, will gain a favorable position for themselves and for our specialty.
