Introduction
The field of “Sensor, Signal, and Imaging Informatics” (SSII) offers an active and
growing domain in biomedical informatics. In addition to systems to gather physiological
data on hospital patients and dedicated signal recording devices for outpatient use,
the variety and quality of signal and sensor data are growing rapidly through the
increased adoption of wearable devices. Imaging informatics, too, remains a dynamic
area of research and clinical applications as the number of clinical departments that
utilize imaging-derived information continues to grow, including radiology, pathology,
dermatology, cardiology, ophthalmology, and other image-rich medical specialties.
Although 2016 has brought numerous advances in machine learning and the rise of deep
learning techniques in a wide spectrum of clinical applications, no field has felt
the impact of these developments more than the fields related to signals and imaging.
Given that signal and imaging datasets tend to be high-dimensional and longitudinal
in nature, these large datasets are ripe for machine learning approaches such as deep
learning to identify spatial and temporal patterns.
The large quantities of data generated in these fields have provided ample material
for those interested in “big data” and for machine learning systems that seek to interpret
findings, make diagnoses, and predict clinical outcomes. For example, a recent example
of training a deep learning to detect diabetic retinopathy on a large cohort of over
120,000 retinal images achieved up to 97% accuracy [[1]]. Such applications of machine learning are also starting to be published in mainstream
clinical journals such as the Journal of the American Medical Association. The rise of interest in this area has led individuals to wonder aloud whether machine
learning will start displacing jobs of clinicians such as radiologists and pathologists
[[2]]. Although there remain numerous challenges related to data quality, interoperability,
integration, and model validation and deployment, medical advances will rely on the
effective use of machine learning to extract actionable information from the growing
amounts of “big data” being generated.
Our selection of publications from 2016 highlights the significant contributions that
medical informatics research has had in the signals and imaging communities, addressing
challenges related to data quality, machine learning, clinical decision support, and
workflow. The six selected papers offer a glimpse of the quality and breadth of the
work being conducted in SSII. In addition, the survey paper by Chennubhotla and colleagues
reviews the developments in a specific area, cancer imaging, that embodies the aspirations
and opportunities that the SSII community is facing [[3]].
About the Paper Selection
To compile the list of eligible papers, we performed a search of two electronic databases,
PubMed/Medline® and Web of Science®. The search was conducted in December 2016 to identify peer-reviewed journal articles
published in 2016, in the English language, related to SSII research in medical informatics.
As in previous years, a wide spectrum of MeSH® keywords and topics was considered, including image and signal processing, pattern
recognition and information extraction, telemedicine, sensor monitoring, and computer-aided
diagnosis. We used free-text (tagged as “tw” in the bibliographic record) and MeSH® terms (tagged as “mh”) as keywords. PubMed/Medline® was queried to test keywords in an iterative process. Consequently, two queries were
built: one based on MeSH® terms used to search the major topics in the PubMed/Medline® database, the second one based on free-text keywords searched in title or abstracts
through PubMed/Medline® and Web of Science® databases. We also added keywords that have been trending this past year such as
deep learning. One of the three section editors (WH) performed the searches. In addition
to the search of electronic databases, manual searches of key themes were performed
in leading journals of biomedical informatics, imaging informatics, and imaging specialty
journals, such as Journal of the American Medical Informatics Association, Journal of Digital Imaging, and Radiology.
The search results were reconciled into a single list of 686 papers. The three section
editors independently screened the titles and abstracts to identify relevant papers.
The section editors classified the papers into three categories: accepted, rejected,
or pending. They then reviewed in detail the accepted and pending full-text articles
to finally reach a consensual list of 15 candidate papers. Papers were considered
according to their originality, scientific and/or clinical impact, and scientific
quality. Individual editors recused themselves in decisions related to papers that
posed potential for conflict of interest. In accordance with the IMIA Yearbook selection process, the 15 candidate best papers were evaluated by the section editors
and by at least two additional external reviewers. Six papers were selected as best
papers ([Table 1]). A content summary of the selected best papers can be found in the appendix of
this synopsis.
Table 1
Best paper selection of articles for the IMIA Yearbook of Medical Informatics 2017
in the section ‘Sensor, Signal and Imaging Informatics’. The articles are listed in
alphabetical order of the first author’s surname.
|
Section
|
|
Sensor, Signal and Imaging Informatics
|
-
Arnold CW, Wallace WD, Chen S, Oh A, Abtin F, Genshaft S, Binder S, Aberle D, Enzmann
D. RadPath: A web-based system for integrating and correlating radiology and pathology
findings during cancer diagnosis. Acad Radiol 2016 Jan;23(1):90-100.
-
Hravnak M, Chen L, Dubrawski A, Bose E, Clermont G, Pinsky MR. Real alerts and artifact
classification in archived multi-signal vital sign monitoring data: implications for
mining big data. J Clin Monit Comput 2016 Dec;30(6):875-88.
-
Kalpathy-Cramer J, Zhao B, Goldgof D, Gu Y, Wang X, Yang H, Tan Y, Gillies R, Napel
S. A comparison of lung nodule segmentation algorithms: methods and results from a
multi-institutional study. J Digit Imaging 2016 Aug;29(4):476-87.
-
Moss TJ, Lake DE, Calland JF, Enfield KB, Delos JB, Fairchild KD, Moorman JR. Signatures
of subacute potentially catastrophic illness in the ICU: model development and validation.
Crit Care Med 2016 Sep;44(9):1639-48.
-
Petousis P, Han SX, Aberle D, Bui AA. Prediction of lung cancer incidence on the low-dose
computed tomography arm of the National Lung Screening Trial: A dynamic Bayesian network.
Artif Intell Med 2016 Sep;72:42-55.
-
Springer DB, Tarassenko L, Clifford GD. Logistic regression-HSMM-based heart sound
segmentation. IEEE Trans Biomed Eng 2016 Apr;63(4):822-32.
|
Conclusions and Outlook
The use of pattern recognition techniques to provide increasingly intelligent diagnostic
understanding is a key area of work in sensor, signal, and imaging informatics. The
candidate papers offered innovative approaches to address the evolving challenges
of analyzing sensor, signal, and imaging data. In signal and sensor informatics, the
data typically represent multiple single-dimensional values that change over time;
the input data’s multidimensionality and temporal fluctuations pose significant challenges.
Large sets of signal data can allow detection of subtle patterns of physiologic changes
in critically ill patients; such changes may precede clinically overt deterioration,
and may allow treatment to be more effective. Moss et al. analyzed 146 patient-years
of vital-sign and ECG time-series data on 9,232 ICU patients to identify physiologic
signatures in the hours before clinical detection and intervention [[4]]. Springer and colleagues addressed the accurate segmentation of the first and second
heart sound within noisy real-world phonocardiogram recordings using a hidden semi-Markov
model (HSMM) extended with the use of logistic regression for emission probability
estimation [[5]]. Their analysis of 10,172 seconds of recordings from 112 patients achieved an average
F1 score of 95.63 ± 0.85%.
In an analysis of physiologic markers of acute emotional stress in young healthy volunteers,
Hernando et al. reported that analysis of respiration and heart-rate variability together
more reliably characterized autonomic nervous response [[6]]. Artifacts pose a significant challenge to accurate classification of signal data.
Hravnak and colleagues developed an approach to distinguish real alerts from artifacts
in archived multi-signal vital sign monitoring data [[7]]. Accurate “data cleaning” steps are critical for mining “big data” sets of high-volume,
real-time physiologic data.
In imaging informatics, input data typically represent two- or three-dimensional spatial
information; some studies may include “dynamic” imaging to visualize motion of the
heart or a joint, or to view physiological changes that vary with time and/or external
stimulation. For example, changes in brain structures over time – particularly of
the hippocampus – may offer an important bio-marker of Alzheimer disease. Chincarini
et al. found that longitudinal volume measurements of the hippocampus can provide
meaningful clinical insight regarding the development and progression of Alzheimer’s
disease [[8]]. Temporal information also plays a role in imaging-based decision making. Petousis
et al. applied a dynamic Bayesian network (DBN), a model that repeats the static interactions
of a conventional Bayesian network over time, to data from the National Lung Screening
Trial to improve the prediction of lung cancer using longitudinal data [[9]].
The papers in the SSII domain addressed novel approaches to improve clinical use of
diagnostic information for cross-specialty correlation and follow-up. Arnold et al.
described a web-based platform called RadPath to correlate and integrate findings
from diagnostic radiology and pathology [[10]]. This work engenders and facilitates communication between specialties to address
the increasing complexity and team-based approaches of modern medical care. Tracking
the recommendations from diagnostic examinations is critical, too. Many actionable
findings – such as an incidentally detected kidney lesion on a CT performed for trauma
– demand follow-up care that may not be addressed during the patient‘s acute presentation
[[11]]. Such systems seek to “close the loop” to assure that recommended follow-up examinations
are completed.
Deep learning algorithms require a large amount of data to fine-tune their parameters,
and acquiring sufficiently large sets of “labeled” data may not be feasible. Sun et
al. combined a relatively small amount of labeled data with a large amount of unlabeled
data to enhance a deep convolutional neural network model for breast cancer diagnosis
[[12]]. Their approach significantly improved the performance of their diagnostic model.
Summers provides an excellent review of machine learning approaches in abdominal CT
interpretation [[13]]. This work, published in a leading clinical radiology journal, points up the progress
made in fully automated identification of organs and blood vessels in abdominal CT
images, and the impressive performance of deep learning-based methods in many of the
described applications.
Emerging research also is looking at how to effectively manage and integrate multi-modal
datasets. Peikari and colleagues used a texture-based approach to “triage” diagnostically
relevant regions from digitized pathology whole slides of breast cancer [[14]]. Their approach outperformed two previously proposed methods and achieved an AUC
of 0.87 on 5,151 specimens from 65 tissue slides. Milchenko et al. developed a framework
to pre-process multi-spectral MRI data acquired from heterogeneous imaging systems
[[15]]. They applied this framework in the domain of neuro-oncology MRI data acquired
from various institutions, different MRI devices, and varying sequence parameters.
The ability to capture whole-embryo morphology and gene expression has made three-dimensional
(3D) imaging an important modality to understand gene-phenotype relationships in developmental
biology. Brown and colleagues presented a software suite to facilitate uploading,
analysis, and dissemination of 3D images of embryos [[16]].
Competitions and open databases provide materials that allow investigators to test
new algorithms and compare their performance to other approaches. Comparative analyses
of algorithms in the literature have been limited by the lack of high-quality, standardized,
and rigorously validated data. Liu and colleagues developed an open-access database
for the evaluation of automated heart sound segmentation and classification [[17]]. Kalpathy-Cramer et al. reported a multi-institutional effort to assess the repeatability
and reproducibility of nodule borders and volume estimate bias of computerized segmentation
algorithms in CT images of lung cancer [[18]].
In conclusion, the selected papers are a small sampling of the many efforts that are
advancing the science of machine learning in healthcare. In connection with the 2016
Yearbook theme of “Learning from experience: secondary use of patient data”, many
of the developments described in the cited papers would not be possible without the
availability of large, curated patient datasets from which algorithms can be trained
and validated. As new datasets that combine clinical, imaging, molecular, and signal
data become available, new insights can be discovered, leading to more powerful and
robust tools for pattern recognition.
Appendix: Content Summaries of the Selected Best Papers for the IMIA Yearbook 2017
in the “Sensor, Signal, and Imaging Informatics” Section
Arnold CW, Wallace WD, Chen S, Oh A, Abtin F, Genshaft S, Binder S, Aberle D, Enzmann
D
RadPath: A web-based system for integrating and correlating radiology and pathology
findings during cancer diagnosis
Acad Radiol 2016 Jan;23(1):90–100
Cancer diagnosis should involve close coordination and communication among physicians
in radiology, pathology, and clinical care in order to assure timely, efficient, and
medically correct treatment decisions. The authors developed the web-based RadPath
system to correlate and integrate diagnostic radiology and pathology report findings.
RadPath retrieves reports, images, and test results from the institution’s clinical
information system to construct a database of relevant information. Radiologists can
correlate imaging findings with the pathology diagnosis; they otherwise might not
receive feedback about pathology results related to cases in which they have interpreted
diagnostic studies or performed biopsies. Medicine’s increasing complexity and team-based
management approaches have demanded improved clinical communication and information
exchange. The authors’ work is a welcome addition.
Hravnak M, Chen L, Dubrawski A, Bose E, Clermont G, Pinsky MR
Real alerts and artifact classification in archived multi-signal vital sign monitoring
data: implications for mining big data
J Clin Monit Comput 2016 Dec;30(6):875-88
Hospital inpatients at high risk for cardio-respiratory instability undergo continuous
non-invasive monitoring of their vital signs, such as heart rate, respiratory rate,
blood pressure, and peripheral arterial oxygen saturation. To assure accurate diagnosis,
it is critical to distinguish truly aberrant vital sign measurements from false measurements
caused by artifacts such as patient movement, electrical noise, or loose sensors.
The authors used machine-learning algorithms trained on labeled vital sign data to
automatically classify alerts as real or artifact. Their algorithms achieved an area
under the curve (AUC) of 0.72 to 0.94 on a test dataset, which suggests that their
approach could be used for accurate automated classification of vital sign alerts
as real or artifact. Such information provides a “data cleaning” step that can support
data mining for future model building.
Kalpathy-Cramer J, Zhao B, Goldgof D, Gu Y, Wang X, Yang H, Tan Y, Gillies R, Napel
S
A comparison of lung nodule segmentation algorithms: methods and results from a multi-institutional
study
J Digit Imaging 2016 Aug;29(4):476-87
Automated systems play an increasingly important role in identifying, classifying,
and measuring lesions in medical images. These systems must provide accurate and reproducible
volumetric measurements for diagnosis, staging, and assessment of treatment response.
For lung nodules on CT, it is often the case that images are acquired on different
scanners, generated with different imaging parameters (e.g., slice thickness, reconstruction
kernel), and analyzed by different nodule-measurement systems. The authors evaluated
three algorithms on 52 nodules from The Cancer Imaging Archive acquired from 40 actual
patients and from an anatomic phantom. The authors found considerable differences
between the algorithms’ measurements, especially in a subset of heterogeneous nodules.
They recommend that longitudinal studies should use the same software at all time
points.
Moss TJ, Lake DE, Calland JF, Enfield KB, Delos JB, Fairchild KD, Moorman JR
Signatures of subacute potentially catastrophic illness in the ICU: model development
and validation
Crit Care Med 2016 Sep;44(9):1639-48
Critically ill patients may develop life-threatening conditions such as sepsis, respiratory
failure, or hemorrhage, which present frequently with abnormal vital signs. To detect
and treat such conditions most effectively, one should identify these conditions’
more subtle physiologic changes. The investigators analyzed 146 patient-years of vital
signs and ECG data on ICU patients to identify early physiologic signatures of cardiorespiratory
failure. They found that they could detect potentially catastrophic, subacute illnesses
in ICU patients hours before clinical detection and intervention. By detecting potentially
catastrophic conditions more rapidly, they hope to draw attention to the patients
at highest risk, enable earlier interventions, and attain better patient-care outcomes.
Petousis P, Han SX, Aberle D, Bui AA
Prediction of lung cancer incidence on the low-dose computed tomography arm of the
National Lung Screening Trial: A dynamic Bayesian network
Artif Intell Med 2016 Sep;72:42–55
The most effective way to improve cancer survival in high-risk individuals is to identify
disease at an early stage. The National Lung Screening Trial (NLST) showed that low-dose
computed tomography (LDCT) reduced mortality more effectively than radiography in
lung cancer screening, and has led to the implementation of CT-based lung cancer screening
programs. However, LDCTs result in a large number of false-positive results. The authors
explored dynamic Bayesian networks (DBN) to help understand how longitudinal data
might inform lung cancer screening decisions. The lung cancer screening DBNs demonstrated
high discrimination and predictive power with the majority of cancer and non-cancer
cases, and the DBN models outperformed logistic regression and naïve Bayes approaches.
This work highlights the use of a large national dataset to create a model for decision
support and demonstrates the potential of a prediction model to outperform radiologist
interpretation in specific instances.
Springer DB, Tarassenko L, Clifford GD
Logistic regression-HSMM-based heart sound segmentation
IEEE Trans Biomed Eng 2016 Apr;63(4):822-32
Segmentation of heart sounds may be relatively simple in noise-free recordings, but
noise – such as speech, patient motion, and intestinal or breathing sounds – makes
the task more difficult. Neural networks and other static machine-learning approaches
that lack temporal duration and ordering information may not perform as well in differentiating
heart sound-like noises from noisy heart sounds. The authors sought to improve identification
of first and second heart sounds in phonocardiogram (PCG) data. Their model incorporated
a priori knowledge about the expected duration of the heart sounds encoded in a hidden semi-Markov
model (HSMM). The authors evaluated their method on a recorded PCG dataset of more
than 10,000 seconds from 112 patients. Their system’s average F1 score of 96% exceeded
that of 86% achieved by conventional systems. Probabilistic models, such as hidden
Markov models, can improve performance over threshold-based segmentation methods in
the analysis of heart sound recordings.
