Introduction
Capsule endoscopy (CE) has changed the field of small-bowel (SB) investigation [1] with the potential to become a panenteric diagnostic tool [2]. Computational methods incorporated into CE reading software can enhance diagnostic
yield (DY) [3]. Several information technology (IT) groups have proposed software for detection
of SB lesions/bleeding, reducing reading time, lesion localization, motility assessment,
video enhancement and/or data management [1]
[3]. Reducing reading time is beneficial, especially in high volume centers. Previous
work has shown that readers’ experience does not improve detection of lesions in CE
[4]. Therefore, computer-aided detection/diagnosis (CAD) can improve DY.
Despite prolific IT research, incorporating artificial intelligence (AI) systems into
CE reading remains difficult [3]. The backbone of AI system development is based on machine learning algorithms (MLAs)
for automatic detection, localization, and recognition of pathology in CE images and
videos. A large amount of data, in the form of annotations, is required to train MLAs.
Semantic annotations describe the content of CE videos and images, whereas graphic
annotations are pixel-level labels indicating regions of interest (ROIs) ( [Fig.1]). Although there are some online databases [5], these usually include the necessary semantic annotations, but lack graphic annotations
of ROIs. Therefore, such material cannot be directly used by IT scientists for intelligent
systems’ training or as a reference for their evaluation.
Fig. 1 Dataset of angioectasia images and their corresponding graphic annotations, seen
within the KID website interface.
A limited number of datasets composed of images with graphic annotations have become
available in the context of IT studies [3]
[6]. A novel database, KID (κάψουλα interactive database; based on Greek for “capsule”)
(http://is-innovation.eu/kid/) was developed to fill this gap. It is available online, upon free registration,
aiming to provide a reference for research on the development of medical decision
support systems (MDSS) for CE, including the study of the performance of human observers
in comparison to others and CAD.
Methods
Database
Open-source database (Oracle MySQL; https://www.mysql.com/) and web-gallery development software (Coppermine; http://coppermine-gallery.net/) were used. Software tools for video manipulation and image annotation were added
to the KID website. To date, six centers (the KID working group) have contributed
anonymized, annotated CE images/videos from various CE models; more than 2500 annotated
CE images and 47 videos have been uploaded. These include images of (a) normal CE;
(b) vascular lesions including angioectasias and/or bleeding; (c) inflammatory lesions,
including mucosal aphthae and ulcers, erythema, cobblestoning, and luminal stenosis;
(d) lymphangiectasias; and (e) polypoid lesions ([Fig. 2]).
Fig. 2 Top row, from left: P1 and P2 angioectasias, aphthae and ulcer, with corresponding
graphic annotations made using Ratsnake beneath each image, showing the position,
size and shape of the lesions in the images. Bottom row, from left: two images of
nodular lymphangiectasias and two images of polypoid lesions, with graphic annotations
below each image.
Image and video standards
Lesion categorization is based on the CE Structured Terminology (CEST) [7]. Contributions are of high quality (original resolution), not distorted by additional
compression. For images, the recommended standard is ISO/IEC 15948 PNG (Portable Network
Graphics), a popular platform-independent format with lossless compression. Other
acceptable standards include: ISO/IEC, 14496-10, MPEG-4, AVC (Advanced Video Coding)
and H.264. Supported formats for videos include F4V & FLV (Flash video).
Image annotation
The usefulness of KID relies on image annotations. Semantic and graphic annotations
are supported by an open access, platform-independent annotation tool (Ratsnake) [8]. The graphic annotation process is shown in [Fig. 3] and [Video 1]. Semantic annotation is done through textual labels, and using standard web ontology
language description logics (OWL DL) [9]. The quality of data and annotations submitted to KID are scrutinized by an international
scientific committee (http://is-innovation.eu/kid/committee.php); contributions not meeting the aforementioned standards are rejected.
Fig. 3 Use of the Ratsnake annotation tool to perform graphical annotation of an angioectasia
on capsule endoscopy (CE). Left: original image. Right: graphic annotation of the
angioectasia.
Video 1: Video showing annotation process using Ratsnake software.
An experiment using the KID database: Computer-aided lesion size measurements based
on color image segmentation
A total of 64 images of gastrointestinal lesions taken with MiroCam® (IntroMedic Co., Seoul, Korea) were used. The lesions were: angioectasias (n = 27),
lymphangiectasias (n = 9), ulcers (n = 9), chylous cysts (n = 8), polypoid lesions
(n = 6), and small-bowel aphthae (n = 5). Graphic annotations made by expert readers
(AK, ER, ET; > 2000 CE readings each) were used as lesion surface size reference standards.
The images were automatically segmented into two regions: a ROI, i. e. the lesion
in question, and the rest of the image. This was performed using the Localized Region-based
Active Contour (LRAC) [10] algorithm, which is capable of segmenting regions characterized by heterogeneity
in grayscale images; see [Fig. 4] for a stepwise graphic presentation. The reader initializes the LRAC by defining
a circular contour roughly on or around the lesion, starting at a random point in
the image. The lesion did not need to be fully included in the initial contour. The
algorithm calculates contours based on intensity histogram information (i. e. information
on image brightness and intensity) from the regions inside and outside the contour.
The calculations are performed locally, around each point along the contour. The algorithm
continues to run until the overall similarity of the histograms inside and outside
the contour is minimized. In this experiment, we extended the algorithm to the three
components of the Commission internationale de l’éclairage-Lab (CIE-Lab) color space
representation (instead of the standard RGB) [11]. Components of this space represent lightness (L), which is approximately equivalent to the respective grayscale image, quantity of
red (a > 0) or quantity of green (– a > 0), quantity of yellow (b > 0) or quantity of blue (– b > 0) of a pixel ([Fig. 5]). [Fig. 6] shows the results of image segmentation using this algorithm applied to the a component of CIE-Lab, compared to in RGB. The Jaccard Index (JI) [12] was used to assess the similarity of the ROI obtained with the aid of LRAC compared
to the graphically annotated ROI obtained by the expert readers (gold standard) per
image, i. e. the agreement between the expert human readers and the algorithm. The
JI is considered to be the most suitable and popular measure for the assessment of
image segmentation algorithms [12]. It quantifies the overlap between two ROIs as the ratio of their intersection to
their union with respect to the human readers. Therefore, it is independent from the
measurement unit, e. g. pixels2 or mm2, used to quantify the measured area. An illustrative example is provided in [Fig. 7].
Fig. 4 Segmentation of image using the Localized Region-based Active Contour (LRAC) algorithm.
a User-defined initial contour. b Contour deformation/morphing based on local histogram information on brightness and
intensity in the various circular neighborhoods at each point on the contour. c Segmented image obtained.
Fig. 5 CIE-Lab color wheel (left) compared to the RGB color wheel (right).
Fig. 6 Image segmentation by Localized Region-based Active Contour (LRAC) algorithm. Top
row, from left: original image of mucosal break with surrounding erythema; image segmentation
using the a component of CIE-Lab; the final result of image segmentation where the contours have
been defined and marked. Bottom row: the image when broken down into red (R), green
(G) and blue (B) channels under the traditional RGB system.
Fig. 7 Agreement between a human reader and the algorithm as quantified by the Jaccard Index
(JI). Given a region annotated by a human expert (left) and a region annotated by
the algorithm (right), the intersection of the two regions corresponds to the True
Positive (TP) pixels, i. e. those actually belonging to the abnormality. The union
of the two regions corresponds to the sum of the False Negative (FN), the False Positive
(FP) and the TP. Thus, if the two regions perfectly coincide, FN = 0, FP = 0 and their
intersection (TP) becomes equal to their union, resulting in JI = 100 %. If there
is no match between the two regions, then TP = 0 and JI = 0.
Results
The algorithm was evaluated for the measurement of six different types of small-bowel
lesions,
for each channel of CIE-Lab color space. The lesion areas were measured in pixel units,
which, in the context of CE, is a more feasible and accurate approach. The average
surface measurements closest to those performed by expert human readers were obtained
by application of LRAC on the red-green scale of the CIE-Lab color space, with a JI
of 67 ± 13 %. This result complements the findings in our previous study, indicating
component a as an informative source of saliency for automated lesion detection [11]. The agreement between human readers and the algorithm per lesion type is summarized
in [Table 1]. The most accurate measurements were obtained for lymphangiectasias, whereas this
algorithm is less suitable for the measurement of ulcers.
Table 1
Agreement between reviewers and software in measuring lesion size for various types
of lesion seen on capsule endoscopy (CE).
|
Lesion
|
JI, mean ± SD, %
|
|
Angioectasias
|
64 ± 11
|
|
Aphthae
|
64 ± 8
|
|
Chylous cysts
|
70 ± 14
|
|
Lymphangiectasias
|
81 ± 6
|
|
Polypoid lesions
|
75 ± 21
|
|
Ulcers
|
56 ± 9
|
Abbreviations: JI, Jaccard Index; SD, standard deviation.
Discussion
Human factors remain a barrier to timely and accurate CE diagnosis [4]. AI systems can improve clinical performance, patient safety, and resource utilization
[1]
[3]. Open interdisciplinary exchange of information is key to technological advancement
and therefore improved clinical outcomes [3]. New technological developments may not always meet pertinent healthcare needs due
to little communication between software engineers and clinicians; furthermore, open
access databases of endoscopic images are scarce, especially those specifically related
to small-bowel CE [5]. This is despite growing clinical demand and use of CE as an investigative modality.
However, such interactive formats are vital for engaging a new generation of clinicians;
this is currently hindered by inadequately developed software [13]. Therefore, KID aims to be a comprehensive and all-encompassing resource for continuous
development of CAD in CE, and to encourage two-way dialog between technological developers
and end-users. For example, KID compiles images from all commercial CE models and
is international, thus increasing its scope.
The experiment detailed above shows that generally good agreement was achieved between
expert human readers and the MLA in measuring the size of common small-bowel lesions.
This implies automated lesion measurement is feasible, and MLAs could eventually replace
or drastically reduce the workload of valuable human resources. In a recent study,
van der Sommen et al. [14] detailed collaboration between IT engineers and clinicians to develop a CAD algorithm
for diagnosis of early neoplasia in Barrett’s esophagus, with good results. An advantage
of the method presented in this study over previous automated measurement approaches
is its suitability for a variety of lesion types. In a recent study [15] using images of angiectasias available in KID, we showed that the interobserver
agreement between CE reviewers, in terms of JI, in lesion annotation ranges between
65 ± 15 % and 67 ± 13 %, and the respective intraobserver agreement, between 69 ± 17 %
and 71 ± 13 %. This dataset was similar in terms of the morphological characteristics
of the displayed angiectasias, indicating that our MLA has a performance comparable
to that of human readers. However, a limitation shown by the experiment is that it
does not perform as well with all mucosal lesions. Further algorithm development is
therefore required, showing the need for platforms such as KID.
In conclusion, KID is, to our knowledge, the only database of CE images and videos
with both graphic and semantic annotations, developed specifically for MDSS research.
It provides a platform for data sharing and CAD software development. The experiments
detailed are proof-of-principle studies demonstrating the potential for KID to fulfill
this role.
