Introduction
Endoscopy is an essential tool for the management of ulcerative colitis (UC) due to
the fact that it allows us to carry out a visual assessment of the severity of the
disease, as well as assessing the efficiency of treatment [1]
[2]. Today, this is of great importance, as endoscopic improvement and complete healing
of the previously observed lesions represent the main aims for most patients. However,
it is not easy to make a correct objective assessment of the severity of lesions on
endoscopy. To improve assessment and homogenize endoscopic findings, different scores
have evolved.
Despite the fact that there are multiple endoscopic indices described in the literature,
the Mayo Endoscopy Score (MES) is the most used endoscopic index due to its simplicity.
Nonetheless, two other indices have been developed over the last few years, and are
being used in recent studies: the Ulcerative Colitis Endoscopy Index of Severity (UCEIS)
and the Ulcerative Colitis Colonoscopy Index of Severity (UCCIS).
The MES was developed in 1987, by Schroeder et al [3], during a controlled trial with slow-release oral mesalazine placebo for the treatment
of active UC, in which a proctosigmoidoscopy was used to observe colonic mucosa and
its improvement. This score assesses the vascular pattern, the reliability and the
presence of erosions, scored from 0 to 3 in a simple way [4]. It has never been validated in any study, but through use in habitual clinical
practice [5]. Other studies were developed afterwards to analyze the interobserver correlation,
which varies from 0.45 to 0.75, as reported by Vashist et al [6].
In 2012, the UCEIS index was created by Travis et al [7] as a tool to accurately predict the general assessment of the endoscopic severity
in UC. This score is a scale of nine points (0–8) which assesses the vascular pattern,
presence of bleeding and presence of erosions/ulcerations, each one having different,
well-defined levels of severity. This index has been validated with an interobserver
Kappa correlation of 0.50 [8]. What is more, this index has been shown to be of great prognostic use in serious
flare-ups of UC, in such a way that a UCEIS score ≥ 7 is correlated with a high likelihood
of rescue treatment being administered (infliximab or cyclosporine) [9].
Samuel et al [10] developed the UCCIS in 2013. This index ranges from 0 to 13. Its score is calculated
by means of a formula that takes into account vascular pattern, granularity, ulcers,
and presence of bleeding. It shows a good to excellent interobserver correlation and
a good correlation with the index of clinical activity, the SCCAI (Simple Clinical
Colitis Activity Index). This index is used to assess UC activity by means of clinical
parameters, (excluding biochemical and endoscopic parameters) and the definitive remission
of the patient (defined as a correlation coefficient of 0.61 to 0.80) [10]. However, the viability and the simplicity of use of UCCIS has not yet been assessed.
A study demonstrating a good correlation between UCEIS and MES (Kappa index 0.713,
P < 0.001) has recently been published [11]. Nonetheless, in the study by Ikeya et al [5], it was observed that the UCEIS detected subtle changes in the mucosa better than
MES, and that those changes correlated with survival. Also, in the study by Xie et
al [12], it was found that UCEIS was superior to MES concerning whether patients needed
a colectomy after a severe outbreak of UC, presenting a ROC area of 0.85, with a sensitivity
of 60.3 % and a specificity of 85.5 %; the ROC curve for MES was 0.65.
Of all the indices, the MES is the most used endoscopic index, both in normal clinical
practice and in clinical trials, due to its simplicity. However, there are few studies
comparing interobserver concordance within this index, let alone comparing the three
indices with each other. This study was carried out to assess the interobserver concordance
of these endoscopic indices and the impact their results have in clinical practice.
Patients and methods
The objective of our study was to analyze the interobserver concordance in endoscopic
assessment of the degree of UC activity among three endoscopists according to three
endoscopic scores (MES, UCEIS and UCCIS) and its impact in clinical practice.
This was a single-cohor,t observational, comparative study of patients diagnosed with
UC who had undergone a colonoscopy, according to normal clinical practice, at the
University Virgen Macarena Hospital (HUVM). These colonoscopies were carried out by
any of the endoscopists at the hospital and a video was recorded anonymously and then
seen by endoscopists A, B and C. Only one of them specialized in inflammatory bowel
disease (A). All of them were familiar with MES and none use UCEIS and UCCIS in clinical
practice.
The illness was classified according to the MES, UCEIS, and UCCIS by three different
expert endoscopists. An expert endoscopist was considered as having more than 15 years
of experience and/or having carried out more than 10,000 colonoscopies.
The video recordings were carried out using the Endobase program on withdrawing the
colonoscopy, from the cecum to the rectum, or at least the most affected section,
for no less than 60 seconds.
Inclusion criteria were age ≥ 18, previous diagnosis of UC, follow-up at the Inflammatory
Bowel Disease section at HUVM, and a good-quality video of the colonoscopy, lasting
at least 60 seconds and with an acceptable bowel preparation (Boston Bowel Preparation
Score [BPPS] ≥ 4). Exclusion criteria were pregnancy, New York Heart Association > 2,
severe chronic obstructive pulmonary disease, refusal to sign the form of consent
for the colonoscopy, inadequate cathartic preparation (BPPS < 4 points), impenetrable
colonic stenosis in sigma, suspicion of toxic megacolon, and an extensive intestinal
resection (subtotal colectomy).
Patients with UC were included beginning in April 2019. A prospective follow-up was
carried out to analyze their clinical evolution. Clinical data and data on the rest
of the variables were gathered from the electronic digitized clinical histories in
the DIRAYA program of the Andalusian Public Health System (SSPA) and tabulated according
to the Mayo Clinic. The analysis was performed carried within an interval of ± 7 days
after colonoscopy, and at 6 and 12 months.
Information on patient treatment when colonoscopy was carried out was gathered from
the electronic digitized clinical histories in the DIRAYA program. Treatment continuation
or change was assessed by the gastroenterologist in charge of the patient, according
to the colonoscopy report. The changes in treatment were correlated with the clinical
evolution of the patients at 6 and 12 months. The endoscopists who evaluated the videos,
(A, B and C) had no prior knowledge of either the clinical situation or the treatment
of the patients.
Data analysis
For the descriptive analysis, the following values were calculated: absolute frequency
(N), relative frequency (%), average values, typical deviation (T.D.), minimum, maximum,
and 25 %, 50 %, and 75 %. For the inference analysis, the confidence level was considered
to be of 95 %, thus, the experimental P value was compared to the level of significance
of 5 %.
To analyze the qualitative variables, a chi-squared test was used. To carry out the
analysis between the categorical variable and another quantitative variable, it was
necessary to know the most adequate type of test to be used according to the data.
Normality tests were done by means of the Shapiro-Wilk or Kolmogorv-Smirnov test.
The tests carried out in the study were as follows. A Mann-Whitney U test was used
for independent samples to analyze the change in treatment and its correlation with
the endoscopic indices. The Friedman two-dimensional variance analysis was used to
assess the evolution of patients according to the MCS.
Also, to indicate the intensity and the tendency in the relationship between two quantitative
variables, a co-relational test was carried out. Pearson coefficient correlation analysis
or the Spearman coefficient correlation analysis was used, depending on the behavior
of those variables.
Finally, to evaluate concordance among the different measurements of the endoscopists,
both Kappa and weighted Kappa scores were calculated, as well as the intraclass correlation
coefficient.
This study was approved by the Research Ethics Board of the Virgen Macarena Hospital.
Results
There were 74 patients included in the study, three of whom were excluded, due to
the fact that they did not have a histological diagnosis; another three, due to a
failure in fulfilling the criteria of inclusion, as well as having Crohn’s disease,
and another due to the fact that the video was invalid ([Fig. 1]). Therefore, the final sample was of 67 patients, whose demographic characteristics
can be found in [Table 1].
Fig. 1 Flow diagram.
Table 1
Demographic characteristics.
|
Demographic characteristics
|
N = 67 (%)
|
|
Sex
|
|
|
31 (46.3 %)
|
|
|
36 (53.7 %)
|
|
Mean age (DE)
|
51.2 (16.7)
|
|
Tobacco
|
|
|
16 (23.9 %)
|
|
|
48 (71.6 %)
|
|
|
3 (4.5 %)
|
|
Extent
|
|
|
26 (38.8 %)
|
|
|
30 (44.8 %)
|
|
|
11 (16.4 %)
|
|
Mean Mayo Clinic index (DE)
|
3.07 (2.54)
|
|
Treatment
|
|
|
1 (1.5 %)
|
|
|
31 (46.3 %)
|
|
|
10 (14.9 %)
|
|
|
12 (17.9 %)
|
|
|
7 (10.4 %)
|
|
|
6 (8.9 %)
|
DE, disease extent; UC, ulcerative colitis.
Interobserver correlation of MES, UCEIS and UCCIS
The weighted Kappa index between endoscopists A and B for MES was 0.8 (good); between
A and C 0.52 (acceptable) and between B and C 0.49 (acceptable). The relationship
between the index of endoscopists A and B according to the Spearman correlation coefficient
was 0.88; between A and C 0.85 and between B and C 0.80. In [Fig. 2], the distribution of the MES results according to the different endoscopists is
shown. If the index of central values (1,2) was compared to extreme values (0,3) a
Kappa index of 0.69 was reached between endoscopists A and B, 0.24 between endoscopists
A and C, and 0.23 between endoscopists B and C.
Fig. 2 Distribution of MES according to the different endoscopists.
For the UCEIS, the interclass correlation coefficient of average values was 0.92 among
the three endoscopists (CI 95 % 0.83–0,96). The interclass correlation coefficient
between endoscopists A and B was 0.94 (excellent); between A and C 0.89 (excellent);
and between B and C 0.92 (excellent). The relationship between the index of endoscopists
A and B according to Spearman’s correlation coefficient was 0.87; between A and C
0.82 and between B and C 0.87. In [Fig. 3] the distribution of UCEIS values according to the different endoscopists is observed.
If the values were divided into two subgroups, the extreme values (UCEIS 0, 1 and
8) and the central values (2, 3, 4, 5, 6 and 7) of the index, a weighted Kappa of
0.71 was obtained between endoscopists A and B; 0.39 between A and C and 0.45 between
endoscopists B and C.
Fig. 3 Distribution of the Ulcerative Colitis Endoscopy Index of Severity according to the
different endoscopists.
For the UCCIS an interclass correlation coefficient of 0.96 between the three endoscopists
was reported, (CI 95 % 0.94–0.97) and on pairing the endoscopists the result was the
following: 0.99 between endoscopists A and B; 0.91 between A and C and 0.91 between
endoscopists B and C. The relationship between endoscopists A and B according to Spearman’s
correlation coefficient was 0.97; 0.85 between A and C and 0.86 between B and C. In
[Fig. 4] the distribution of UCCIS values according to the different endoscopists is shown.
Comparing the central values (2, 3, 4, 5, 6, 7, 8, 9) from the extreme values (0,
1, 10) we obtained a Kappa index of 0.85 between endoscopists A and B; 0.36 between
A and C and 0.34 between endoscopists B and C.
Fig. 4 Distribution of Ulcerative Colitis Colonoscopy Index of Severity according to the
different endoscopists.
Correlation of the endoscopic scores with the clinical situation of the patient and
changes in treatment
The mean MCS in the patients on carrying out the colonoscopy was 3.1 (SD ± 2.54) with
an average of 3 (CI 1–5), at 6 months it was 1.47 (SD ± 1.80) 1 (CI 0–3) (P = 0.008) and at 12 months it was 0.94 (SD ± 1.27) 1 (CI 0–1) (P = 0.003). The correlation between the three endoscopic indices and the patients’
clinical situation (according to the MCS) is found in [Table 2].
Table 2
Correlation between the endoscopic index and the Mayo Clinic Score.
|
Spearman correlation coefficient
|
|
Endoscopist A
Mayo Clinic
|
Endoscopist B
Mayo Clinic
|
Endoscopist C
Mayo Clinic
|
|
MES
|
0.53
|
0.53
|
0.60
|
|
UCEIS
|
0.56
|
0.65
|
0.63
|
|
UCCIS
|
0.66
|
0.66
|
0.61
|
MES, Mayo Endoscopy Score; UCEIS, Ulcerative Colitis Endoscopy Index of Severity;
UCCIS, Ulcerative Colitis Colonoscopy Index of Severity.
In 34.3 % of the patients, the result of the colonoscopy led to changes in treatment,
of whom 87 % underwent an intensified treatment, and in 13 %, the treatment was withdrawn
or de-intensified. The median basal MCS in those patients who had a change in their
treatment after colonoscopy was 4.63 (SD ± 2,36), with an average of 4.5 (CI 3–6.75).
While there were 1.59 patients (SD ± 2.01), an average of only one patient (CI 0–3)
had no change in treatment (P < 0.001). An MCS ≥ 2 was present in 95.8 % of patients whose treatment was changed
after colonoscopy (P < 0.001). In the group of patients who underwent a change in their treatment, the
MCS decreased from 4.5 at baseline to 3 points at 6 months (P = 0.075) and 1 point at 12 months (P = 0.007). In those patients who had no change in treatment, the MCS was on average
1 at baseline, 0 at 6 months, and 0 at 12 months (the difference between the baseline
and that at 12 months was not statistically significant, P = 0.119).
The relationship between the change in treatment after colonoscopy and the findings
according to the different endoscopists was studied. These results are shown in [Table 3].
Table 3
Relationship between the change in treatment after carrying out colonoscopy and findings
per endoscopist according to the MES, UCEIS and UCCIS indices.
|
Endoscopist A
|
Endoscopist B
|
Endoscopist C
|
|
MES 3. Change of treatment
|
66.7 % (P = 0.008)
|
75.0 % (P = 0.002)
|
92.9 % (P = 0.018)
|
|
MES 0. Maintenance of treatment
|
88.2 % (P = 0.008)
|
93.3 % (P = 0.002)
|
55.6 % (P = 0.018)
|
|
UCEIS ≥ 3. Change of treatment
|
58.3 % (P = 0.012)
|
62.5 % (P = 0.002)
|
79.2 % (P = 0.014)
|
|
UCEIS < 3. Maintenance of treatment
|
75.0 % (P = 0.012)
|
78.1 % (P = 0.002)
|
53.1 % (P = 0.014)
|
|
UCCIS ≥ 3. Change of treatment
|
79.2 % (P = 0.001)
|
75.0 % (P < 0.001)
|
100 % (P < 0.001)
|
|
UCCIS < 3. Maintenance of treatment
|
65.6 % (P = 0.001)
|
75.0 % (P < 0.001)
|
40.6 % (P < 0.001)
|
MES, Mayo Endoscopy Score; UCEIS, Ulcerative Colitis Endoscopy Index of Severity;
UCCIS, Ulcerative Colitis Colonoscopy Index of Severity.
Discussion
To diagnose UC, it is essential to obtain clinical, biochemical, endoscopic, and pathologic
data. Thus, colonoscopy plays an essential role in making a differential diagnosis
in patients with UC, and it allows us to visualize and assess the severity of the
disease, help plan therapeutic management and estimate the disease’s prognosis. However,
it is not easy to make a correct objective assessment of the mucosa, let alone have
all of the endoscopists grade the severity of the illness in the same way.
In a recent review by Vashist et al [6], the different studies assessing the interobserver correlation of the indices were
analyzed. The Kappa index was reported to vary on MES from 0.45 to 0.75 (acceptable
to good). An interobserver correlation with a Kappa index of 0.53 (CI 95 %: 0.47–0.56)
for expert endoscopists, and 0.71 (CI 95 %:0.67–0.76) for non-expert endoscopists
was observed [13]. In our study, a good correlation in MES was found, with a weighted Kappa index
of between 0.8 (good) and 0.49 (acceptable), as reported in other papers.
On the other hand, dividing variables between extreme values and central values, it
has been observed that this correlation decreases to 0.23 to 0.69. Similar findings
were reported in the study by Fernándes et al [14], in which the correlation of MES was 0.47 (CI 95 % 0.41–0.54), yet when only Mayo
0 was analyzed, this correlation increased to 0.89 (CI 95 % 0.73–1). This points to
the difficulty of this index to catalog endoscopic findings within the central subgroup
(1 and 2) and the extremes (0 and 3). Indeed, this is in line with the study by Ikeya
et al [5], which observed that the UCEIS was better than MES in detecting subtle changes in
the mucosa, probably due to the fine line between the central categories.
In the study by Travis et al [7], every item assessed in the UCEIS index was analyzed. The weighted Kappa index for
the reliability value was 0.3 (not acceptable), while it was 0.45 (acceptable) for
the values of erosions and ulcers. Later, in another study by the same author, the
interobserver correlation for the same index was analyzed. The weighted Kappa index
score was 0.47 (acceptable) (CI 95 % = 0.46, 0.49) and 0.47 (acceptable) (CI 95 % = 0.44,
0.50) for blinded and non-blinded readers, respectively [15]. In our study, the interobserver variability of the index was analyzed globally,
and a good correlation was found (0.92) with good scores for Rho Spearman (between
0.82 and 0.87). This indicates that there was good concordance among the three endoscopists
regarding assessing the mucosa of the patients with UC. The improvement in our results
is probably due to the fact that a general analysis was carried out, as opposed to
subgroups within said index, and its different subsections (mucosa assessment, reliability,
and erosions/ulcers), on the one hand, and, on the other, the number of patients included
in each study (41 in the study by Travis versus 67 in ours).
Concerning the general analysis of the correlation of the UCEIS index and its results,
when we analyzed the interobserver correlation differentiating central values (2,
3, 4, 5, 6, 7) to extreme values (0, 1, 8), the results dropped quite significantly,
with a Kappa index of 0.71 between endoscopists A and B (as the best result) and 0.39
between endoscopists A and C.
In the review carried out by Vashist et al [6], the correlation for the UCCIS index was also analyzed, with results taken from
other studies ranging from 0.56 to 0.88. Our result was somewhat higher, with an interclass
correlation of 0.96 (CI 95 % 0.94–0.97). The fact that this is an index used less
in clinical practice probably had an effect on this result, as indeed did the fact
that the endoscopists carried out the assessment using this index after the parameters
taken into account within the index were explained.
In the 2011 study by Daperno et al [16], in which 171 gastroenterologists assessed five different videos of colonoscopies
before and after receiving specific training for the MES, it was reported that the
correlation improved after the training from 0.45 to 0.71. The gastroenterologist’s
assessment will have an effect on the decisions that will be taken, as indeed we have
seen in our study, in which 34.3 % of the patients underwent a change in their treatment
after the specialist saw the results. Thus, it is extremely important that the gastroenterologists
who assess this type of patient should be properly trained.
Regarding the relationship that exists among the endoscopic indices and the clinical
situation of a patient, according to our results, all three indices (MES, UCEIS and
UCCIS) have a good correlation, which ranged from 0.53 to 0.65. This is similar to
findings reported in literature, such as the study by Fluxá [17] which showed a moderate correlation for MES with a patient’s clinical situation.
The study by Samuel et al is another example for UCCIS [10]. The index correlation was carried out using the clinical scale SCCAI and the CAI
scale or the Rachmilewitz score, with results similar to ours (0.62; P < 0.0001 and 0.52; P < 0.001, respectively). However, in the study by Travis that was published in 2015,
a higher correlation between the index and the symptomatology of a patient was reported
(with an average of 0.899 and 0.933 in the blinded and non-blinded groups, respectively)
[15].
In 34.3 % of patients in our cohort, the result of colonoscopy led to changes in treatment,
showing a drop by 4 points throughout the 12-month-follow-up in the MCS (from 5 at
base level to 1 at 12 months), which was statistically significant (P = 0.007). Said change in treatment was established after seeing the endoscopic result,
which in itself reflects the importance of the examination and the impact it may have
on a patient’s symptoms and improvement. Likewise, patients who did not undergo any
change in their treatment, had an average baseline MCS of 1 and 0 at 6 and 12 months
(P = 0.03). Also, according to the different endoscopists, more than 70 % to 80 % of
the patients presented an MES of 0, a UCEIS < 3 and a UCCIS < 3, thus showing the
clinical stability after 1 year of patients who had no change in treatment after the
result of the colonoscopy.
Therefore, the importance of not only carrying out a colonoscopy, but also its correct
assessment, is clearly evident. It represents a detailed study which allows us to
grade the illness, carry out a more adequate and objective follow-up, as well as decide
on therapeutic modifications that help improve and maintain the patient’s condition.
In this way, as demonstrated in studies such as that by Barreiro et al, mucosal healing
improves the evolution of the illness [18].
As far as we know, our study is the first to directly compare the three endoscopic
indices used to assess UC activity. Our results allow us to demonstrate that the UCEIS
and the UCCIS indices are superior to MES, regarding interobserver correlation. Thus,
although their interpretation could be considered as more complex, we consider they
should be used more in clinical practice.
Our study, however, does have some limitations worthy of analysis. On the one hand,
not all the videos showed a complete examination. This could indeed lead to an underestimation
of the stage of the illness. Nonetheless, the three endoscopists did receive the same
video, thus, the assessment was carried out on the same images of the mucosa. Moreover,
the quality of the images on screen live is better than on the video, which could
indeed influence their assessment. On the other hand, not all of the patients had
good or excellent cathartic preparation with a BPPS of 8 to 9. This could impede better
visualization of the colonic mucosa and certain patterns, for example, the vascular
pattern. We consider that the fact that all of the patients were from the same center
should not be taken as a limitation; however, the fact that all of the endoscopists
were from the same center could indeed be considered as such or as a methodological
bias within the study. A histological correlation was not carried out. Indeed, it
is true to say that in some recent studies, the histological healing of the mucosa
has been the aim. Up until now, however, this has not been proven to be efficient.
What is more, there is no standardized point system for this. Finally, calprotectine
and its correlation with the endoscopic findings were not investigated. This certainly
would have been interesting; however, at the beginning of the study, we did not have
access to said test in our center, therefore, we were unable to carry it out.
Conclusion
In conclusion, we consider that there is an adequate, but not perfect, correlation
among the different endoscopists for MES, UCEIS and UCCIS (the latter being the best).
These indices have an acceptable correlation with the clinical situation of a patient
and their results affect therapeutic decisions. Indeed, an improvement was reported
in patients who were analyzed at 12 months.
