Performance measures for lower gastrointestinal endoscopy: a European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Initiative

Abbreviations

ADR: adenoma detection rate

ASGE: American Society for Gastrointestinal Endoscopy

CI: confidence interval

CRC: colorectal cancer

EPAGE: European Panel on the Appropriateness of Gastrointestinal Endoscopy

ESGE: European Society of Gastrointestinal Endoscopy

FIT: fecal immunochemical test

FOBT: fecal occult blood test

ISFU: Importance, Scientific acceptability, Feasibility, Usability

LGI: lower gastrointestinal tract

LST: laterally spreading tumor

PICO: population/patient; intervention/indicator; comparator/control; outcome

PDR: polyp detection rate

QIC: Quality Improvement Committee

UEG: United European Gastroenterology

Introduction

The European Society of Gastrointestinal Endoscopy (ESGE) and United European Gastroenterology (UEG) have identified quality of endoscopy as a major priority. We described our rationale for this priority in a recent manuscript that also addressed the methodology of the current quality initiative process [1].

Because of the variation in physicians’ performance and the introduction of nationwide colorectal cancer (CRC) screening programs, lower gastrointestinal (LGI) endoscopy was the first area of endoscopy to address quality [2] [3] [4]. Over more than a decade, several potential measures of quality in LGI endoscopy have been identified. In consequence, many professional societies have published recommendations on performance measures for LGI endoscopy [5] [6] [7]. These recommendations are however numerous (44 different performance measures) [5] [6] [7], country specific, and not always evidence based, which has limited their wider adoption in Europe.

The aim of the ESGE LGI working group was to identify a short list of key performance measures for LGI endoscopy that were widely applicable to endoscopy services throughout Europe. This list would ideally consist of performance measures with the following requirements: proven impact on significant clinical outcomes or quality of life; a well-defined, reliable, and simple method/approach for measurement; susceptibility for improvement; and application to all levels of endoscopy services.

This paper reports the agreed list of key performance measures for LGI endoscopy and describes the methodological process applied in the development of these measures.

Methodology

We previously described the multistep process for producing such performance measures [1]. In brief, at the United European Gastroenterology Week in 2014, we used a modified Delphi consensus process to develop quality measures in the following domains: pre-procedure, completeness of procedure, identification of pathology, management of pathology, complications, procedure numbers, patient experience, and post-procedure [1] [8] [9]. We decided to have one or two key performance measures for each quality domain.

In order to identify key performance measures, we first created a list of all possible performance measures for LGI endoscopy through email correspondence and teleconferences that took place between December 5, 2014 and February 7, 2015. All possible performance measures that were identified by this process were then structured using the PICO framework (where P stands for Population/Patient; I for Intervention/Indicator; C for Comparator/Control, and O for Outcome) to inform searches for available evidence to support the performance measures. This process resulted in 38 PICOs. Detailed literature searches were performed by an expert team of methodologists and yielded results for 29 PICOs (see Supporting Information; available online). Working group members also identified additional articles relevant for the performance measures in question.

The PICOs and the clinical statements derived from these were adapted or omitted during iterative rounds of comments and suggestions from the working group members during the Delphi process. The evolution and adaptation of the different PICOs and clinical statements during the Delphi process can be reviewed in the Supporting Information. The domain addressing the competence of endoscopists’ quality (including procedure numbers), along with its associated PICOs and clinical statements, was moved for future initiatives.

In total, working group members participated in a maximum of three rounds of voting to agree on performance measures in predefined domains and their respective thresholds, as discussed below. Statements were discarded if agreement was not reached over the three voting rounds. The agreement that is given for the different statements refers to the last voting round in the Delphi process. The key performance measures were distinguished from the minor performance measures based on the ISFU criteria (Importance, Scientific acceptability, Feasibility, Usability, and comparison with competing measures), and expressed by mean voting scores.

The performance measures are displayed in boxes under the relevant quality domain. Each box describes the performance measure, the level of agreement during the modified Delphi process, the grading of available evidence (the evidence was graded according to the Grading of Recommendations Assessment, Development and Evaluation [GRADE] system) [10], how the performance measure should be measured, and recommendations supporting its adoption. The boxes further list the measurement of agreement (scores), the desired threshold, and suggestions on how to deal with underperformance.

The minimum number needed to assess whether the threshold for a certain performance measure is reached can be calculated by estimating the 95 % confidence intervals (CIs) around the predefined threshold for different sample sizes [8] [9] [11]. For the sake of practicality and to simplify implementation and auditing, we suggest that at least 100 consecutive procedures (or all, if < 100 performed) should be measured to assess a performance measure. Continuous monitoring should however be the preferred method of measurement.

Performance measures for lower gastrointestinal endoscopy

The evidence derived by the literature search group and input from the working group members were used to formulate a total of 34 clinical statements addressing 27 potential performance measures grouped into eight quality domains. Over the course of two voting rounds, consensus agreement was reached for 18 statements regarding 14 potential performance measures (agreement in both voting rounds). The remaining 16 statements were again rephrased and subjected to a third and final voting round, with a further four statements being accepted. In total, 22 statements regarding 18 performance measures were accepted after three voting rounds. Over the course of voting, we decided that the quality domain on competence of endoscopists (including three accepted statements and three performance measures) would be discarded from these guidelines and left for future initiatives. Therefore, a final total of 15 performance measures (19 statements) attributed to seven quality domains were accepted for these guidelines (see [Fig. 1]). The entire process of performance measure development can be reviewed in the Supporting Information. The statement numbers correspond to those used in Supporting Information.

Fig. 1 The domains and performance measures chosen by the working group. N/A, not available.

We used the highest mean voting scores to identify one key performance measure for each of the seven quality domains ([Fig. 1]). The remaining performance measures were considered minor performance measures. In the management of pathology domain, there were two performance measures (“Appropriate polypectomy technique” and “Tattooing resection sites”) that had similar voting scores. We decided to select “Appropriate polypectomy technique” as the key performance measure for this domain, based on its wider usability and better feasibility.

All performance measures were deemed valuable by the working group members and were obtained after a rigorous process, as described above. From a practical viewpoint, it may however be desirable to implement the key performance measures first in units that are not monitoring any performance measures at this time. Once a culture of quality measurement (with the aim of improving practice, outcomes, and patient experience) is accepted and software is available, the minor performance measures may then further aid the monitoring of quality in LGI endoscopy. The use of appropriate endoscopy reporting systems is key to facilitate data retrieval on identified performance measures [12].

All of the performance measures are presented below using the descriptive framework developed by the Quality Improvement Committee (QIC) and a short summary of the evidence for the ISFU criteria. The performance measures are listed according to the domain to which they were attributed (for a summary, see [Fig. 1]).

1 Domain: Pre-procedure

Key performance measure	Rate of adequate bowel preparation
Description	The percentage of patients with an adequately prepared bowel
Domain	Pre-procedure
Category	Process
Rationale	It has been shown that the quality of bowel preparation affects the rates of cecal intubation and adenoma detection Inadequate bowel preparation results in increased costs and inconvenience as the examination has to be rescheduled or alternative investigations have to be organized
Construct	Denominator: Patients undergoing colonoscopy Numerator: Patients in the denominator with adequate bowel preparation (assessed with a validated scale, preferably the Boston Bowel Preparation Scale [BBPS; score ≥ 6], Ottawa Scale [score ≤ 7], Aronchick Scale [excellent, good or fair]) Exclusions: Emergency colonoscopies Calculation: Proportion (%) Level of analysis: Service and individual level Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a yearly audit of a sample of 100 consecutive LGI endoscopies
Standards	Minimum standard: ≥ 90 % Target standard: ≥ 95 % Bowel preparation quality, assessed using a validated scale, such as the BBPS, the Ottawa Scale, or the Aronchick Scale, should be included in every colonoscopy report If the minimum standard is not reached, analysis of the factors influencing bowel preparation should be performed on a service level (information given to patients, dietary restrictions, cleansing agent used, colonoscopy timing) After evaluation and adjustment, close monitoring should be performed with a further audit within 6 months
Consensus agreement	100 %
PICO	1.1 – 1.2 (see Supporting Information)
Evidence grading	Moderate quality evidence

The acceptance of this performance measure is based on agreement with the following statements:

• In patients undergoing screening or diagnostic colonoscopy, bowel preparation quality should be recorded using a validated scale with high intraobserver reliability. (Statement number N1.1) Agreement: 100 %

• A service should have a minimum of ≥ 90 % procedures and a target of ≥ 95 % procedures with adequate bowel preparation, assessed using a validated scale with high intraobserver reliability. (N1.2) Agreement: 100 %

The quality of bowel preparation is important for the efficacy of colonoscopy. As pointed out in the ESGE guidelines on bowel preparation for colonoscopy [13], the quality of bowel preparation is associated with two other important performance measures for colonoscopy, namely adenoma detection rate (ADR) and cecal intubation rate [14]. Suboptimal bowel preparation results in further costs and inconvenience because the examination has to be repeated or an alternative examination has to be arranged [15].

To determine the scientific acceptability of measuring bowel preparation quality, we focused on the performance of different bowel preparation scales and the quantification of adequacy of bowel preparation. There were no direct comparisons of performance between the bowel preparation scales (see Supporting Information). Three bowel preparation scales have undergone comprehensive validation and have shown sufficient validity and reliability: the Boston Bowel Preparation Scale (BBPS) [16], the Ottawa Scale [17], and the Aronchick Scale [18]. The BBPS is the most thoroughly validated scale and should be the preferred one [19]. There were no significant differences between intermediate and high quality bowel preparation (regardless of the scale used) in terms of the detection rates for adenomas or advanced adenomas (see Supporting Information) [20]. Therefore, adequate bowel preparation may be defined as: BBPS ≥ 6, Ottawa Scale ≤ 7, or Aronchick Scale excellent, good, or fair. The adoption of validated scales for bowel preparation quality assessment has been proven to be feasible in routine practice [21].

The proposed minimum ( ≥ 90 %) and target standard (≥ 95 %) rates of adequate bowel preparation were based on values reported in recent population-based studies [22] [23] [24] and on randomized clinical trials of split-dose bowel cleansing regimens [25] [26], respectively.

Minor performance measure	Time slot allotted for colonoscopy
Description	Time allotted for each colonoscopy in daily schedule
Domain	Pre-procedure
Category	Structure
Rationale	Colonoscopy needs adequate time allocated for the entire procedure (including discussion with the patient, sedation, insertion, withdrawal, and therapy) Time pressure due to inadequate time slots may impair colonoscopy quality
Construct	Denominator: Number of colonoscopies scheduled in an outpatient colonoscopy list (session) Numerator: Outpatient colonoscopy list (session) working hours Exclusions: Emergency colonoscopy Calculation: Average time length (minutes) Level of analysis: Service level Frequency: Two-yearly check of booking log
Standards	Minimum standard: 30 minutes for clinical and primary screening colonoscopy; 45 minutes for colonoscopy following positive fecal occult blood testing Target standard: no target standard set If the minimum standard is not reached, a systematic approach to schedule modification should be applied
Consensus agreement	100 %
PICO	1.3 (see Supporting Information)
Evidence grading	No evidence

The acceptance of this performance measure is based on agreement with the following statement:

• Colonoscopy needs adequate time allocated for insertion, withdrawal, and therapy. Routine colonoscopy should be allocated a minimum of 30 minutes. Colonoscopies following positive fecal occult blood testing should be allocated a minimum of 45 minutes to allow for therapeutic intervention. (N1.3) Agreement: 100 %

There is some evidence that productivity pressure may negatively affect the quality of colonoscopy [27]. Although it has been shown that working behind schedule is not associated with lower ADRs [28], the effect of a very tight schedule on colonoscopy performance is unknown (see Supporting Information). The working group members suggested that 30 minutes and 45 minutes are minimum times that should be allotted for routine colonoscopy and colonoscopy after positive fecal occult blood testing (longer time to accommodate high prevalence of large polyps), respectively. These values correspond well with mean total procedure times for colonoscopy reported in recent studies [29] [30].

Minor performance measure	Indication for colonoscopy
Description	The colonoscopy report should include an explicit indication for the procedure, categorized according to existing guidelines on appropriate use of colonoscopy (the ASGE or the EPAGE II guidelines)
Domain	Pre-procedure
Category	Process
Rationale	Colonoscopies with an appropriate indication are associated with higher diagnostic yield for relevant lesions than colonoscopies without an appropriate indication
Construct	Denominator: All colonoscopies performed Numerator: Colonoscopies with appropriate and “uncertain” indication (according to ASGE or EPAGE II) Exclusions: None Calculation: Proportion (%) Level of analysis: Service level Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a yearly audit of a sample of 100 consecutive LGI endoscopies
Standards	Minimum standard: ≥ 85 % Target standard: ≥ 95 % All reports from colonoscopies performed should include an appropriate indication according to the ASGE or EPAGE II guidelines When performed for screening, the colonoscopy report should state this and it must be ensured that the subject meets the criteria for screening A colonoscopy reporting system with a drop-down menu for indication is ideal to ensure proper recording of the indication and later auditing If the minimum standard is not met, a systematic approach to validate the appropriateness of colonoscopies should be applied (i. e. validation of appropriateness before colonoscopy scheduling) After evaluation and adjustment, close monitoring should be performed with a further audit within 6 months
Consensus agreement	93.8 %
PICO	1.4 (see Supporting Information)
Evidence grading	Moderate quality evidence

The acceptance of this performance measure is based on agreement with the following statement:

• For audit purposes, the colonoscopy report should include an explicit indication for the procedure, categorized according to existing guidelines on appropriateness of colonoscopy use. (N1.4) Agreement: 93.8 %

Appropriate referrals for colonoscopy may help to optimize the use of limited resources and protect patients from the potential harms of unnecessary invasive procedures. Colonoscopies with an appropriate indication are associated with significantly higher diagnostic yields for cancer and other relevant lesions than colonoscopies without an appropriate indication [31] [32] [33] [34]. The American Society for Gastrointestinal Endoscopy (ASGE) and the European Panel on the Appropriateness of Gastrointestinal Endoscopy (EPAGE) II guidelines on the appropriateness of colonoscopy use [35] [36] consistently show 67 % – 96 % sensitivity and 13 % – 40 % specificity for the detection of relevant findings (see Supporting Information) [31] [32] [33] [34].

The proposed minimum standard of appropriate indication for colonoscopy ( ≥ 85 %) was based on values achieved in studies from academic and non-academic centers over the last 5 years [32] [33] [37]. The use of appropriate endoscopy reporting systems with a drop-down menu for indication is key to facilitate data acquisition for this performance measure [12].

2 Domain: Completeness of procedure

Key performance measure	Cecal intubation rate
Description	The percentage of colonoscopies reaching and visualizing the whole cecum and its landmarks
Domain	Completeness of procedure
Category	Process
Rationale	Whole bowel examination is a prerequisite for complete and reliable inspection of the mucosa in search of lesions A low cecal intubation rate is associated with an increased risk of interval colorectal cancer Incomplete colonoscopy leads to increased costs and inconvenience as the examination has to be repeated
Construct	Denominator: All screening or diagnostic colonoscopies Numerator: Procedures in the denominator that report reaching the cecum (documented in written form and by photo/video) Exclusions: Therapeutic procedures with no indication to reach the cecum Emergency colonoscopies Calculation: Proportion (%) Level of analysis: Service and endoscopist level Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a yearly audit of a sample of 100 consecutive LGI endoscopies
Standards	Minimum standard: ≥ 90 % Target standard: ≥ 95 % Cecal intubation, meaning complete visualization of the whole cecum and its landmarks, should be documented in a written report, as well as with photo or video documentation If the minimum standard is not reached for an individual endoscopist, additional training should be offered If the minimum standard is not reached on a service level, an audit to determine the cause should be performed After evaluation and adjustment, close monitoring should be performed with a further audit within 6 months
Consensus agreement	97.9 %
PICO	2.1 – 2.3 (see Supporting Information)
Evidence grading	Moderate quality evidence

The acceptance of this performance measure is based on agreement with the following statements:

• Complete colonoscopy requires cecal intubation with complete visualization of the whole cecum and its landmarks. (N2.1) Agreement: 100 %

• A service should have a minimum unadjusted cecal intubation rate of ≥ 90 % and a target rate of ≥ 95 % as a measure of the completeness of colonoscopy examination. (N2.2) Agreement: 93.8 %

• Complete colonoscopy (cecal intubation) should be documented both in written form and in a photo or video report. (N2.3) Agreement: 100 %

Cecal intubation is a prerequisite for complete visualization of the colorectum. Cecal intubation must be confirmed with photo or video documentation. Clear cecal image documentation is associated with a higher polyp detection rate (PDR) [38]. For the purpose of colorectal neoplasia detection, terminal ileum intubation is useful only to confirm completion of the colonoscopy when classic cecal landmarks are not confidently seen [39].

Failed cecal intubation results in further costs and inconvenience as the examination must be rescheduled or an alternative investigation organized. A cecal intubation rate < 80 % is associated with significantly higher risks of proximal and distal interval CRCs when compared with higher completion rates [40]. Adjustment of the cecal intubation rate for inadequate bowel preparation or impassable strictures makes the measurement less feasible and harbors the risk of gaming. In recent large population-based studies, unadjusted cecal intubation rates always exceeded 90 % and were usually above 95 % [22] [41] [42] [43] [44] [45]. The effect of raising the target standard beyond the minimum of 95 % is uncertain.

3 Domain: Identification of pathology

Key performance measure	Adenoma detection rate (ADR)
Description	Percentage of colonoscopies with at least one adenoma identified
Domain	Identification of pathology
Category	Process
Rationale	ADR reflects adequate inspection of the bowel mucosa ADR is associated with interval CRC and CRC death, with improvement in the ADR lowering the risk for CRC and CRC death
Construct	Denominator: All colonoscopies in patients aged 50 years or older Numerator: Procedures in the denominator in which at least one adenoma was identified Exclusions: Emergency colonoscopy Endoscopy with a specific therapeutic indication, including work-up of a previously detected lesion or follow-up of disease activity in inflammatory bowel disease Calculation: Proportion (%) Level of analysis: Service and endoscopist level Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a yearly audit of a sample of 100 consecutive LGI endoscopies
Standards	Minimum standard: ≥ 25 % Target standard: no current target standard defined ADR should be monitored in all settings (screening and out-patient), which requires routine access to histopathology reports If the minimum standard is not met by an individual endoscopist, appropriate feedback followed by a competence assessment (with special consideration of withdrawal time and technique) should be given If the minimum standard is not met on a service level, comprehensive training for the center leader should be considered
Consensus agreement	100 %
PICO	3.1 – 3.4 (see Supporting Information)
Evidence grading	Moderate to high quality evidence

The acceptance of this performance measure is based on agreement with the following statement:

• Adenoma detection rate should be used as a measure of adequate inspection at screening or diagnostic colonoscopy in patients aged 50 years or more. (N3.1) Agreement: 100 %

The detection and removal of adenomas, which are major precursor lesions for CRC, is seen as a key aspect of CRC prevention. However, there is a wide variation between endoscopists in terms of their skills at detecting adenomas, expressed as the ADR [22] [43] [46] [47] [48]. ADR has been inversely associated with the risk of interval CRC [46] and CRC death [47]. A similar relationship with the incidence of distal interval CRC was confirmed for flexible sigmoidoscopy screening [49]. Of note, the detection rate of serrated polyps has been shown to strongly correlate with the ADR [43]. Although ADR is considered a surrogate for meticulous inspection of the colorectal mucosa, the correlation with other important, but non-neoplastic, findings has never been studied.

Several interventions, including education, creating awareness, feedback, and benchmarking on colonoscopy quality, have all helped to improve the ADR [50] [51] [52] [53]. Recently, it has been shown that an improved ADR translates to risk reductions for interval CRC and death, which closes the quality improvement loop [54].

It has been postulated that ADR has an inherent limitation of not measuring the total number of adenomas detected [41]. A potentially more accurate measure, namely number of adenomas per colonoscopy, has been proposed, but this was proven not to be superior to ADR in a recent study [55].

It is challenging to set the standards for ADR, especially in populations enriched with fecal occult blood test (FOBT)-positive patients. In a primary colonoscopy screening setting, a 1 % increase in ADR predicted a 3 % decrease in the risk of interval CRC within the observed ADR range of 7.35 % – 52.5 % [47]. In another study, an ADR above 24.6 % was associated with a reduced risk of interval CRC and subsequent death [54]. In recent population-based studies, a proposed minimum standard ADR of 25 % was met by the majority of endoscopists [22] [47] [51]. In fecal immunochemical test (FIT) positive-enriched populations, the minimum standard may need to be higher; however, the exact value is yet to be established.

Minor performance measure	Withdrawal time
Description	Time spent on withdrawal of the endoscope from cecum to anal canal and inspection of the entire bowel mucosa at negative (no biopsy or therapy) screening or diagnostic colonoscopy
Domain	Identification of pathology
Category	Process
Rationale	A mean withdrawal time of 6 minutes or longer was associated with higher ADRs and lower interval cancer rates as compared to shorter withdrawal times
Construct	Withdrawal time is measured from cecum to anal sphincter Denominator: Number of negative (no biopsy/therapy) screening or diagnostic colonoscopies Numerator: Sum of withdrawal time in colonoscopies included in the numerator Exclusions: Emergency colonoscopy Incomplete colonoscopy Calculation: Mean time in minutes Level of analysis: Endoscopist level Frequency: Measured only if the ADR is insufficient, using a sample of 100 consecutive colonoscopies
Standards	Minimum standard: mean 6 minutes Target standard: mean 10 minutes Time can be measured by different methods: stopwatch operated by a nurse, time stamp on photodocumentation of the cecum and rectum, length of video recording, or external device (this requires inclusion of the withdrawal time in the colonoscopy report) Withdrawal time should be measured only when the ADR is insufficient Feedback on mean withdrawal time should be given to endoscopists
Consensus agreement	87.5 %
PICO	3.6 (see Supporting Information)
Evidence grading	Moderate quality evidence

The acceptance of this performance measure is based on agreement with the following statement:

• A mean withdrawal time of at least 6 minutes should be used as a supportive measure of adequate identification of pathology at negative screening or diagnostic colonoscopy. (N3.6) Agreement: 87.5 %

Colonoscope withdrawal time provides information about the time that endoscopists spend identifying pathology. A mean withdrawal time of > 6 minutes has been associated with higher ADRs [56]. Although the association between withdrawal time and ADR was not observed in all studies [57], a recent large population-based analysis confirmed the positive relation between these two measures, with a 3.6 % absolute increase in ADR per minute increase in withdrawal time [24]. Importantly, the latter study also showed an inverse association between mean withdrawal time and the incidence of interval CRC [24]. The observed association was not linear and the risk of interval CRC leveled off at a mean withdrawal time of 8 minutes (the most significant difference was observed for the 6-minute cut-off). In another study, an increase in mean withdrawal time beyond 10 minutes had minimal effect on ADR [58]. Therefore, the minimum standard mean withdrawal time of 6 minutes and the target standard of 10 minutes are quite well defined.

Monitoring withdrawal time or institution policy on withdrawal time above a certain threshold showed inconsistent effects on ADRs [59] [60] [61]. The explanation could be that the variation in withdrawal technique is more important than the withdrawal time [62]. Therefore, it appears that the withdrawal time is particularly useful as a supportive tool when the observed ADR is less than the minimum standard of 25 % [63].

Minor performance measure	Polyp detection rate (PDR)
Description	Percentage of colonoscopies in patients aged 50 years or older in which at least one polyp was identified
Domain	Identification of pathology
Category	Process
Rationale	PDR reflects adequate inspection of bowel mucosa PDR correlates with ADR and polypectomy rate is weakly associated with interval CRC risk
Construct	Denominator: All screening and diagnostic colonoscopies in patients aged 50 years or older Numerator: Procedures in the denominator with at least one polyp identified Exclusions: Emergency colonoscopy Endoscopy with a specific therapeutic indication, including work-up of a previously detected lesion or follow-up of disease activity in inflammatory bowel disease Calculation: Proportion (%) Level of analysis: Service and endoscopist level Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a yearly audit of a sample of 100 consecutive LGI endoscopies
Standards	Minimum standard: 40 % Target standard: no current target standard defined PDR is an approximation of ADR and should only be used when there is limited access to histopathology reports; however, caution is needed because PDR is susceptible to gaming If the minimum standard is not met, there should be an attempt to obtain histopathology reports and calculate the ADR
Consensus agreement	84.6 %
PICO	3.1 (see Supporting Information)
Evidence grading	Low quality evidence.

The acceptance of this performance measure is based on agreement with the following statement:

• Polyp detection rate should be used as a measure of adequate inspection at screening or diagnostic colonoscopy in patients aged 50 years or more. (N3.5) Agreement: 84.6 %

PDR is a surrogate for ADR and is more feasible to measure as it does not require histological verification. In some studies, PDR has been shown to correlate well with ADR [64] [65] [66]; however, in others the correlation was poor for polyps in the distal colorectum [67] [68]. In one study, polypectomy rates of at least 25 % were associated with a significantly lower risk of proximal interval CRC [40]. In a recent study, PDR was found to be non-inferior to ADR in predicting the risk of interval CRC [55]. With an average adenoma to polyp detection quotient of 0.64, the minimum standard PDR was estimated at 40 %, which corresponds with an ADR of 25 % [66]. The detection of adenomas and non-neoplastic polyps are however associated, which may inflate the PDR [67]. The use of PDR instead of ADR could therefore be considered if there is limited availability of histopathology data, accepting the potential risks of gaming. We note that the increased pressure on quality may force endoscopists to detect and remove non-neoplastic lesions that would otherwise be undetected so as to inflate the rate of detection of “so-called” polyps.

4 Domain: Management of pathology

Key performance measure	Appropriate polypectomy technique
Description	Adequate resection technique of colorectal polyps includes biopsy forceps removal of polyps ≤ 3 mm in size, and snare (cold or with diathermy) polypectomy for larger polyps. Polyp size estimated by endoscopists has to be included in the endoscopy report
Domain	Management of pathology
Category	Process
Rationale	Inappropriate polypectomy technique increases the risk of incomplete polyp removal Incomplete polyp removal leads to further costs and inconvenience as the examination has to be repeated Incomplete polyp removal is also considered to contribute to the development of interval CRCs
Construct	Denominator: Polyps > 3 mm in size removed at colonoscopy (polyp size estimated by endoscopist) Numerator: Polyps in the denominator removed with snare polypectomy (cold or with diathermy) Exclusions: None Calculation: proportion (%) Level of analysis: Service and endoscopist Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a yearly audit of a sample of 100 consecutive LGI endoscopies
Standards	Minimum standard: ≥ 80 % Target standard: ≥ 90 % Colonoscopy reports must include information on polyp resection technique If the minimum standard is not met, the rate of complete polyp resection should be measured and feedback should be given to the endoscopist or service. Additional training on basic polypectomy technique should be considered After evaluation and adjustment, close monitoring should be performed with a further audit within 6 months
Consensus agreement	93.3 %
PICO	4.6 (see Supporting Information)
Evidence grading	Low quality evidence

The acceptance of this performance measure is based on agreement with the following statement:

• Adequate resection technique of small and diminutive colorectal polyps includes biopsy forceps removal of polyps ≤ 3 mm in size and snare polypectomy for larger polyps. (N4.6) Agreement: 93.3 %

Incomplete polypectomy is considered the cause for up to 25 % of interval CRCs [69] [70]. Incomplete resection of polyps 5 – 20 mm in size varies from 6.5 % to 22.7 % among endoscopists [71]; however, completeness of polyp resection is considered challenging to measure, and statements regarding this topic have not reached agreement in the current Delphi process (see Supporting Information).

Biopsy forceps resection of polyps 4 – 5 mm in size or larger has been shown to be inferior to snare techniques, with regard to completeness of resection [72] [73]. Therefore, the appropriate resection technique for colorectal polyps includes biopsy forceps removal of polyps ≤ 3 mm in size, and snare (cold or with diathermy) polypectomy for larger polyps. Despite this, in a recent large cohort study, it was demonstrated that 28.2 % of lesions ≥ 5 mm in size were resected using biopsy forceps instead of a snare technique [74]. Contrary to this, in a large study from the UK, over 90 % of polyps larger than 3 mm in size were removed using a snare [75].

There are insufficient data to set the minimum and target standards reliably, but the proposed values for the use of appropriate polypectomy techniques of ≥ 80 % and ≥ 90 %, respectively, seem relatively easy to achieve.

Minor performance measure	Tattooing resection sites
Description	In patients undergoing removal of colorectal non-pedunculated lesions 20 mm in size or larger, or with suspicious macroscopic features regardless of size, the resection site should be tattooed to improve future re-location of the resection site
Domain	Management of pathology
Category	Process
Rationale	Facilitates detection of the post-polypectomy site at surveillance colonoscopy or surgical resection
Construct	Tattooing the resection site of the abovementioned lesions should be applied in all cases. A service must provide appropriate equipment Denominator: Colonoscopies with removal of non-pedunculated lesions 20 mm in size or larger, or with suspicious macroscopic features regardless of size Numerator: Procedures in the denominator where the resection site was marked with a tattoo Exclusions: None Calculation: Proportion (%) Level of analysis: Service level Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a 3-yearly audit of all colonoscopies performed over a 3-month period
Standards	Minimum standard: Unknown Target standard: 100 % Every endoscopy report for procedures where removal of the abovementioned lesions was performed should include written information on tattooing the resection site If tattooing is not performed in all cases, feedback should be given to the service and all endoscopists
Consensus agreement	93.3 %
PICO	4.5 (see Supporting Information)
Evidence grading	Very low quality evidence

The acceptance of this performance measure is based on agreement with the following statement:

• In patients undergoing removal of colorectal lesions with a depressed component (0-IIc, according to the Paris classification) or non-granular or mixed-type laterally spreading tumors, located between the ascending and the sigmoid colon, the resection site should be tattooed to improve future re-location of the resection site. (N4.1) Agreement: 93.3 %

Colorectal lesions with a depressed component and non-granular or mixed-type laterally spreading tumors (LSTs) harbor an increased risk of malignancy [76] [77] [78]. Therefore, the site of endoscopic removal of these lesions often needs to be re-located to identify recurrence or to guide surgical management. It has been shown that tattooing significantly shortens the time to re-locate the resection site on endoscopy [79]. There is however no evidence that tattooing the resection site increases the rate of re-location of lesions (see Supporting Information). Preoperative tattooing using prepacked kits was proven to be a very effective method of tumor localization in laparoscopic surgery [80]. Moreover, some studies have shown that tattooing improves lymph node yield and facilitates the harvesting of suspicious lymph nodes during colorectal surgery [81] [82].

Although the accepted statement focused only on lesions with an increased risk of malignancy, for audit purposes it will be much more feasible to track the tattooing of resection sites for all lesions larger than 20 mm in size. These lesions are frequently removed piecemeal, which increases the risk of recurrence [83], and have a considerable risk of malignancy [84]. The minimum standard for tattooing resection sites is unknown.

Minor performance measure	Polyp retrieval rate
Description	Percentage of polyps removed that were retrieved for histopathology
Domain	Management of pathology
Category	Process
Rationale	The retrieval of polyps is required for histopathological diagnosis and is a prerequisite for recommendations on proper post-polypectomy surveillance interval
Construct	Denominator: Polypectomies of polyps > 5 mm Numerator: Polyps in the denominator that were retrieved for histopathology examination Exclusions: Removal of diminutive polyps ( ≤ 5 mm) Calculation: Proportion (%) Level of analysis: Service and endoscopist level Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a yearly audit of a sample of 100 consecutive LGI endoscopies
Standards	Minimum standard: ≥ 90 % Target standard: ≥ 95 % Colonoscopy reports must include information on non-retrieval of non-diminutive polyps If the minimum standard is not reached, feedback should be given on the importance of this performance measure
Consensus agreement	86.7 %
PICO	no PICO (see Supporting Information)
Evidence grading	Very low quality evidence

The acceptance of this performance measure is based on agreement with the following statement:

• The non-diminutive polyp retrieval rate should be monitored. A service should have a polyp retrieval rate of ≥ 90 %. (N4.2) Agreement: 86.7 %

The retrieval of polyps after endoscopic resection is a “sine qua non” requirement for histopathology examination. Histopathology examination guides further management including post-polypectomy surveillance. Diminutive polyps (≤ 5 mm in size) harbor a very low risk of cancer or advanced histology and are considered amenable for a resect-and-discard policy following in vivo optical diagnosis under strictly controlled conditions [85]. Furthermore, diminutive polyps are frequently removed using biopsy forceps, which makes their retrieval quite straightforward.

It has therefore been decided to monitor only the retrieval of polyps larger than 5 mm in size. Their retrieval is not only more important from the clinical perspective but also technically more difficult because it requires the transected polyp to be suctioned into a trap, ensnared, or grasped using a Roth net, so that it can be removed together with the endoscope [86] [87]. Even though the need for polyp retrieval seems obvious, it is unknown what the effect of substandard retrieval is on repeat colonoscopy rates or the appropriateness of recommended post-polypectomy surveillance.

The proposed minimum standard (≥ 90 %) and target standard ( ≥ 95 %) for polyp retrieval rate were based on values reported in recent large studies [41] [45] [88] [89]. Polyp retrieval rate seems feasible to measure and is amenable for improvement through education and competitive feedback [90].

Minor performance measure	Advanced imaging assessment
Description	In patients undergoing removal of colorectal lesions with a depressed component (0-IIc, according to the Paris classification) or non-granular or mixed-type laterally spreading tumors (LSTs), conventional or virtual chromoendoscopy should be used to improve delineation of the lesion margins and to predict the potential depth of invasion
Domain	Management of pathology
Category	Process
Rationale	Polyps with a depressed component (0-IIc) and non-granular or mixed type LSTs harbor a higher risk of submucosal invasion Such polyps frequently have indistinct borders, therefore better margin delineation is warranted Improved delineation and prediction of deep invasion may optimize management of these lesions
Construct	Advanced imaging assessment should always be used before an attempt to remove the abovementioned lesions. A service offering removal of these types of lesions must provide dedicated equipment Denominator: Colonoscopies with removal of lesions with a depressed component (0-IIc) or non-granular or mixed-type LSTs Numerator: Procedures in the denominator where virtual or conventional chromoendoscopy was used to improve delineation of the lesion margins (described in the report) Exclusions: None Calculation: Proportion (%) Level of analysis: Service and endoscopist Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a 3-yearly audit of all colonoscopies performed over a 3-month period
Standards	Minimum standard: Unknown Target standard: 100 % If the target standard is not met, feedback on the appropriate use of advanced imaging assessment is warranted At a service level, the availability of equipment should be analyzed and facilitated After evaluation and adjustment, close monitoring should be performed with a further audit within 6 months
Consensus agreement	93.3 %
PICO	4.4 (see Supporting Information)
Evidence grading	No evidence

The acceptance of this performance measure is based on agreement with the following statement:

• In patients undergoing removal of colorectal lesions with a depressed component (0-IIc, according to the Paris classification) or non-granular or mixed-type laterally spreading tumors, conventional or virtual chromoendoscopy should be used to improve delineation of lesion margins and predict potential depth of invasion. (N4.4) Agreement: 93.3 %

In 2014, the ESGE issued guidelines on advanced endoscopic imaging for the detection and differentiation of colorectal neoplasia in which it suggested the use of advanced endoscopic imaging for margin assessment and prediction of deep submucosal invasion in lesions with a depressed component (0-IIc) or non-granular or mixed-type LSTs [85]. The quality of evidence supporting these recommendations was considered very low and moderate for margin delineation and assessment of depth of submucosal invasion, respectively. Since then no new evidence with clinically relevant endpoints for the patients (incomplete resection, interrupted procedure, cancer detection) has been published to further support its use (see Supporting Information).

The availability, feasibility, and minimum standard of advanced imaging use, particularly in the community setting, are unknown. Colonoscopy services should set up structured monitoring and initiate audit to generate further evidence for advanced imaging.

Minor performance measure	Adequate description of polyp morphology
Description	The Paris classification should be routinely used to describe the morphology of non-pedunculated lesions identified at colonoscopy
Domain	Management of pathology
Category	Process
Rationale	The Paris classification is a helpful tool to assess the risk of invasion When polyp description is adequate, removal of polyps harboring suspicious features is likely to be avoided
Construct	Denominator: Colonoscopies with removal of non-pedunculated lesions Numerator: Procedures in the denominator where the Paris classification was used to describe lesions Exclusions: None Calculation: Proportion (%) Level of analysis: Service and endoscopist Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a 3-yearly audit of all colonoscopies performed over a 3-month period
Standards	Minimum standard: Unknown Target standard: 100 % Written colonoscopy reports should include a lesion description based on the Paris classification If the target standard is not met, feedback on adequate description of polyp morphology is warranted After evaluation and adjustment, close monitoring should be performed with a further audit within 6 months.
Consensus agreement	84.6 %
PICO	3.9 (see Supporting Information)
Evidence grading	Very low quality evidence

The acceptance of this performance measure is based on agreement with the following statement:

• The Paris classification should be routinely used to describe the morphology of non-polypoid lesions identified at colonoscopy. (N4.5) Agreement: 84.6 %

The Paris classification was developed with the aim of standardizing the terminology of superficial colorectal lesion morphology [76]. It divided lesions into two main groups: polypoid and non-polypoid, further defining four subtypes of the latter. Although its use is widely endorsed, it has never been fully validated. Recent studies have shown only moderate interobserver agreement for the Paris classification, even among experts [91] [92]. More importantly, short training sessions are not sufficient to improve the agreement, suggesting that refinement of the classification is needed [91]. Adoption of the classification in the community setting is unknown. The introduction of the Paris classification did however have two important effects: it raised awareness of subtle colorectal lesions among Western endoscopists [93] and helped to predict submucosal invasion of colorectal lesions before their removal [78] [93].

In light of the lack of better classifications, the Paris classification should be routinely used to describe the morphology of non-polypoid lesions identified at colonoscopy and its usage should be monitored. No minimum standard for this key performance measure was defined because of lack of evidence.

5 Domain: Complications

Key performance measure	Complication rate
Description	Percentage of patients in which complications (immediate, 7-day readmission rate, and 30-day mortality rate) occur after screening, diagnostic, or therapeutic colonoscopy
Domain	Complications
Category	Outcome
Rationale	Monitoring the rate of complications after screening, diagnostic, and therapeutic colonoscopy is important to assess the safety of procedures, to identify possible targets for improvement, and to allow accurate informed consent of patients
Construct	Record the following parameters: Early complications, adverse events, and harms 7-day readmission rate (30-day readmission rates, where there are reliable registries and sufficient resources) 30-day mortality rate Assessment should be done using a reliable method that allows identification of immediate and delayed complications, such as: Direct contact (e. g. telephone call) with the patient Analysis of hospital records (readmission rate) Analysis of registries (readmission rate and mortality rate) Denominator: All colonoscopies Numerator: Procedures in the denominator with a complication registered (separately for early, 7-day readmission [30], and 30-day mortality) Exclusions: None Calculation: Proportion (%) (separate for each parameter) Level of analysis: Service Frequency: Yearly for all colonoscopies performed at a service level
Standards	Minimum standard: ≤ 0.5 % for 7-day readmission rate, standards not set for 30-day mortality rate or immediate complication rate Target standard: no target standard set Endoscopic reporting systems should allow the reporting of early (in-hospital) complications, including the type of complication, description of any action relating to the complication (need for transfusion, hospitalization, or prolonged hospitalization; surgery; death; need for endoscopic re-intervention), and time from endoscopic procedure to onset of the complication Regular morbidity and mortality conferences are encouraged to assess the causes of any complications and to discuss solutions to avoid them
Consensus agreement	93.8 %
PICO	5.1 – 5.2 (see Supporting Information)
Evidence grading	Low quality evidence

The acceptance of this performance measure is based on agreement with the following statement:

• In patients undergoing colonoscopy, a 6-day readmission rate and 30-day mortality rate should be monitored using a reliable system. (N5.1) Agreement: 93.8 %

The rate of complications, adverse events, and harms are important outcome measures of colonoscopy performance. Some studies and guidelines have reported rates for specific complications such as perforation, bleeding, or sedation-related cardiopulmonary adverse events [6] [45] [94] [95] [96]. These specific outcomes are however difficult to compare across services because they are infrequent, have variable definitions, and depend on case mix. For feasibility reasons, we propose to measure adverse outcomes, as defined in previous studies [97] [98] [99] [100], to give an overall rate of complications and to drill down into specific outcomes only if the standard is not met.

The definitions of complications are of paramount importance because the differences between major and minor complications or between minor complications and routine events encountered during the course of the procedure can be vague. The all-cause 30-day mortality rate is certainly well defined and important to measure. In large clinical or administrative databases, the rate of all-cause 30-day mortality has been estimated at 0.07 % (1 in 1500) [95] [96] [97] [100] [101] [102] and the colonoscopy-specific mortality at more than 10 times lower (1 in 15 000 or lower) [95] [96] [102] [103]. Although all-cause 30-day mortality rates would be impossible to compare across services, all deaths should be discussed during morbidity and mortality conferences [104]. The LGI working group members decided that, although the accepted statement focused on the 6-day readmission rate, this should be changed to a 7-day readmission rate in order to make it more comparable with the published literature. The 7-day or 30-day hospital admission/readmission rate is a well-defined and objective way to track late complications of colonoscopy [95] [96] [97] [99] [100].

Late complications represent over half of all colonoscopy-associated complications [98]. Furthermore, the 6-day readmission rate was shown to predict 30-day all-cause mortality [99]. The reported all-cause 7-day and 30-day hospital admission/readmission rates were 0.5 % [99] and 1.1 % – 3.8 %, respectively [95] [97] [100] (0.5 % for colonoscopy-specific readmission rates) [95]. Therefore, the minimum standard of 0.5 % seems acceptable for 7-day overall or 30-day colonoscopy-specific readmission rates.

The early complication rate (diagnosed immediately during the procedure or before patient discharge) is relatively easy to measure using appropriate endoscopy reporting systems [12]. The definition of an early complication is however more challenging and, in the view of the working group, should only include complications that result in one of the following: (i) lengthening of the hospital stay; (ii) unscheduled further endoscopic procedure; or (iii) emergency intervention, including blood transfusion or surgery [6].

Reliable recording of all colonoscopy complications is a major concern [98]. A direct telephone call with a patient [101], analysis of hospital records [100], and analysis of administrative data claims [97] [100] have all been used for this purpose, but it is uncertain which method is the most feasible and reliable (see Supporting Information) [98].

6 Domain: Patient experience

Key performance measure	Patient experience
Description	Patient experience during and after colonoscopy and sigmoidoscopy should be routinely measured and self-reported by patients using validated scales
Domain	Patient experience
Category	Outcome
Rationale	Colonoscopy can be an unpleasant experience. Moreover, there are considerable differences between endoscopists and between different sedation modalities with regards to patient-reported pain and discomfort Patient experience and its improvement is crucial for the acceptance of procedures
Construct	Denominator: All colonoscopies Numerator: Procedures in the denominator in which patient experience was measured using a validated scale (the Global Rating Scale, the Gastronet, or others) Exclusions: Emergency colonoscopies Calculation: Proportion (%) Level of analysis: Individual endoscopist and service Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a yearly audit of a sample of 100 consecutive LGI endoscopies
Standards	Minimum standard: Unknown Target standard: ≥ 90 % Currently there is no standard approach to measuring patient experience: different questionnaires are available and their comparative performance is unclear. Ideally, patient experience should be self-reported using a standardized and validated reporting method Audits should be performed on both service and individual endoscopist level to assess patient-reported outcomes In case of substandard results (for example if one endoscopist performs worse than others in the same service), additional training and feedback should be considered
Consensus agreement	93.8 %
PICO	7.1 – 7.4 (see Supporting Information)
Evidence grading	Very low quality evidence

The acceptance of this performance measure is based on agreement with the following statements:

• Patient experience during and after unsedated or moderately sedated colonoscopy or sigmoidoscopy should be routinely measured. (N7.1) Agreement: 93.8 %

• Patient experience with colonoscopy or sigmoidoscopy should be self-reported by a patient using a validated scale. (N7.2) Agreement: 93.8 %

Colonoscopy may be perceived to be a painful and embarrassing procedure and this perception hampers patient participation in screening programs, adherence to surveillance recommendations, and even diagnostic work-up for large bowel symptoms [105] [106] [107]. Although sedation may decrease pain during colonoscopy, it does not eliminate it [108], has little effect on post-procedure pain [22], and increases the risk of complications [109]. Therefore, monitoring patient experience, including intra- and post-procedure pain levels, is crucial.

Monitoring patient experience is feasible, yet it is not universal and no standardized approach exists. The two most widely used and validated questionnaires for assessing patient experience are the Global Rating Scale [110] [111] and the Gastronet [22] [108] [112] [113] [114] [115]. Patient coverage and response rates varied across services from less than 80 % to over 90 % [22] [116] [117] and sustained compliance is a concern [116]. Of note, there is poor to moderate correlation between physician- or nurse-recorded and patient-reported pain levels, therefore the latter measure should be the preferred one [118]. The two main validated scales for pain assessment are a Visual Analog Scale and 4-point Verbal Rating Scale. Three studies have shown similar sensitivities for these scales (see Supporting Information) [119] [120] [121].

7 Domain: Post-procedure

Key performance measure	Appropriate post-polypectomy surveillance recommendations
Description	Adherence to post-polypectomy surveillance recommendations should be monitored and the reason for deviation from national/European guidelines should always be provided
Domain	Post-procedure
Category	Process
Rationale	Post-polypectomy surveillance recommendations reflect the best evidence-based balance between benefit and harm Too frequent surveillance wastes resources and exposes patients to complications of an invasive procedure Too infrequent surveillance may limit the effectiveness of surveillance
Construct	This performance measure takes into account not only patients’ adherence to the recommendations but also whether there were any written recommendations (letter to the patient or the patient’s general practitioner) Denominator: Patients who underwent colorectal polypectomy Numerator: Patients in the denominator who received proper (national or European) surveillance recommendations Exclusions: Reason provided for deviation from the actual surveillance recommendations Calculation: Proportion (%) Level of analysis: Service and individual endoscopist Frequency: Continuous monitoring using novel endoscopy reporting systems [12] should be the preferred approach; an alternative approach is a yearly audit of a sample of 100 consecutive LGI endoscopies
Standards	Minimum standard: no standard defined Target standard: ≥ 95 % All endoscopists should follow national or European guidelines for post-polypectomy surveillance and any deviation from these guidelines should be clearly stated When no written recommendation is given, this should be treated as a missing recommendation Endoscopic reporting systems should contain data about surveillance recommendations issued to the patient If there is suboptimal performance, an automated system that issues surveillance recommendations from the endoscopy database and reminders to the patients should be considered
Consensus agreement	93.8 %
PICO	No PICO (see Supporting Information)
Evidence grading	Low quality evidence

The acceptance of this performance measure is based on agreement with the following statement:

• Adherence to post-polypectomy surveillance recommendations should be monitored. The reason for deviation from national/European guidelines should always be provided. (N8.1) Agreement: 93.8 %

Patients who have had adenomas removed are believed to be at increased risk of developing new adenomas or cancer in the future [122] [123] [124]. In order to mitigate this risk, professional societies recommend patients undergo colonoscopy surveillance depending on age, comorbidity, and adenoma characteristics [125] [126]. Surveillance intervals recommended in the guidelines represent the best evidence-based balance between the benefits (protection against CRC) and harms (too frequent invasive examinations) of subsequent colonoscopies.

Adherence to these recommendations is key to the efficacy and efficiency of colonoscopy surveillance. Unfortunately, studies from the Netherlands and Canada have shown that less than 30 % of patients who have undergone adenoma removal receive appropriate surveillance [127] [128]. One of the key reasons for inappropriate surveillance is inappropriate recommendations given by gastroenterologists, surgeons, or primary care physicians [129] [130]. The adherence of physicians to the post-polypectomy surveillance recommendations could be relatively easily monitored using modern endoscopy reporting systems [12]. Any deviation from guideline recommendations should be clearly stated in the reporting system, with the rationale for this provided.

No minimum standard for this key performance measure was defined because of lack of evidence.

General conclusions, research priorities, and future prospects

This paper describes a short list of key performance measures for LGI endoscopy that have the best evidence-based impact on clinical outcomes, while being feasible to measure and susceptible to improvement.

The systematic process of development of these key performance measures revealed broad variation in the available evidence between the performance measures in different quality domains. Although the domains of completeness of procedure, identification of pathology, and pre-procedure have relatively robust scientific support, others, such as management of pathology and patient experience, are rather understudied. Indeed, these two quality domains were listed among the key research priorities by the ESGE research committee and are considered key research questions by the LGI working group (see [Table 1]) [131].

Table 1 Areas for further research.

Domain	Key research questions
1 Pre-procedure	What kind of intervention improves the rate of adequate bowel preparation? What is the appropriate time that should be allotted for screening and diagnostic colonoscopies?
2 Completeness of procedure	What is the diagnostic yield (and interval cancer rate) relative to increasing cecal intubation rate? What is the benefit of cecal intubation documented within a written report only or within a written and photo report?
3 Identification of pathology	What is the target standard for adenoma detection rate? What performance measure reflects the identification of pathology outside the CRC screening/surveillance setting?
4 Management of pathology	What is the most reliable and feasible method of measuring completeness of polyp removal? What is the effectiveness of add-on techniques/scales (chromoendoscopy/Paris classification/tattooing resection sites) in the management of pathology?
5 Complications	What is the most reliable and feasible method to monitor complication rates? Does monitoring help to reduce complication rates?
6 Patient experience	What is the most reliable and feasible method to monitor patient experience? How can patient experience with colonoscopy be optimized?
7 Post-procedure	What are the optimal surveillance intervals following removal of colorectal polyps? What is the effect of monitoring appropriate post-polypectomy surveillance recommendations on adherence to surveillance colonoscopy?

CRC, colorectal cancer.

The other notable feature of the identified performance measures is that the evidence behind them comes almost exclusively from the field of CRC prevention and early detection. Although performance measures from the pre-procedure and completeness of procedure domains are largely universal, performance measures within the identification of pathology, management of pathology, and post-procedure domains are not applicable outside of the CRC screening/surveillance setting. Further research on these topics is warranted (see [Table 1]).

The first step now is to implement these key performance measures in endoscopy practice throughout Europe. We encourage individual endoscopists, as well as heads of endoscopy units, to start implementation of the performance measures without delay. Implementing performance measures is important to identify services and individual endoscopists with substandard levels of performance. The aim is not to penalize these endoscopists or services but to have a tool to improve the quality of endoscopy. Feedback and benchmarking of colonoscopy performance measures are usually sufficient to positively influence the overall quality of colonoscopy [54] [132]. If the provision of such information turns out to be insufficient to promote improvement, the next step is to provide assistance and additional training [50] [52].

At a service level, the implementation of key performance measures may well require investment in hardware to accommodate a more efficient auditing process. We want to encourage hospital management to support the implementation of these performance measures in their endoscopy services. We think that, in an era where general hospital accreditation has become increasingly important, hospital administrations will be more susceptible to support such actions. Moreover, we owe it to our patients to overcome individual or financial barriers to ensure that endoscopy services are of the highest quality and to set research priorities to gather data that will inform the next generation of performance measures.

Supporting information

The detailed literature searches performed by an expert team of methodologists, as well as evolution and adaptation of the different PICOs and clinical statements during the Delphi voting process can be viewed in Supporting Information on the ESGE website.
online content viewable at: http://www.esge.com/performance-measures-for-lower-gastrointestinal-endoscopy.html

• References

• 1 Rutter MD, Senore C, Bisschops R. et al. The European Society of Gastrointestinal Endoscopy Quality Improvement Initiative: developing performance measures. Endoscopy 2016; 48: 81-89
• 2 Minoli G, Meucci G, Prada A. et al. Quality assurance and colonoscopy. Endoscopy 1999; 31: 522-527
• 3 Ball JE, Osbourne J, Jowett S. et al. Quality improvement programme to achieve acceptable colonoscopy completion rates: prospective before and after study. BMJ 2004; 329: 665-667
• 4 Rex DK, Bond JH, Winawer S. et al. Quality in the technical performance of colonoscopy and the continuous quality improvement process for colonoscopy: recommendations of the U.S. Multi-Society Task Force on Colorectal Cancer. Am J Gastroenterol 2002; 97: 1296-1308
• 5 Valori R, Rey JF, Atkin WS. et al. European guidelines for quality assurance in colorectal cancer screening and diagnosis. First Edition--Quality assurance in endoscopy in colorectal cancer screening and diagnosis. Endoscopy 2012; 44 (Suppl. 03) SE88-SE105
• 6 Rembacken B, Hassan C, Riemann JF. et al. Quality in screening colonoscopy: position statement of the European Society of Gastrointestinal Endoscopy (ESGE). Endoscopy 2012; 44: 957-968
• 7 Rex DK, Petrini JL, Baron TH. et al. Quality indicators for colonoscopy. Am J Gastroenterol 2006; 101: 873-885
• 8 Bisschops R, Areia M, Coron E. et al. Performance measures for upper gastrointestinal endoscopy: a European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Initiative. Endoscopy 2016; 48: 843-864
• 9 Bisschops R, Areia M, Coron E. et al. Performance measures for upper gastrointestinal endoscopy: A European Society of Gastrointestinal Endoscopy quality improvement initiative. United European Gastroenterol J 2016; 4: 629-656
• 10 Guyatt GH, Oxman AD, Vist GE. et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008; 336: 924-926
• 11 Do A, Weinberg J, Kakkar A. et al. Reliability of adenoma detection rate is based on procedural volume. Gastrointest Endosc 2013; 77: 376-380
• 12 Bretthauer M, Aabakken L, Dekker E. et al. Reporting systems in gastrointestinal endoscopy: Requirements and standards facilitating quality improvement: European Society of Gastrointestinal Endoscopy position statement. United European Gastroenterol J 2016; 4: 172-176
• 13 Hassan C, Bretthauer M, Kaminski MF. et al. Bowel preparation for colonoscopy: European Society of Gastrointestinal Endoscopy (ESGE) guideline. Endoscopy 2013; 45: 142-150
• 14 Froehlich F, Wietlisbach V, Gonvers JJ. et al. Impact of colonic cleansing on quality and diagnostic yield of colonoscopy: the European Panel of Appropriateness of Gastrointestinal Endoscopy European multicenter study. Gastrointest Endosc 2005; 61: 378-384
• 15 Rex DK, Imperiale TF, Latinovich DR. et al. Impact of bowel preparation on efficiency and cost of colonoscopy. Am J Gastroenterol 2002; 97: 1696-1700
• 16 Calderwood AH, Jacobson BC. Comprehensive validation of the Boston Bowel Preparation Scale. Gastrointest Endosc 2010; 72: 686-692
• 17 Rostom A, Jolicoeur E. Validation of a new scale for the assessment of bowel preparation quality. Gastrointest Endosc 2004; 59: 482-486
• 18 Aronchick CA, Lipshutz WH, Wright SH. et al. A novel tableted purgative for colonoscopic preparation: efficacy and safety comparisons with Colyte and Fleet Phospho-Soda. Gastrointest Endosc 2000; 52: 346-352
• 19 Parmar R, Martel M, Rostom A. et al. Validated scales for colon cleansing: a systematic review. Am J Gastroenterol 2016; 111: 197-204 (quiz 205)
• 20 Clark BT, Protiva P, Nagar A. et al. Quantification of adequate bowel preparation for screening or surveillance colonoscopy in men. Gastroenterology 2016; 150: 396-405 (quiz e14–e15)
• 21 Calderwood AH, Logan JR, Zurfluh M. et al. Validity of a web-based educational program to disseminate a standardized bowel preparation rating scale. J Clin Gastroenterol 2014; 48: 856-861
• 22 Bretthauer M, Kaminski MF, Loberg M. et al. Population-based colonoscopy screening for colorectal cancer: a randomized clinical trial. JAMA Intern Med 2016; 176: 894-902
• 23 Calderwood AH, Schroy 3rd PC, Lieberman DA. et al. Boston Bowel Preparation Scale scores provide a standardized definition of adequate for describing bowel cleanliness. Gastrointest Endosc 2014; 80: 269-276
• 24 Shaukat A, Rector TS, Church TR. et al. Longer withdrawal time is associated with a reduced incidence of interval cancer after screening colonoscopy. Gastroenterology 2015; 149: 952-957
• 25 Zorzi M, Valiante F, Germana B. et al. Comparison between different colon cleansing products for screening colonoscopy. A noninferiority trial in population-based screening programs in Italy. Endoscopy 2016; 48: 223-231
• 26 Radaelli F, Paggi S, Hassan C. et al. Split-dose preparation for colonoscopy increases adenoma detection rate: a randomised controlled trial in an organised screening programme. Gut 2017; 66: 270-277
• 27 Whitson MJ, Bodian CA, Aisenberg J. et al. Is production pressure jeopardizing the quality of colonoscopy? A survey of U.S. endoscopists' practices and perceptions. Gastrointest Endosc 2012; 75: 641-648
• 28 Condiotte AM, Robertson DJ, Blodgett C. et al. “Running late” and adenoma detection – is there an association?. Endoscopy 2015; 47: 232-237
• 29 Jain D, Goyal A, Zavala S. Predicting colonoscopy time: a quality improvement initiative. Clin Endosc 2016; 49: 555-559
• 30 Moritz V, Holme O, Leblanc M. et al. An explorative study from the Norwegian Quality Register Gastronet comparing self-estimated versus registered quality in colonoscopy performance. Endosc Int Open 2016; 4: E326-E332
• 31 Hassan C, Di Giulio E, Marmo R. et al. Appropriateness of the indication for colonoscopy: systematic review and meta-analysis. J Gastrointestin Liver Dis 2011; 20: 279-286
• 32 Gimeno Garcia AZ, Gonzalez Y, Quintero E. et al. Clinical validation of the European Panel on the Appropriateness of Gastrointestinal Endoscopy (EPAGE) II criteria in an open-access unit: a prospective study. Endoscopy 2012; 44: 32-37
• 33 Mangualde J, Cremers MI, Vieira AM. et al. Appropriateness of outpatient gastrointestinal endoscopy in a non-academic hospital. World J Gastrointest Endosc 2011; 3: 195-200
• 34 Carrion S, Marin I, Lorenzo-Zuniga V. et al. [Appropriateness of colonoscopy indications according to the new EPAGE II criteria]. Gastroenterol Hepatol 2010; 33: 484-489
• 35 Appropriate use of gastrointestinal endoscopy. American Society for Gastrointestinal Endoscopy. Gastrointest Endosc 2000; 52: 831-837
• 36 Juillerat P, Peytremann-Bridevaux I, Vader JP. et al. Appropriateness of colonoscopy in Europe (EPAGE II). Presentation of methodology, general results, and analysis of complications. Endoscopy 2009; 41: 240-246
• 37 Eskeland SL, Dalen E, Sponheim J. et al. European Panel on the Appropriateness of Gastrointestinal Endoscopy II guidelines help in selecting and prioritizing patients referred to colonoscopy - -a quality control study. Scand J Gastroenterol 2014; 49: 492-500
• 38 Thoufeeq MH, Rembacken BJ. Meticulous cecal image documentation at colonoscopy is associated with improved polyp detection. Endosc Int Open 2015; 3: E629-E633
• 39 Neilson LJ, Bevan R, Panter S. et al. Terminal ileal intubation and biopsy in routine colonoscopy practice. Expert Rev Gastroenterol Hepatol 2015; 9: 567-574
• 40 Baxter NN, Sutradhar R, Forbes SS. et al. Analysis of administrative data finds endoscopist quality measures associated with postcolonoscopy colorectal cancer. Gastroenterology 2011; 140: 65-72
• 41 Lee TJ, Rutter MD, Blanks RG. et al. Colonoscopy quality measures: experience from the NHS Bowel Cancer Screening Programme. Gut 2012; 61: 1050-1057
• 42 Belderbos TD, Grobbee EJ, van Oijen MG. et al. Comparison of cecal intubation and adenoma detection between hospitals can provide incentives to improve quality of colonoscopy. Endoscopy 2015; 47: 703-709
• 43 Zorzi M, Senore C, Da Re F. et al. Detection rate and predictive factors of sessile serrated polyps in an organised colorectal cancer screening programme with immunochemical faecal occult blood test: the EQuIPE study (Evaluating Quality Indicators of the Performance of Endoscopy). Gut 2000;
• 44 Jover R, Zapater P, Polania E. et al. Modifiable endoscopic factors that influence the adenoma detection rate in colorectal cancer screening colonoscopies. Gastrointest Endosc 2013; 77: 381-389 e1
• 45 Gavin DR, Valori RM, Anderson JT. et al. The national colonoscopy audit: a nationwide assessment of the quality and safety of colonoscopy in the UK. Gut 2013; 62: 242-249
• 46 Kaminski MF, Regula J, Kraszewska E. et al. Quality indicators for colonoscopy and the risk of interval cancer. NEJM 2010; 362: 1795-1803
• 47 Corley DA, Jensen CD, Marks AR. et al. Adenoma detection rate and risk of colorectal cancer and death. NEJM 2014; 370: 1298-1306
• 48 van Rijn JC, Reitsma JB, Stoker J. et al. Polyp miss rate determined by tandem colonoscopy: a systematic review. Am J Gastroenterol 2006; 101: 343-350
• 49 Rogal SS, Pinsky PF, Schoen RE. Relationship between detection of adenomas by flexible sigmoidoscopy and interval distal colorectal cancer. Clin Gastroenterol Hepatol 2013; 11: 73-78
• 50 Kaminski MF, Anderson J, Valori R. et al. Leadership training to improve adenoma detection rate in screening colonoscopy: a randomised trial. Gut 2016; 65: 616-624
• 51 Brenner H, Altenhofen L, Kretschmann J. et al. Trends in adenoma detection rates during the first 10 years of the German screening colonoscopy program. Gastroenterology 2015; 149: 356-366 e1
• 52 Coe SG, Crook JE, Diehl NN. et al. An endoscopic quality improvement program improves detection of colorectal adenomas. Am J Gastroenterol 2013; 108: 219-226 (quiz 227)
• 53 Corley DA, Jensen CD, Marks AR. Can we improve adenoma detection rates? A systematic review of intervention studies. Gastrointest Endosc 2011; 74: 656-665
• 54 Kaminski MF, Rupinski M, Wieszczy P. et al. Effect of adenoma detection rate improvement on the risk of colorectal cancer and death. Gastroenterology 2015; 148: S189
• 55 Kaminski MF, Wieszczy P, Kolacz A. et al. Comparison of quality measures for detection of neoplasia at screening colonoscopy. Gastrointestinal Endoscopy 2016; 83: AB527
• 56 Barclay RL, Vicari JJ, Doughty AS. et al. Colonoscopic withdrawal times and adenoma detection during screening colonoscopy. NEJM 2006; 355: 2533-2541
• 57 Moritz V, Bretthauer M, Ruud HK. et al. Withdrawal time as a quality indicator for colonoscopy - a nationwide analysis. Endoscopy 2012; 44: 476-481
• 58 Lee TJ, Blanks RG, Rees CJ. et al. Longer mean colonoscopy withdrawal time is associated with increased adenoma detection: evidence from the Bowel Cancer Screening Programme in England. Endoscopy 2013; 45: 20-26
• 59 Sawhney MS, Cury MS, Neeman N. et al. Effect of institution-wide policy of colonoscopy withdrawal time > or = 7 minutes on polyp detection. Gastroenterology 2008; 135: 1892-1898
• 60 Barclay RL, Vicari JJ, Greenlaw RL. Effect of a time-dependent colonoscopic withdrawal protocol on adenoma detection during screening colonoscopy. Clin Gastroenterol Hepatol 2008; 6: 1091-1098
• 61 Vavricka SR, Sulz MC, Degen L. et al. Monitoring colonoscopy withdrawal time significantly improves the adenoma detection rate and the performance of endoscopists. Endoscopy 2016; 48: 256-262
• 62 Lee RH, Tang RS, Muthusamy VR. et al. Quality of colonoscopy withdrawal technique and variability in adenoma detection rates (with videos). Gastrointest Endosc 2011; 74: 128-134
• 63 Rutter MD, Chilton A, Patnick J. Monitoring colonoscopy withdrawal times remains important. Endoscopy 2013; 45: 73
• 64 Patel NC, Islam RS, Wu Q. et al. Measurement of polypectomy rate by using administrative claims data with validation against the adenoma detection rate. Gastrointest Endosc 2013; 77: 390-394
• 65 Williams JE, Holub JL, Faigel DO. Polypectomy rate is a valid quality measure for colonoscopy: results from a national endoscopy database. Gastrointest Endosc 2012; 75: 576-582
• 66 Francis DL, Rodriguez-Correa DT, Buchner A. et al. Application of a conversion factor to estimate the adenoma detection rate from the polyp detection rate. Gastrointest Endosc 2011; 73: 493-497
• 67 Atia MA, Patel NC, Ratuapli SK. et al. Nonneoplastic polypectomy during screening colonoscopy: the impact on polyp detection rate, adenoma detection rate, and overall cost. Gastrointest Endosc 2015; 82: 370-375.e1
• 68 Boroff ES, Gurudu SR, Hentz JG. et al. Polyp and adenoma detection rates in the proximal and distal colon. Am J Gastroenterol 2013; 108: 993-999
• 69 Robertson DJ, Lieberman DA, Winawer SJ. et al. Colorectal cancers soon after colonoscopy: a pooled multicohort analysis. Gut 2014; 63: 949-956
• 70 le Clercq CM, Bouwens MW, Rondagh EJ. et al. Postcolonoscopy colorectal cancers are preventable: a population-based study. Gut 2014; 63: 957-963
• 71 Pohl H, Srivastava A, Bensen SP. et al. Incomplete polyp resection during colonoscopy-results of the complete adenoma resection (CARE) study. Gastroenterology 2013; 144: 74-80 e1
• 72 Kim JS, Lee BI, Choi H. et al. Cold snare polypectomy versus cold forceps polypectomy for diminutive and small colorectal polyps: a randomized controlled trial. Gastrointest Endosc 2015; 81: 741-747
• 73 Lee CK, Shim JJ, Jang JY. Cold snare polypectomy vs. cold forceps polypectomy using double-biopsy technique for removal of diminutive colorectal polyps: a prospective randomized study. Am J Gastroenterol 2013; 108: 1593-1600
• 74 Britto-Arias M, Waldmann E, Jeschek P. et al. Forceps versus snare polypectomies in colorectal cancer screening: are we adhering to the guidelines?. Endoscopy 2015; 47: 898-902
• 75 Din S, Ball AJ, Taylor E. et al. Polypectomy practices of sub-centimeter polyps in the English Bowel Cancer Screening Programme. Surg Endosc 2015; 29: 3224-3230
• 76 Kudo S, Lambert R, Allen JI. et al. Nonpolypoid neoplastic lesions of the colorectal mucosa. Gastrointest Endosc 2008; 68: S3-S47
• 77 Lambert R, Kudo SE, Vieth M. et al. Pragmatic classification of superficial neoplastic colorectal lesions. Gastrointest Endosc 2009; 70: 1182-1199
• 78 Moss A, Bourke MJ, Williams SJ. et al. Endoscopic mucosal resection outcomes and prediction of submucosal cancer from advanced colonic mucosal neoplasia. Gastroenterology 2011; 140: 1909-1918
• 79 Wang R, Wang Y, Li D. et al. Application of carbon nanoparticles to mark locations for re-inspection after colonic polypectomy. Surg Endosc 2016; 30: 1530-1533
• 80 Park JW, Sohn DK, Hong CW. et al. The usefulness of preoperative colonoscopic tattooing using a saline test injection method with prepackaged sterile India ink for localization in laparoscopic colorectal surgery. Surg Endosc 2008; 22: 501-505
• 81 Bartels SA, van der Zaag ES, Dekker E. et al. The effect of colonoscopic tattooing on lymph node retrieval and sentinel lymph node mapping. Gastrointest Endosc 2012; 76: 793-800
• 82 Kang J, Park HS, Kim IK. et al. Effect of preoperative colonoscopic tattooing on lymph node harvest in T1 colorectal cancer. Int J Colorectal Dis 2015; 30: 1349-1355
• 83 Belderbos TD, Leenders M, Moons LM. et al. Local recurrence after endoscopic mucosal resection of nonpedunculated colorectal lesions: systematic review and meta-analysis. Endoscopy 2014; 46: 388-402
• 84 Zafar A, Mustafa M, Chapman M. Colorectal polyps: when should we tattoo?. Surg Endosc 2012; 26: 3264-3266
• 85 Kaminski MF, Hassan C, Bisschops R. et al. Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) Guideline. Endoscopy 2014; 46: 435-457
• 86 Deenadayalu VP, Rex DK. Colon polyp retrieval after cold snaring. Gastrointest Endosc 2005; 62: 253-256
• 87 Ye F, Feng Y, Lin J. Retrieval of colorectal polyps following snare polypectomy: Experience of the multiple-suction technique in 602 cases. Int J Colorectal Dis 2008; 23: 431-436
• 88 Fernandes C, Pinho R, Ribeiro I. et al. Risk factors for polyp retrieval failure in colonoscopy. United European Gastroenterol J 2015; 3: 387-392
• 89 Komeda Y, Suzuki N, Sarah M. et al. Factors associated with failed polyp retrieval at screening colonoscopy. Gastrointest Endosc 2013; 77: 395-400
• 90 Belderbos TD, van Oijen MG, Moons LM. et al. The “golden retriever” study: improving polyp retrieval rates by providing education and competitive feedback. Gastrointest Endosc 2016; 83: 596-601
• 91 van Doorn SC, Hazewinkel Y, East JE. et al. Polyp morphology: an interobserver evaluation for the Paris classification among international experts. Am J Gastroenterol 2015; 110: 180-187
• 92 Aziz Aadam A, Wani S, Kahi C. et al. Physician assessment and management of complex colon polyps: a multicenter video-based survey study. Am J Gastroenterol 2014; 109: 1312-1324
• 93 Soetikno R, Friedland S, Kaltenbach T. et al. Nonpolypoid (flat and depressed) colorectal neoplasms. Gastroenterology 2006; 130: 566-576 (quiz 588–589)
• 94 Fisher DA, Maple JT, Ben-Menachem T. et al. Complications of colonoscopy. Gastrointest Endosc 2011; 74: 745-752
• 95 Levin TR, Zhao W, Conell C. et al. Complications of colonoscopy in an integrated health care delivery system. Ann Intern Med 2006; 145: 880-886
• 96 Rabeneck L, Paszat LF, Hilsden RJ. et al. Bleeding and perforation after outpatient colonoscopy and their risk factors in usual clinical practice. Gastroenterology 2008; 135: 1899-1906 1906.e1
• 97 Warren JL, Klabunde CN, Mariotto AB. et al. Adverse events after outpatient colonoscopy in the Medicare population. Ann Intern Med 2009; 150: 849-857 w152
• 98 Adler A, Lieberman D, Aminalai A. et al. Data quality of the German screening colonoscopy registry. Endoscopy 2013; 45: 813-818
• 99 Sarkar S, Geraghty J, Moore AR. et al. A multicentre study to determine the incidence, demographics, aetiology and outcomes of 6-day emergency readmission following day-case endoscopy. Eur J Gastroenterol Hepatol 2012; 24: 1438-1446
• 100 Saraste D, Martling A, Nilsson PJ. et al. Complications after colonoscopy and surgery in a population-based colorectal cancer screening programme. J Med Screen 2016; 23: 135-140
• 101 Ko CW, Riffle S, Michaels L. et al. Serious complications within 30 days of screening and surveillance colonoscopy are uncommon. Clin Gastroenterol Hepatol 2010; 8: 166-173
• 102 Ko CW, Dominitz JA. Complications of colonoscopy: magnitude and management. Gastrointest Endosc Clin N Am 2010; 20: 659-671
• 103 Reumkens A, Rondagh EJ, Bakker CM. et al. Post-colonoscopy complications: a systematic review, time trends, and meta-analysis of population-based studies. Am J Gastroenterol 2016; 111: 1092-1101
• 104 Tapper EB, Leffler DA. The Morbidity and Mortality Conference in Gastroenterology and Hepatology: an important cornerstone of patient safety and optimal care. Gastroenterology 2016; 150: 19-23
• 105 McLachlan SA, Clements A, Austoker J. Patients' experiences and reported barriers to colonoscopy in the screening context--a systematic review of the literature. Patient Educ Couns 2012; 86: 137-146
• 106 Denters MJ, Schreuder M, Depla AC. et al. Patients' perception of colonoscopy: patients with inflammatory bowel disease and irritable bowel syndrome experience the largest burden. Eur J Gastroenterol Hepatol 2013; 25: 964-972
• 107 Pylvanainen K, Kairaluoma M, Mecklin JP. Compliance and satisfaction with long-term surveillance in Finnish HNPCC families. Fam Cancer 2006; 5: 175-178
• 108 Seip B, Bretthauer M, Dahler S. et al. Patient satisfaction with on-demand sedation for outpatient colonoscopy. Endoscopy 2010; 42: 639-646
• 109 Wernli KJ, Brenner AT, Rutter CM. et al. Risks associated with anesthesia services during colonoscopy. Gastroenterology 2016; 150: 888-894 (quiz e18)
• 110 Sint Nicolaas J, de Jonge V, Korfage IJ. et al. Benchmarking patient experiences in colonoscopy using the Global Rating Scale. Endoscopy 2012; 44: 462-472
• 111 de Jonge V, Sint Nicolaas J, Lalor EA. et al. A prospective audit of patient experiences in colonoscopy using the Global Rating Scale: a cohort of 1,187 patients. Can J Gastroenterol 2010; 24: 607-613
• 112 Hoff G, Bretthauer M, Huppertz-Hauss G. et al. The Norwegian Gastronet project: Continuous quality improvement of colonoscopy in 14 Norwegian centres. Scand J Gastroenterol 2006; 41: 481-487
• 113 Holme O, de Lange T, Stallemo A. et al. Routine vs. on-demand analgesia in colonoscopy: a randomized clinical trial. Endoscopy 2016; 48: 823-828
• 114 Moritz V, Bretthauer M, Holme O. et al. Time trends in quality indicators of colonoscopy. United European Gastroenterol J 2016; 4: 110-120
• 115 Kaminski MF, Kraszewska E, Rupinski M. et al. Design of the Polish Colonoscopy Screening Program: a randomized health services study. Endoscopy 2015; 47: 1144-1150
• 116 Seip B, Bretthauer M, Dahler S. et al. Sustaining the vitality of colonoscopy quality improvement programmes over time. Experience from the Norwegian Gastronet programme. Scand J Gastroenterol 2010; 45: 362-369
• 117 Ghanouni A, Plumb A, Hewitson P. et al. Patients' experience of colonoscopy in the English Bowel Cancer Screening Programme. Endoscopy 2016; 48: 232-240
• 118 Rostom A, Ross ED, Dube C. et al. Development and validation of a nurse-assessed patient comfort score for colonoscopy. Gastrointest Endosc 2013; 77: 255-261
• 119 Skovlund E, Flaten O. Response measures in the acute treatment of migraine. Cephalalgia 1995; 15: 519-522 ; discussion 450–451
• 120 Skovlund E, Bretthauer M, Grotmol T. et al. Sensitivity of pain rating scales in an endoscopy trial. Clin J Pain 2005; 21: 292-296
• 121 Breivik EK, Bjornsson GA, Skovlund E. A comparison of pain rating scales by sampling from clinical trial data. Clin J Pain 2000; 16: 22-28
• 122 Cottet V, Jooste V, Fournel I. et al. Long-term risk of colorectal cancer after adenoma removal: a population-based cohort study. Gut 2012; 61: 1180-1186
• 123 Martinez ME, Baron JA, Lieberman DA. et al. A pooled analysis of advanced colorectal neoplasia diagnoses after colonoscopic polypectomy. Gastroenterology 2009; 136: 832-841
• 124 Loberg M, Kalager M, Holme O. et al. Long-term colorectal-cancer mortality after adenoma removal. NEJM 2014; 371: 799-807
• 125 Atkin WS, Valori R, Kuipers EJ. et al. European guidelines for quality assurance in colorectal cancer screening and diagnosis. First Edition--Colonoscopic surveillance following adenoma removal. Endoscopy 2012; 44 (Suppl. 03) SE151-SE163
• 126 Hassan C, Quintero E, Dumonceau JM. et al. Post-polypectomy colonoscopy surveillance: European Society of Gastrointestinal Endoscopy (ESGE) Guideline. Endoscopy 2013; 45: 842-851
• 127 van Heijningen EM, Lansdorp-Vogelaar I, Steyerberg EW. et al. Adherence to surveillance guidelines after removal of colorectal adenomas: a large, community-based study. Gut 2015; 64: 1584-1592
• 128 Schreuders E, Sint Nicolaas J, de Jonge V. et al. The appropriateness of surveillance colonoscopy intervals after polypectomy. Can J Gastroenterol 2013; 27: 33-38
• 129 Mysliwiec PA, Brown ML, Klabunde CN. et al. Are physicians doing too much colonoscopy? A national survey of colorectal surveillance after polypectomy. Ann Intern Med 2004; 141: 264-271
• 130 Boolchand V, Olds G, Singh J. et al. Colorectal screening after polypectomy: a national survey study of primary care physicians. Ann Intern Med 2006; 145: 654-659
• 131 Rees C, Ngu WS, Regula J. et al. European Society of Gastrointestinal Endoscopy – Establishing the key unanswered research questions within gastrointestinal endoscopy. Endoscopy 2016; 48: 1-11
• 132 Kiefe CI, Allison JJ, Williams OD. et al. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA 2001; 285: 2871-2879