European guidelines for quality assurance in colorectal cancer screening and diagnosis. First Edition – Evaluation and interpretation of screening outcomes

Multidisciplinary, evidence-based guidelines for quality assurance in colorectal cancer screening and diagnosis have been developed by experts in a project coordinated by the International Agency for Research on Cancer. The full guideline document covers the entire process of population-based screening. It consists of 10 chapters and over 250 recommendations, graded according to the strength of the recommendation and the supporting evidence. The 450-page guidelines and the extensive evidence base have been published by the European Commission. The chapter on evaluation and interpretation of screening outcomes includes 20 graded recommendations. The content of the chapter is presented here to promote international discussion and collaboration by making the principles and standards recommended in the new EU Guidelines known to a wider professional and scientific community. Following these recommendations has the potential to enhance the control of colorectal cancer through improvement in the quality and effectiveness of the screening process, including multi-disciplinary diagnosis and management of the disease.

Keywords

mass screening - colorectal neoplasms - evaluation - monitoring - screening outcomes - evidence-based guidelines - population-based programmes

Background

According to the most recent estimates by the International Agency for Research on Cancer [12] colorectal cancer (CRC) is the most common cancer in Europe with 432 000 new cases in men and women reported annually. It is the second most common cause of cancer deaths in Europe with 212 000 deaths reported in 2008. Worldwide CRC ranks third in incidence and fourth in mortality with an estimated 1.2 million cases and 0.6 million deaths annually. The European Union (EU) recommends population-based screening for breast, cervical and colorectal cancer using evidence-based tests with quality assurance of the entire screening process including diagnosis and management of patients with screen-detected lesions [5]. The EU policy takes into account the principles of cancer screening developed by the World Health Organization [50] and the extensive experience in the EU in piloting and implementing population-based cancer screening programmes [46]. Screening is an important tool in cancer control in countries with a significant burden of CRC, provided the screening services are high quality [47]. The presently reported multidisciplinary, evidence-based guidelines for quality assurance in colorectal cancer screening and diagnosis have been developed by experts and published by the EU [37].

Methods

The methods used are described in detail elsewhere in this supplement [29]. Briefly, a multidisciplinary group of authors and editors experienced in programme implementation and quality assurance in colorectal cancer screening and in guideline development collaborated with a literature group consisting of epidemiologists with special expertise in the field of CRC and in performing systematic literature reviews. The literature group systematically retrieved, evaluated and synthesized relevant publications according to defined clinical questions (modified Patient-Intervention-Comparison-Outcome-Study method). Bibliographic searches for most clinical questions were limited to the years 2000 to 2008 and were performed on Medline, and in many cases also on Embase and The Cochrane Library. Additional searches were conducted without date restrictions or starting before 2000 if the authors or editors who were experts in the field knew that there were relevant articles published before 2000. Articles of adequate quality recommended by authors because of their clinical relevance were also included.

Only scientific publications in English, Italian, French and Spanish were included. Priority was given to recently published, systematic reviews or clinical guidelines. If systematic reviews of high methodological quality were retrieved, the search for primary studies was limited to those published after the last search date of the most recently published systematic review, i. e. if the systematic review had searched primary studies until February 2006, primary studies published after February 2006 were sought. If no systematic reviews were found, a search for primary studies published since 2000 was performed.

In selected cases references not identified by the above process were included in the evidence base, i. e. when authors of the chapters found relevant articles published after 2008 during the period when chapter manuscripts were drafted and revised prior to publication. The criteria for relevance were: articles concerning new and emerging technologies where the research grows rapidly, high-quality and updated systematic reviews, and large trials giving high contribution to the robustness of the results or allowing upgrading of the level of evidence.

The methodological quality of the retrieved publications was assessed using the criteria obtained from published and validated check lists. Evidence tables were prepared for the selected studies. The evidence tables, clinical questions and bibliographic literature searches are documented elsewhere [28].

In the full guidelines document prepared by the authors and editors [37] over 250 recommendations were formulated according to the level of the evidence and the strength of the recommendation using the following grading scales.

Level of evidence

I multiple randomised controlled trials (RCTs) of reasonable sample size, or systematic reviews (SRs) of RCTs

II one RCT of reasonable sample size, or 3 or less RCTs with small sample size

III prospective or retrospective cohort studies or SRs of cohort studies; diagnostic cross-sectional accuracy studies

IV retrospective case-control studies or SRs of case-control studies, time-series analyses

V case series; before/after studies without control group, cross-sectional surveys

VI expert opinion

Strength of recommendation:

A intervention strongly recommended for all patients or targeted individuals

B intervention recommended

C intervention to be considered but with uncertainty about its impact

D intervention not recommended

E intervention strongly not recommended

Some statements of advisory character considered to be good practice but not sufficiently important to warrant formal grading were included in the text.

Results

Twenty graded recommendations are provided in Chapter 3.

Recommendations[1]

3.1  The development of comprehensive systems for documentation of the screening processes, monitoring of data acquisition and quality, and accurate compilation and reporting of results are essential to the evaluation of population screening programmes (VI – A).^{Sect 3.1}

3.2  Detailed eligibility criteria should be predefined, based on a pre-specified protocol (see also Ch. 2 [26], Rec. 2.4, Sect. 2.3.1.1) (VI – B).^{Sect 3.2.1}

3.3  A database consisting of individual records (one record per person for each screening episode) is essential in order to produce results on screening performance (VI – A).^{Sect 3.2.1}

3.4  Quality control procedures for the database should be available and run regularly to check the quality of the data and to correct data entry errors (VI – A).^{Sect 3.2.1}

3.5  For monitoring the programme, tables presenting performance indicators should be produced at regular intervals (at least annually) by age and gender and by type of screening test using the collected data (VI – A).^{Sect 3.2.5}

3.6  All indicators should be calculated and reported for age-gender subgroups (VI – A).^{Sect 3.3}

3.7  Invitation coverage should be high (95 %) in order to maximise screening impact (VI – A).^{Sect 3.3.1}

3.8  A minimum uptake of 45 % is acceptable (III – A), but it is recommended to aim for a rate of at least 65 % (III – A).^{Sect 3.3.1}

3.9  Rates of inadequate FOBTs should remain low. These reflect the understanding of the people who are using the test and therefore the quality of the information given to the population. Less than 3 % is acceptable, less than 1 % is desirable (See Ch. 4 [15], Rec. 4.21) (III – A).^{Sect 3.3.2; 4.3.4}

3.10 High rates of referral to follow-up colonoscopy should be achieved for people with a positive screening test or examination requiring follow-up (90 % is acceptable, > 95 % is desirable) (VI – A).^{Sect 3.3.2; 3.3.3}

3.11 The proportion of screening and follow-up colonoscopies that are incomplete should be recorded separately. A completeness rate of > 90 % is acceptable, > 95 % is desirable (see also Ch. 5 [44], Rec. 5.41) (III – A).^{Sect 3.3.2; 3.3.3; 5.4.5.1}

3.12 A favourable stage distribution in screen-detected cancers compared to clinically diagnosed cancers should be observed. In absence of this condition a screening programme could not be effective (I – A).^{Sect 3.3.2}

3.13 The rate of serious adverse effects should be monitored carefully (III – A).^{Sect 3.3.2; 3.3.3}

3.14 High rates of compliance with follow-up colonoscopy should be achieved (85 % is acceptable, > 90 % is desirable) (III – A).^{Sect 3.3.2; 3.3.3}

3.15 The time in days, between completion of a screening test and receipt of results by the participant should be as short as possible: acceptable standard > 90 % within 15 days (VI – A).^{Sect 3.3.4}

3.16 Follow-up colonoscopy after positive screening (any modality) should be scheduled within 31 days of referral (acceptable standard is > 90 %, desirable > 95 %). (See Ch. 5 [44], Rec. 5.19) (VI – B).^{Sect 3.3.4; 5.3.5}

3.17 The time interval between positive FS or colonoscopy and definitive management should be minimised and in 95 % of cases should be no more than 31 days (acceptable standard) (see Ch. 8 [41], Rec. 8.2) (VI – B).^{Sect 3.3.4; 8.2}

3.18 The evaluation of surrogate outcome measures requires rigorous data collection of colorectal cancer registrations and stage of disease in the target population. Such data should also be collected for the time period leading directly up to the introduction of a screening programme to allow trends to be analysed (VI – A).^{Sect 3.4}

3.19 Data on interval cancers should be collected and reported (VI – A).^{Sect 3.4.1}

3.20 Evaluation of interval cancer rates requires careful linkage of cancer registrations with screening history to allow cancers to be classified (i. e. as screen detected, interval, non-responders, other). A link with the cancer registry should be established (VI – A).^{Sect 3.4.1}

3.1 Introduction

Evaluation and interpretation of screening outcomes are essential to recognising whether a colorectal cancer screening programme is achieving the goals for which it has been established. It is recognised that the context and logistics of screening programmes will differ by country and even by region. For example, the prior existence of a population register facilitates issuing personalised invitations, whereas the absence of such a register may lead to recruitment by open invitation. Many of these contextual differences will affect the measured outcomes.

The effectiveness of a programme is a function of the quality of its individual components. Success of the programme is measured not only by its impact on public health, but also by its organisation, implementation, and acceptability.

The organisational aspects of a screening programme, described in Chapter 2 [26] of these Guidelines influence the evaluation and interpretation of screening outcomes. Therefore all aspects of the programme should be monitored and evaluated.

To determine whether a programme has been effective with regard to its impact on morbidity and mortality requires continuous follow-up of the target population over an extended time-frame. Therefore early-performance indicators using standard definitions, available early in the lifetime of a screening programme are essential to measure the quality of the programme and its potential longer-term impact.

A key component in the evaluation of population screening programmes is data collection. Colorectal cancer screening can be performed using various tests or techniques. Data collection necessary for evaluation can be common to all tests or specific to particular tests. The examples given in these Guidelines refer to in vitro stool tests based on detection of faecal occult blood (FOBT) that are currently the most widely used, and to endoscopic tests i. e. flexible sigmoidoscopy (FS) or colonoscopy (CS). In the text, gFOBT refers to guaiac-based FOBTs, and iFOBT to immunological FOBTs.

This chapter includes only the minimum data variables and indicators that should be collected and measured for the purposes of programme evaluation. It does not discuss quality indicators such as those used to measure endoscopist performance or patient satisfaction; a number of such indicators are described elsewhere in the Guidelines.

It should be noted that in a setting where opportunistic screening (for example by colonoscopy) has been taking place for some time, the uptake and performance of an organised programme may differ markedly from those in a setting where no such screening has been taking place. The majority of the values of the indicators described below will relate to the latter setting.

Recommendation

• The development of comprehensive systems for documentation of the screening processes, monitoring of data acquisition and quality, and accurate compilation and reporting of results are essential to the evaluation of a population screening programme [7] (VI – A).^{Rec 3.1}

3.2 Data items necessary for evaluation

This section describes the data items and information that must be collected, recorded and stored in order to generate the indicators, analyses and reports required for evaluation.

3.2.1 Programme conditions

Programme type

As mentioned above, the organisational aspects of a screening programme influence the evaluation and interpretation of screening outcomes. Population-based programmes are recommended because they require an infrastructure that is conducive to implementation of quality assurance and evaluation, such as through linkage of screening data and cancer registry data [47]. It is therefore important to document the type of programme (population-based or non-population-based) and to describe the sources of population data used for identification and invitation of the eligible target population (e. g. population registry). Data on screening outcomes should be linked with data from other registries in order to monitor and evaluate the programme.

Primary screening test

Currently only the faecal occult blood test (FOBT) is recommended by the EU for CRC screening. However endoscopic screening programmes with flexible sigmoidoscopy (FS) or colonoscopy (CS) as primary screening tests are currently running in a number of Member States. Given the potential impact of the type of primary screening test or tests used in a programme on the respective results and performance, the type of primary screening test should always be indicated when documenting results and reporting.

Population base

A screening programme is population based when every member of the target population in the area designated to be served by the programme is known to the programme, and when the eligible members of the target population are individually invited to participate.

The availability and reliability of target population data will depend on the existence, quality and accessibility of population registers in the region where the programme is being set up. Population registers are not always available and demographic data for identifying the target population might be obtained from various sources, e. g. census data, electoral registers, private or statutory health care registers or health insurance funds registers. The choice of the target population database for issuing invitations will depend on the completeness of the database and on the individuals or variables included, e. g. electoral registers might not include eligible foreigners or dates of birth.

A database consisting of individual records (one record per person for each screening episode) is essential in order to produce results on organisational aspects of the programme (coverage, participation) and screening performance. The data collected should respect a logical order and follow the development of the screening process (identification of person [date of birth, gender], date of invitation, date of reminder, date of test, test results, date of the examination performed during assessment, results, colonoscopy date, results, adverse effects, treatment). The location in the bowel of any detected lesions or cancers (Tumour site) should also be recorded [Rectum, sigmoid, descending colon (distal colon) transverse colon, splenic flexure, ascending colon].

Each variable should be precisely defined. All data collected for each round should be kept and updated information should not overwrite data provided during preceding rounds. All information on the timing of events during each screening episode, including invitation history, should be recorded as calendar dates. This ensures maximal flexibility of the database for future evaluation efforts and participation in multi-centre studies. It also permits distinguishing between the first and subsequent screening episodes and between participants with different patterns of attendance (see Section 3.3).

• Self registrations
  Self registrations are defined as eligible residents of the designated area served by the programme, who request screening but who are not identified by the target population register used to generate invitations. Their number should be reported separately.

• Self referrals
  Self referrals are defined as people requesting screening before receipt of an invitation or outside the invited age-range. They should not be included in coverage by invitation, or in participation rate if in the relevant age range, but their number should be reported separately.

Recommendations

• Detailed eligibility criteria should be pre-defined based on a pre-specified protocol (see also Ch. 2 [26], Rec. 2.4, Sect. 2.3.1.1) (VI – B).^{Rec 3.2}

• A database consisting of individual records (one record per person for each screening episode) is essential in order to produce results on screening performance (VI – A).^{Rec 3.3}

• Quality control procedures for the database should be available and run regularly to check the quality of the data and to correct any data entry errors. (VI – A).^{Rec 3.4}

3.2.2 Invitation variables

Target population

The target population are those people of eligible age according to the programme policy residing in the area designated to be served by the screening programme.

Eligible population

The eligible population are those people in the target population who fulfil the eligibility criteria specified in the programme policy.

Invited

The invited are those members of the eligible population who have received an invitation for screening according to the programme policy/process; e. g. invited by mail, by primary care practitioner. N.B. Not all invitations sent may be received.

3.2.3 Process variables of primary screening and follow up

3.2.3.1 Process variables in screening with the faecal occult blood test (FOBT) and other in vitro tests

The following process variables are described in the context of screening with faecal occult blood testing because FOBT is the only screening test currently recommended by the EU. In principle, the same definitions apply to other in vitro tests. It is recommended that the type of test used for screening is indicated when reporting data

• Screened/tested
  The group of screened or tested participants are those who have used and returned an FOBT irrespective of the result. This includes people with inadequate/incomplete results. Note that each person is counted once regardless of the number of tests performed.

• Inadequate test
  An inadequate FOBT is a test returned by a participant, the results of which cannot be reliably determined (see Chapter 4 [15]). The quality is insufficient for processing and the test cannot be used for recording a result according to the programme policy.

• Positive test
  A positive i. e. abnormal FOBT result is a result based on the last adequate test that according to the programme policy leads directly to referral to follow-up colonoscopy.

• Referral to follow-up colonoscopy [2]
  This variable refers to participants with a positive FOBT who require an appointment for follow-up colonoscopy. Ideally all participants with positive FOBTs would be referred to follow-up colonoscopy.

3.2.3.2 Variables in endoscopic screening

The following process variables are described in the context of CRC screening in which either flexible sigmoidoscopy (FS) or colonoscopy (CS) is used as the primary screening test.

• Screened
  The group of screened participants comprises those people who have attended the FS or CS screening examination, irrespective of the result. This includes people with inadequate/incomplete results. Note that each person is counted once regardless of the number of exams performed.

• Inadequate test
  This group comprises those participants who attended the FS or CS screening examination, the results of which could not be interpreted because of inadequate preparation, and who do not have an adequate screening FS or CS in the reporting period. In such cases a new screening examination should be performed.

• Positive test
  A positive i. e. abnormal screening FS or CS is one resulting either directly in diagnosis of cancer or removal of an adenoma or other lesion, or in referral for further investigation according to the programme policy (see Chapters 2 [26] and 5 [44]).

• Referral to follow-up colonoscopy
  Included in this group are the participants with a positive screening FS or CS who require a medical appointment for follow-up colonoscopy.[3]

• Referral to surgery or tertiary endoscopy
  This group of participants includes those who require an appointment for surgery or tertiary endoscopy for removal of challenging lesions following a positive screening FS or CS (or as a consequence of follow-up colonoscopy after primary screening with FS or CS).

• Severe complications requiring hospitalisation
  A very small number of participants will develop severe complications such as hospitalisation within 30 days due to serious haemorrhage involving transfusion, or due to perforation, vagal syndrome or peritonitis-like syndrome as a consequence of primary screening with FS or CS (or as a consequence of follow-up colonoscopy for any primary screening test).

• 30-day mortality
  In a much smaller number of participants than those experiencing severe complications requiring hospitalisation, death may occur within 30 days after having undergone primary screening with FS or CS or follow-up colonoscopy, whether diagnostic or therapeutic, for any screening test. If the death is attributed to complications caused by the endoscopy, the participant should be counted in this group.

3.2.4 Programme outcome variables

The following outcome variables apply to CRC screening performed with any of the currently available primary screening tests.

Follow-up colonoscopy

Participants in the group on which diagnostic or therapeutic colonoscopy[4] has been performed to follow-up primary screening according to programme policy include participants, the screening endoscopy of which was inadequate or incomplete. Note that each person is counted once regardless of the number of follow-up colonoscopies that were performed. Where more than one colonoscopy or other follow-up investigation is performed, the reported result should be that of the complete diagnostic or therapeutic work-up.

Definitions of what is included in the reported result (e. g. grade of neoplasia,[5] TNM stage, other lesions) are given in Chapter 7 [33] (Sect. 7.2, Table  7.1, Rec. 7.1 – 7.5, 7.8).

If more than one lesion is found, then the lesion with the worst prognosis (see Ch. 7 [33]) should be indicated as the outcome of screening.

In the event of more than one detected lesion in a person where it is not possible to determine difference in prognosis, then the lesion requiring the most invasive procedure (see Ch. 7 [33]) and Ch. 8 [41]) should be recorded.

Lesions

Any lesion removed or biopsied at endoscopy or surgery (whether or not they were diagnosed as adenomas) should be recorded.

Adenomas

Pathological specimens removed at endoscopy or surgery, that have been reported by a pathologist to be adenomatous should be recorded.

Advanced adenoma

If it is not possible to collect such details for organisational reasons, the programme should at least focus on collecting and reporting data on adenomas ≥ 10 mm in size (see Ch. 9 [1], Sect. 9.1). For definition, see Ch. 7 [33], Sect. 7.2, and footnote 5 on previous page.

Cancers

Colorectal cancer diagnosed by the screening programme, or diagnosed as a direct result of participating in the screening programme (see Ch. 7 [33], Sect 7.2 for definition).

Severe complications requiring hospitalisation

For definition, see Sect. 3.2.3.2.

30 day mortality

For details, see Sect. 3.2.3.2.

3.2.5 Data tables

Recommendation

• For monitoring the programme, tables presenting performance indicators should be produced at regular intervals (at least annually) by age and gender and by type of screening test using the collected data (VI – A).^{Rec 3.5}

Tables should present data for people, not data for tests, and therefore each person is counted once regardless of the number of tests performed (see [Tab. 3.1]).

Table 3.1 List of recommended data tables to be produced by CRC screening programmes

1. Targeted

2. Eligible

3. Invited

4. Screened/tested at first screening and at subsequent screening episodes

5. Inadequate tests

6. Positive test or screening

7. Follow-up colonoscopy examination attended (diagnostic assessment and/or treatment)

8. Negative follow-up colonoscopy examination (diagnostic assessment and/or treatment)

9. Positive follow-up colonoscopy examination (diagnostic assessment and/or treatment)

10. Lesion detected (at least one)

11. Adenoma detected (at least one)

12. Non-advanced adenoma detected (at least one)

13. Advanced/high-risk adenoma detected (at least one)

14. Cancer detected by stage

Tables should record the number of people by age, sex and type of screening test in the respective reporting period. Where applicable, data should be broken down by initial and subsequent screening episodes.

They should present the participation in the programme, the main results of testing, and the main detection outcomes. When processing the data, decisions should be made regarding age. Age can be calculated according to different events (age at invitation, age at time of screening, age at time of diagnosis). Age at time of screening is preferable for indicators pertaining to the testing procedure, results and outcome. Age should be presented in 5-year groups.

3.3 Early performance indicators

Several rounds of screening are required before the impact of a screening programme on CRC mortality in the target population can be measured. Early performance indicators using standard definitions must therefore be used early in the lifetime of a screening programme to measure the quality of the screening process and to assess its potential longer-term impact. The accumulating experience in piloting and implementing population-based screening programmes provides an evidence base that can be used to establish and refine standards and set performance targets.

Factors affecting performance indicators

Coverage and uptake, i. e. participation, are organisational parameters that apply to CRC screening programmes using any kind of primary screening test. They have a substantial impact on the potential effectiveness of any screening programme because they reflect the degree to which the population is exposed to the screening intervention. Coverage and uptake in turn will be affected by the age and gender distribution of the target population due to differential uptake rates. Screening performance indicators will be affected by the age and gender distribution of the population screened due to variation in underlying incidence of disease.

Recommendation

• All indicators should be calculated and reported for age-gender subgroups (VI – A).^{Rec 3.6}

In addition, age-gender standardised measurements should be developed for comparative purposes.

Age should be recorded as the age of the person at the time of the invitation (for measurement of coverage/participation) or at time of screening (for measurement of screening outcome) for the respective screening round. The outcome of the screening examination for a person should thus be recorded in the same age category throughout a particular screening episode.

Screening performance indicators will also be affected by the background incidence in the target population in the absence of screening. Efforts should therefore be made to document age-gender specific incidence rates in the target population for the period immediately prior to the introduction of the screening programme.

If high-risk subjects are identified, managed, and/or excluded from the programme and reported separately, this should be stated.

Performance indicators will also vary according to whether the screen is a prevalent (first) screen for those invited for the first time, an incident (repeat) screen for those previously screened at the routine interval, or a screen for previous non-responders. Indicators at subsequent rounds will vary according to the screening interval.

Only the first organised screening round will consist entirely of subjects invited and attending for the first time; all additional rounds will comprise subjects falling into each of the categories described above. The cut-off point for separating ‘subsequent regular’ from ‘subsequent irregular’ screening should be established, taking into consideration that most programmes do not succeed in inviting each individual participant at the routine screening interval (e. g. a cut-off point at 30 months for a programme with a 2-year screening interval).

Data should be analysed separately for those invited/screened at:

• initial screening, i. e. the first invitation of individual people within the screening programme, regardless of the organisational screening round;

• subsequent invitation for previous never responders;

• subsequent invitation for those previously screened[6];

• screens as a result of self-referral (defined as people requesting screening before reception of an invitation or outside the invited age range); and

• screened following self-registration (those not recorded in target population).

[Tab. 3.2 – Tab. 3.5] list the key performance indicators for gFOBT, iFOBT, FS and colonoscopy respectively that have been reported from randomised controlled trials and from population-based programmes. For the majority of indicators the published values will have been influenced by the screening policy adopted in the respective trials and programmes. Other than those related to participation, the values reported here have therefore not been used to define acceptable levels.

Table 3.4 Evidence on performance indicators for flexible sigmoidoscopy

	Range from RCTs[1]	Range from population studies[2]
Uptake rate	32.4 % – 83.5 %	7 % – 55 %
Inadequate rate	11 % – 12.7 %	–
Positive rate	10.2 % – 27.7 %	1st round 5.4 % 2nd round 3.9 %
Referral rate for further investigation	8.3 % – 19.5 %	–
Adenoma detection rate	8.7 % – 20.6 %	14 % 5yr recall 11 %
Cancer detection rate	2.9‰ – 5.8‰	4‰ 5yr recall 0.0‰
Proportion of screen detected cancers Dukes stage A	54 % – 62 %	69 % (Stage I)
Severe complications Perforations Severe haemorrhage	0.02 % – 0.03 % Near to 0 %	–

¹ SCORE [39] age range 55 – 64
UKFS [43]age range 55 – 64
NORCCAP [13]age range 55 – 64
PLCO[48]age range 55 – 74
SCORE2 [38]age range 55 – 64
SCORE3 [40]age range 55 – 64
Netherlands[18]age range 50 – 74

² Italy [10]age range 50 – 74
UK [3]age range 60 – 64
Australia [45]age range 55 – 64
Italy [52]age range 50 – 69

There are a large number of possible process indicators, reflecting specific parts of the screening process. The present outline is confined to those that have epidemiological importance as identified within the trials. They measure participation, quality, efficacy, and organisation. Except for measures of participation, all other indicators are presented separately for in vitro tests (FOBT) and for endoscopic tests (FS or colonoscopy).

3.3.1 Programme coverage and uptake

Coverage and uptake, i. e. participation are organisational parameters that apply to CRC screening programmes using any kind of primary screening test.

Coverage by invitation

Coverage of the screening programme by invitation is the extent to which the invitations sent out by the screening programme within the defined screening interval include the eligible population. It gives information on the performance of the organisation of the programme in inviting the target population within the defined screening period.

The eligible population is defined in Ch. 2 [26], Sect. 2.3.1.1 (inclusion/exclusion criteria).

N people invited during the time frame*

N eligible people in the target population during the time frame** equal to the defined screening interval or reporting period, e. g. 12 months in the case of yearly reporting.Recommendation

• Invitation coverage should be high (95 %) in order to maximise screening impact. (VI – A).^{Rec 3.7}

Coverage by examination

Coverage of the screening programme by examination is the extent to which screening examinations have actually been delivered to the eligible population.

Screened here is defined as people tested at least once regardless of whether the result was adequate or inadequate and includes self referrals but not self registrations. The latter should be counted but reported separately. Coverage of the target age range for invitation will by definition exclude self referrals outside the age range. This is important in programmes where no comprehensive population lists are available and self referral or self registration can account for a large proportion of subjects screened.

Both of the coverage indicators (by invitation and examination) are useful at a local level to assess completeness of population lists and target population’s database.

N screened/tested during the time frame*

N eligible people in the target population during the time frame** equal to the defined screening interval or reporting period

Uptake (participation) rate

The number of people who have been screened, within a defined time frame following an invitation, as a proportion of all people who are invited to attend for screening.

The effectiveness of the programme will depend on the participation rate. In the randomised FOBT trials, uptake at the first round was between 49.5 % and 66.8 % ([Tab. 3.2]); uptake at subsequent rounds varied according to the policy for reinvitation. In a US study that recruited volunteers 75 % – 78 % of subjects invited were screened at least once [27]. Reported uptake in population-based programmes ranges from 17.2 % to 90.1 % at the first round; the range at subsequent rounds is smaller (22.3 % – 52.1 %) (see [Tab. 3.2] and [Tab. 3.3]).

Table 3.2 Evidence on performance indicators for guaiac based FOB testing.

	Range from RCTs[1]	Range from population-based programmes[2]
Uptake rate 1st round Subsequent round	49.5 % – 66.8 % 60 % – 94 %	17.2 % – 70.8 % 22.3 % – 52.1 %
Inadequate rate	–	0.4 % – 4.5 %
Positive rate for FOBT	1.2 % – 3.8 % (1.7 % – 15.4 %) (with rehydration)	1st screen 1.5 % – 8.5 % Subsequent screen 0.8 % – 1.8 %
Colonoscopy compliance rate	73 %[3] – 95 %	87.8 % – 91.7 %
Colonoscopy completion rate	89 % – 100 %	72 % – 95 %
Adenoma detection rate 1st screen Subsequent screen	5 – 14.5‰ 3.8‰	5.2 – 10.5‰ 3.3 – 4.7‰
Cancer detection rate1st screen Subsequent screen	1 – 2.5‰ 1.1 – 1.4‰	1.2 – 2.3‰ 0.9 – 0.94‰
Proportion of screen detected cancers that are stage A	26 % – 36 %	–
PPV for adenoma as the most severe lesion	14.6 % – 54.8 % (6.0 % – 11.0 %) (with rehydration)	30.3 % 26.8 %
PPV for cancer	5.2 % – 18.7 % (0.9 % – 6.1 %) (with rehydration)	1st screen 6.2 % – 8.5 % Subsequent screen 5.3 % – 10.6 %
Adverse effects (perforation, serious haemorrhage)	0.5 % – 1.6 % of subjects undergoing colonoscopy	–

¹ Minnesota[27] age range 50 – 80 annual and biennial, Hemoccult, 82.5 % rehydrated.
Goteborg [21] age range 60 – 64 2 screens at 16 – 24 month interval, Hemoccult II, majority hydrated.
Funen[22] age range 45 – 75 biennial, Hemoccult II not rehydrated.
Nottingham [16] age range 45 – 74 biennial, Hemoccult not rehydrated.
Netherlands[18]age range 50 – 74

² Greece [4]age range 50 + 
France [8]age range 50 – 74
Italy [10]age range 50 – 74
UK[17]age range 41 – 65
Spain [32]age range 50 – 69
UK [49]age range 50 – 69
Finland [25]age range 60 – 69

³ Others had an alternative such as barium enema

For flexible sigmoidoscopy, uptake rates in RCTs ranged from 32.4 % to 83.5 %, again with high rates being associated with recruitment of volunteers or those who had expressed interest in participation). In population-based programmes, uptake rates range from 7 % to 55 % ([Tab. 3.4]).

N people invited and screened/tested during the time frame*

N eligible people invited during the time frame ** equal to the defined screening interval or reporting period

Table 3.3 Evidence on performance indicators for iFOB testing

	Data from RCT[1]	Range from population-based programmes[2]
Uptake (participation) rate	61.5 %	17 % – 90.1 %
Inadequate rate	–	–
Positive rateRound 1 Any round Round 2	4.8 %	4.4 % – 11.1 %  7.1 %  3.9 %
Colonoscopy compliance rate	96 %	60 % – 93.1 %
Colonoscopy completion rate	98 %	–
Adenoma detection rate 1st screen	27.6‰	13.3 – 22.3‰
Cancer detection rate 1st screen 2nd screen	4.7‰	1.8 – 9.5‰  1.3‰
PPV adenoma 1st screen	59.8 %	19.6 % – 40.3 %
PPV cancer 1st screen 2nd screen	10.2 %	4.5 % – 8.6 %  4.0 %

¹ Netherlands[18]age range 50 – 74

² Italy[6]age range 50 – 74
Italy[14]age range 50 – 70
Uruguay[11]age range 50 + 
Japan[35]age range 40 + 

Recommendation

• A minimum uptake of at least 45 % is acceptable (III – A), but it is recommended to aim for a rate of at least 65 % [9] [52] (III – A).^{Rec 3.8}

3.3.2 Outcomes with faecal occult blood testing (FOBT) for primary screening

A table should be made to present the test results (positive, negative, or inadequate) by gender and age. Results should also be broken down by initial and subsequent screens as described above (Section 3.3). FOBT indicators will vary according to the type of test used and programme policy, and therefore these should be reported.

Inadequate FOBT rate

The rate of inadequate tests is defined as the proportion of people screened with one or more FOBT returned during the respective time frame (e. g. a 12-month period) none of which were adequate.

Rates of inadequate tests should remain low. They reflect, among other things, the understanding of the people who are using a test and therefore also the quality of the information provided to them.

In population-based programmes, inadequate gFOBT rates between 0.4 % and 4.5 % ([Tab. 3.2]) have been reported. No data are available yet for iFOBT.

N people who returned only inadequate FOBTs during the time frame*

N people tested during the time frame** equal to the defined screening interval or reporting period

Recommendation

• An inadequate rate of FOBT less than 3 % is acceptable, less than 1 % is desirable (see Ch.4 [15], Rec 4.21, Sect. 4.3.4) (III – A).^{Rec 3.9}

Positive FOBT rate

In the RCTs of gFOBT, the positive rate without rehydration was 1.2 % – 3.8 %, and with rehydration 1.7 % – 15.4 %. In average risk population-based programmes the positive rate for gFOBT in participants aged 50 – 69 years was 1.5 % – 8.5 % in the first round. Only two studies have reported rates at subsequent rounds, with positive rates of 0.8 % and 1.8 % ([Tab. 3.2]).

For iFOBT the range of positive rates in population-based studies was 4.4 % – 11.1 % in the first round, with one study reporting a rate in subsequent rounds of 3.9 % [52] ([Tab. 3.3]).

Positive test rates for gFOBT will depend on the method of slide handling used, and will be higher if the slides are rehydrated. The positive rate for iFOBT will vary according to the cut-off level adopted (see Chapter 4 [15]).

Positive rates should be presented in a table by 5 – year age groups and gender. Positive rates are higher in men than in women and increase with age in both genders reflecting the natural history of the disease.

N people with a positive FOBT result during the time frame*

N people adequately tested during the time frame** equal to the defined screening interval or reporting period

Referral to follow-up colonoscopy after FOBT

The rate of referral for follow-up colonoscopy after a positive FOBT is defined as the proportion of people screened with a positive test and referred to colonoscopy among those presenting with a positive/abnormal test during the respective time frame.

N people presenting with a positive test and referred for colonoscopy during the time frame *

N people presenting with a positive/abnormal test during the time frame** equal to the defined screening interval or reporting period

Recommendation

• High rates of referral to follow-up colonoscopy should be achieved for people with a positive screening test or examination requiring follow-up (90 % is acceptable, > 95 % is desirable) (VI – A).^{Rec 3.10}

Follow-up colonoscopy compliance rate

In the RCTs using gFOBT, colonoscopy compliance rates range from 73 % to 95 %; in population programmes rates between 88 % and 92 % have been reported. ([Tab. 3.2])

N people having attended a colonoscopy examination during the time frame*

N people presenting with a positive screening test and referred during the time frame** equal to the defined screening interval or reporting period

Recommendation

• High rates of compliance with follow-up colonoscopy should be achieved (85 % is acceptable, > 90 % is desirable) (III – A).^{Rec 3.14}

Follow-up colonoscopy outcome, detection rates

A table should be made to present colonoscopy results by gender and age:

• Negative, (defined as no identified lesions, adenomas or cancers);

• Presence of adenomas of any size;

• Presence of non-advanced adenomas;

• Presence of advanced adenomas; and

• Presence of advanced cancers.

The above colonoscopy indicators are essential programme indicators of efficacy.

Completion of follow-up colonoscopy after FOBT

The proportion of incomplete colonoscopies should be recorded (see Chapter 5 [44] for definition). One RCT of FOB testing reported a completion rate at follow-up colonoscopy of 89 % [22].

Recommendation

• A completion rate of follow-up colonoscopy of > 90 % is acceptable, > 95 % is desirable (see also Ch. 5 [44], Rec. 5.41) (III – A).^{Rec 3.11}

If more than one lesion is found, the lesion with the worst prognosis should be used for evaluation purposes as the result of follow-up colonoscopy.

In the event of more than one detected lesion in a person where it is not possible to determine difference in prognosis, then the lesion requiring the most invasive procedure should be recorded, (see Ch. 1 [23] and Ch. 7 [33]).

Detection rates of FOBT screening programme

• Lesion detection rate
  The lesion detection rate is reported in % and is defined as the proportion of participants with at least one detected lesion among those adequately tested during the respective time frame.

N people with at least one detected lesion during the time frame*

N people adequately tested during the time frame** equal to the defined screening interval or reporting period

• Adenoma detection rate
  The adenoma detection rate is reported per 1000 (‰) and is defined as the proportion of participants with at least one detected adenoma among those adequately tested during the respective time frame.

N people with at least one detected adenoma during the time frame*

N people adequately tested during the time frame** equal to the defined screening interval or reporting period

• Advanced adenoma detection rate
  The advanced adenoma detection rate is reported per 1000 (‰) and is defined as the proportion of participants with at least one detected advanced adenoma among those adequately tested during the respective time frame.

N people with at least one detected advanced adenoma during the time frame*

N people adequately tested during the time frame** equal to the defined screening interval or reporting period

• Cancer detection rate
  Detection rates for cancers and adenomas observed in population-based programmes using FOBT are summarised in [Tab. 3.2] and [Tab. 3.3]. Cancer detection rates range from 1.2‰ to 9.5‰ at the first round, with lower rates at subsequent rounds. Detection rates of all adenomas range from 5.2‰ to 22.3‰ at the first round, with lower rates at subsequent rounds. (However some studies report only advanced or high-risk adenomas.)

N people with at least one detected cancer during the time frame*

N people adequately tested during the time frame** equal to the defined screening interval or reporting period

• Stage of screen-detected cancers
  The stage distribution of screen-detected cancers should be reported by screening round, age and gender. In the RCTs using only gFOBT, the proportion of screen-detected cancers that were Dukes Stage A ranged from 26 % to 36 % ([Tab. 3.2]).

The staging of colon cancer should use firstly the international TNM classification and secondly the Dukes classification (see Chapter 7 [33]).

Recommendation

• A favourable stage distribution in screen-detected cancers compared to clinically diagnosed cancers should be observed. In absence of this condition a screening programme could not be effective (I – A).^{Rec 3.12}

Positive predictive values for FOBT screening programmes

Since lesions can only be detected if follow-up colonoscopy is performed, the definitions below take into account whether or not follow-up CS was actually performed. Other positive predictive values can be calculated, such as the PPV of the positive test without any further adjustment. In this case, the denominator would be the number of people presenting with a positive test result leading to referral for colonoscopy.

• PPV for detection of lesions
  The positive predictive value (PPV) for detection of a lesion through an FOBT screening programme is defined as the percentage of people with detection of at least one lesion at follow-up CS among those with positive FOBT tests who have attended follow-up CS.

N people with at least one detected lesion during the time frame*

N people positive to FOBT having attended a colonoscopy during the time frame** equal to the defined screening interval or reporting period

• PPV for detection of adenoma
  The positive predictive value for detection of an adenoma through an FOBT screening programme is defined as the percentage of people with detection of at least one adenoma at follow-up CS among those with positive FOBT tests who have attended follow-up CS.

N people with at least one detected adenoma during time frame*

N people positive to FOBT having attended a colonoscopy during the time frame** equal to the defined screening interval or reporting period

• PPV for detection of advanced adenoma
  The positive predictive value for detection of an advanced adenoma through an FOBT screening programme is defined as the percentage of people with detection of at least one advanced adenoma at follow-up CS among those with positive FOBT tests who have attended follow-up CS.

Values varied between 14.6 % and 54.5 % in the RCTs using only gFOBT without rehydration and from 6.0 % to 11.0 % with rehydration.

N people with at least one detected advanced adenoma advanced adenoma during time frame*

N people positive to FOBT having attended a colonoscopy during the time frame** equal to the defined screening interval or reporting period

• PPV for detection of cancer
  The positive predictive value for detection of a cancer through an FOBT screening programme is defined as the percentage of people with detection of at least one cancer at follow-up CS among those with positive FOBT tests who have attended follow-up CS. Values varied between 5.2 % and 18.7 % in the RCTs without rehydration and from 4.5 % to 8.6 % in the initial round of population-based programmes (5.3 % to 10.6 % in subsequent screening) ([Tab. 3.2] and [Tab. 3.3]).

N people with at least one detected cancer during time frame*

N people positive to FOBT having attended a colonoscopy during the time frame** equal to the defined screening interval or reporting period

Endoscopic complications in FOBT screening programme

In addition to death within 30 days, other serious complications that may be attributable to the endoscopic examination are described in Sect. 3.2.3.2. However, many different endoscopic complications can occur in FOBT screening programmes, all complications should be recorded as well as the respective cause, if ascertainable.

For any complication the rate is defined as the proportion of participants presenting with a complication among those having attended a colonoscopy during the respective time frame. The rate should be calculated in total and separately for screening and follow-up colonoscopy if applicable.

N people presenting with complication during the time frame*

N people having attended a colonoscopy during the time frame** equal to the defined screening interval or reporting period

Recommendation

• The rate of serious adverse effects should be monitored carefully (VI – A).^{Rec 3.13}

The RCTs in Nottingham and Minnesota showed that approximately 16 major complications due to follow-up CS occurred per 1 million persons screened with FOBT. This corresponds approximately to the risk of major complications from follow-up colonoscopy in a well-organised high-quality flexible sigmoidoscopy screening programme (see Ch. 1 [23], Sect. 1.2.1.4 and 1.3.1.4).

3.3.3 Outcomes with flexible sigmoidoscopy (FS) or colonoscopy (CS) as primary screening tests

A table should be made to present the test results (positive, negative, or inadequate) by gender and age. Results should also be broken down by initial and subsequent screens as described above (Sect. 3.3).

Inadequate FS or CS rates

An inadequate FS or CS occurs when the examination cannot be performed because of inadequate preparation. In two RCTs inadequate FS rates ranged from 11 % to 12.7 % ([Tab. 3.4]) [40] [48].

N people with an inadequate FS or CS, respectively, during the time frame*

N people screened with FS or CS, respectively, during the time frame** equal to the defined screening interval or reporting period

Complete FS or CS rates

FS and CS examinations are considered complete when conducted under adequate bowel preparation and with visualisation of the colon beyond the sigmoid-descending-colon-junction (FS), or the caecum (CS).

One RCT has reported a rate of incomplete CS examination of 7.5 % [40]. Other authors reported rates of 1.3 % and 8.9 % for CS [34] [36]. The recommended standard (unadjusted caecal intubation rate, see Ch. 5 [44], Sect 5.4.5.1) is > 90 %.

N people with complete FS or CS, respectively*

N people screened with FS or CS, respectively, under adequate bowel preparation* equal to the defined screening interval or reporting period

Endoscopy outcome tables

A table should be made to present the screening endoscopy results by gender and age:

• Negative, (defined as no identified lesions, adenomas or cancers);

• Presence of adenomas of any size;

• Presence of non-advanced adenomas;

• Presence of advanced adenomas; and

• Presence of cancers.

A similar table should be made to present the endoscopic results of follow-up colonoscopy in participants with positive FS or CS screening exams who are referred to follow-up colonoscopy (see below).

To calculate the following detection rates, the data of the two tables should be combined. Separate analysis of screening and follow-up endoscopy is also recommended for quality assurance purposes (see below: “Follow-up colonoscopy outcome tables”).

Positive FS or CS rate

The positive FS rate reported in different studies depends on the definition used (for example whether removed lesions not requiring further surveillance are recorded as a positive result or a negative result). The reported rates varied from 17.6 % to 27.7 % in 4 RCTs ([Tab. 3.4]). Positive CS rates ranging from 20.4 % to 53.8 % have been reported from population studies [24] [34] [36]. The latter rate was reported in a study with a high percentage of participants with a family history of CRC.

N people with a positive FS or CS result, respectively, during the time frame*

N people screened with FS or CS, respectively, during the time frame** equal to the defined screening interval or reporting period

Detection rates of FS or CS screening programmes

• Lesion detection rate
  The lesion detection rate is reported in % and is defined as the proportion of participants with at least one detected lesion among those adequately tested during the respective time frame.

Detection rates should be presented in a table by 5-year age groups and gender.

N people with at least one detected lesion during the time frame*

N people adequately tested with FS or CS, respectively, during the time frame** equal to the defined screening interval or reporting period

• Adenoma detection rate
  The adenoma detection rate is reported in % and is defined as the proportion of participants with at least one detected adenoma among those adequately tested during the respective time period.

N people with at least one detected adenoma during the time frame*

N people adequately tested with FS or CS, respectively, during the time frame** equal to the defined screening interval or reporting period

In the RCTs of flexible sigmoidoscopy, adenoma detection rates ranged from 8.7 % to 12.1 % ([Tab. 3.4]).

• Advanced adenoma detection rate
  The advanced adenoma detection rate is reported in % and is defined as the proportion of participants with at least one detected advanced adenoma among those adequately tested during the respective time period.

N people with at least one detected advanced adenoma during the time frame*

N people adequately tested with FS or CS, respectively, during the time frame** equal to the defined screening interval or reporting period

Advanced adenoma detection rates of 4.9 % to 8.6 % have been reported in population studies of colonoscopy [24] [34] [36] ([Tab. 3.5]).

• Cancer detection rate
  The cancer detection rate is determined as the proportion of FS or CS screening participants, respectively, with at least one detected colorectal cancer among those adequately examined during the respective time period. In the RCTs of flexible sigmoidoscopy, detection rates ranged from 2.9‰ to 5.4‰ ([Tab. 3.4]). Somewhat higher rates can be expected for screening CS due to inspection of the entire colon.

N people with at least one detected cancer during the time frame*

N people adequately tested with FS or CS, respectively, during the time frame** equal to the defined screening interval or reporting period

Referral to follow-up colonoscopy after FS or CS

The respective rate of referral for follow-up colonoscopy after a positive screening FS or CS is defined as the proportion of people with a positive screening examination and referred to colonoscopy among those presenting with a positive/abnormal screening exam during the respective time frame and requiring follow-up CS according to the programme policy. In the RCTs of flexible sigmoidoscopy, referral rates ranged from 8.3 % to 19.5 % of all participants with a positive FS ([Tab. 3.4]).

N people presenting with a positive FS or CS, respectively, and referred for follow-up CS during time frame*

N people presenting with a positive/abnormal FS or CS, respectively, and requiring follow-up during the time frame** equal to the defined screening interval or reporting period

As a percentage of all people with a positive test result, referral rates for follow-up colonoscopy will be much higher in FOBT-based screening programmes, than in FS screening programmes, depending on the programme policy for referral after a positive screening FS. Referral for follow-up CS after screening CS will be much less common than after screening FS because most lesions found at screening can be removed during screening CS. However, as a proportion of all people referred to follow-up according to the programme policy, compliance should be high irrespective of type of primary screening test.

Recommendation

• High rates of referral to follow-up colonoscopy should be achieved for people with a positive screening FS or CS requiring follow-up (90 % is acceptable, > 95 % is desirable) (VI – A).^{Rec 3.10}

Follow-up colonoscopy compliance rate after screening FS or CS

The rate of compliance with referral to follow-up colonoscopy after a positive endoscopic screening examination is defined as the proportion of people having attended a follow-up CS during the time frame among those presenting with a positive screening FS or CS, respectively, who were referred during the time frame.

N people having attended a follow-up CS examination during the time frame*

N people presenting with a positive screening FS or CS, respectively, and referred during the time frame** equal to the defined screening interval or reporting period

Recommendation

• High rates of compliance with follow-up colonoscopy should be achieved (85 % is acceptable, > 90 % is desirable) (VI – A).^{Rec 3.14}

Follow-up colonoscopy outcome tables

A table should be made to present colonoscopy results by gender and age:

• Negative (defined as no identified lesions, adenomas or cancer);

• Presence of adenomas of any size;

• Presence of non-advanced adenomas;

• Presence of advanced adenomas; and

• Presence of cancer.

As mentioned above, a similar table should be made to present the results of primary screening endoscopic exams. To calculate the programme detection rates of lesions, adenomas and cancers, the data of the two tables should be combined.

Completion of follow-up colonoscopy after FS or CS

The proportion of follow-up colonoscopies that are incomplete (lack of visualisation of the caecum, see Ch. 5 [44], Sect. 5.4.5.1) should be recorded.

Recommendation

• For follow-up colonoscopy after FS or screening CS, a completion rate of 90 % is acceptable, > 95 % is desirable (see also Ch. 5 [44], Rec. 5.41) (III – A).^{Rec 3.11}

If more than one lesion is found during follow-up colonoscopy, then the lesion with the worst prognosis should be used for the programme evaluation.

In the event of more than one detected lesion in a person where it is not possible to determine difference in prognosis, then the lesion requiring the most invasive procedure should be used for the evaluation database (see Sect. 3.2.4; Ch. 7 [33]).

Endoscopic complications of FS or CS screening programmes

The endoscopic complications that can appear in CRC screening programmes using FS or CS are the same as those described above with respect to follow-up colonoscopy performed in an FOBT screening programme (see Sect. 3.3.2).

The following complications are defined as serious: death within 30 days; or hospitalisation within 30 days due to serious haemorrhage involving transfusion, or due to perforation, vagal syndrome or peritonitis-like syndrome. All complications should be recorded as well as the respective cause, if discernible.

For any complication the rate is defined as the proportion of participants presenting with a complication among those having attended the respective type of endoscopic exam (FS or CS). Rates should be broken down by exams performed for primary screening and exams performed for follow-up of positive screening results.

N people presenting with complication of FS or CS, respectively, during time frame*

N people having attended the respective exam (FS or CS) during the time frame** equal to the defined screening interval or reporting period

In RCTs, rates of severe complications of FS have been reported at 0.02 % to 0.03 %[40] [48] . Three studies of colonoscopy screening have reported rates of severe complications of 0.0 % to 0.3 % [24] [34] [36]. In a well-organised high-quality flexible sigmoidoscopy screening programme the risk of major complications is about 0.3 % – 0.5 % for follow-up colonoscopy (III) (see also Ch. 1 [23], Sect. 1.2.1.4 and 1.3.1.4).

Recommendation

• The rate of serious adverse effects should be carefully monitored (VI – A).^{Rec 3.13}

Table 3.5 Evidence on performance indicators for screening colonoscopy

	Population studies[1]
Positive rate	20.4 % – 53.8 %[2]
Any adenoma or cancer detection rate	14.9 % – 37.5 %[2]
Advanced neoplasia detection rate	4.9 % – 10.5 %
Advanced adenoma detection rate	4.9 % – 8.6 %
Complication rate	0.0 % – 0.3 %

¹ US [36]women age range 50 – 79
US [24]men age range 50 – 75
Poland[34]age range 50 – 66

² High percentage of participants with family history of CRC Screening organisation

3.3.4 Screening organisation

A number of indicators can be used to monitor the organisational performance of a screening programme.

Time interval between completion of test and receipt of results

The time interval between performing a test and receipt of results will affect patient outcomes in terms of anxiety and potentially screening outcomes in terms of stage of diagnosis of disease.

Recommendation

• The time interval between completion of test and receipt of results by the subject should be as short as possible, (acceptable standard: > 90 % within 15 days) (VI – A).^{Rec 3.15}

Time interval between positive test and follow-up colonoscopy

A timely procedure is not critical in the context of primary screening but it is very important when endoscopy is performed following a previous positive screening test. A delayed procedure may not be critical biologically, but it can cause unnecessary anxiety for the screenee.

To ensure that patient anxiety is not unnecessarily increased, it is recommended that follow-up colonoscopy after positive screening be performed as soon as reasonably possible but no later than within 31 days of referral.

Recommendation

• Follow-up colonoscopy after positive screening (any modality) should be scheduled within 31 days of referral (an acceptable standard is > 90 %, > 95 % is desirable). (See Ch. 5 [44], Rec. 5.19, Sect. 5.3.5). (VI – B).^{Rec 3.16}

Time interval between positive endoscopy (CS or FS) and start of definitive management

The interval between the diagnosis of screen-detected disease and the start of definitive management is a time of anxiety for the patient and affords the opportunity, if prolonged, for disease progression. For these reasons, standards aimed at minimising delay have set the maximum interval at 31 days [31] (see Ch. 8 [41], Rec. 8.2, Sect. 8.2).

Recommendation

• The time interval between the diagnosis of screen-detected disease and the start of definitive management should be minimised. Acceptable standard: > 90 %, desirable > 95 % within 31 days (see Ch. 8 [41], Rec. 8.2) (VI – B). ^{Rec 3.17}

Time interval between consecutive primary screening tests

The time interval between two consecutive primary screening tests will affect the coverage of the programme by invitation/screening.

The interval between two consecutive primary screening tests should be monitored to remain within an acceptable level (depending on the screening interval). People should be re-invited according to the date of their last test and not that of their last invitation.

If possible data pertaining to endoscopic surveillance should be monitored.

Proportion of people referred for endoscopic surveillance and proportion of people complying to endoscopic surveillance.

3.4 Long-term impact indicators

The primary objective of screening for CRC is to achieve a reduction in disease-specific mortality; in the case of FS or colonoscopy screening this will be achieved largely by a reduction in the incidence of CRC. However such a reduction in either mortality or incidence will not be discernible until many years after the introduction of the screening programme. (In some areas, opportunistic screening by colonoscopy may be widespread before the start of the programme, therefore diluting the effect of a programme). Methods for studying mortality reduction are discussed later in this chapter. In the meantime other indicators of the impact of screening on disease incidence and mortality should be monitored. These include rates of interval cancers, and surrogate outcome measures that can be used to predict the impact of screening on CRC mortality (or on the incidence of invasive disease) such as rates of overall (age-specific) incidence, stage-specific incidence rates [8].

Costs associated with all aspects of the programme should be recorded. Estimates of cost effectiveness will vary according to the health care system in the area. Costs should be monitored carefully, but comparisons between countries will be complex. (Aspects of cost-effectiveness are discussed in Chapter 1 [23]).

Finding the appropriate networking level for evaluation of incidence and mortality depends on the organisational structure of the programme. In some programmes (e. g. UK) this will be at a national level, whereas for others it will be at a regional level.

Recommendation

• Evaluation of surrogate outcome measures requires rigorous data collection of bowel cancer registrations and stage of disease in the target population. It is also preferable that such data are collected for the time period leading directly up to the introduction of a screening programme to allow trends to be analysed (VI – A).^{Rec 3.18}

3.4.1 Interval cancers

Interval cancers are those that occur following a negative screening episode, in the interval before the next invitation to screening is due. For faecal occult blood testing interval cancers may occur following a negative FOBT, or following a positive test result with negative further assessment (colonoscopy). Rates of interval cancers reflect both the sensitivity of the screening test (false negatives), and the incidence of newly-arising cases not present at the time of screening. With increasing time since negative test, the rate and proportion of the latter will increase. In the absence of repeat screening, incidence rates would eventually reach the background level again. Rates of interval cancers should therefore be presented by time period (years) since previous screen.

For endoscopy screening and for colonoscopy follow-up of FOBT, interval cancers reflect the quality of screening as well as the sensitivity of the screening test.

Recommendation

• Data on interval cancers should be collected and reported (VI – A).^{Rec 3.19}

Recommendation

• Evaluation of interval cancer rates requires careful linkage of cancer registrations with screening history to allow cancers to be classified (i. e. as screen detected, interval, non-responders, other). The requisite linkage must be established with the cancer registry (VI – A). ^{Rec 3.20}

Rates of interval cancers will depend on the underlying incidence in the population. They will also depend on the extent of selection bias, whereby rates in those not participating in screening differ from the general population rates. For this reason it is important that (age- and gender-specific) incidence rates in non-responders are also monitored, to allow for the underlying incidence in responders to be estimated.

Background incidence rates can be estimated from rates prior to the introduction of screening (although time trends need to be considered) or from areas not covered by the screening programme (when geographic differences need to be considered).

The interval cancer rate can therefore be expressed as a proportion of the background incidence rate, standardised for age and gender, by dividing the number of interval cancers in the specific age/gender group (I) by the ones expected based on the background incidence for that age/gender group (C), or as a proportion of the background incidence rate adjusted for non-participants (C*). The adjusted rate can be calculated as:

C* = (C – (1 – P) N) / P

P: participation rate

N: rate in non-responders

The comparisons can be adjusted for differences in age and gender.

The rate of interval cancers in the period after a negative screening provides information on the sensitivity of the programme. The sensitivity of gFOBT-based program for detection of cancer has been estimated as 55 % – 57 % using this method. In the Nottingham trial the estimate was based on overall rates of interval cancers of 0.64 per 1000 person-years in the two year period after screening [30]. Using the same method, the sensitivity of iFOBT-based programme has been reported as 82 % [51].

No data are available yet on the sensitivity of FS or colonoscopy-based programmes.

3.4.2 CRC incidence rates

Immediately following the introduction of a screening programme, incidence rates in the target age range should increase due to the detection of prevalent disease by screening. At re-screening, rates should return to background level apart from the advancement of the age of diagnosis by screening.

Age- and gender-specific incidence rates should therefore be reported over time. FS screening should eventually lead to a reduction in incidence rates due to detection and removal of adenomas of the distal colon, but as discussed above this is a long-term effect. Screening FOBT may also have an eventual impact in reducing incidence rates, but the effect will be less due to lower detection rates of adenomas.

Cumulative incidence rates or proxies should be used to monitor potential over-diagnosis of cancer, that is cancer that would not otherwise appear during the lifetime of the individual.

3.4.3 Rates of advanced-stage disease

Screening (both FOBT and FS) should result in a reduction in the overall population incidence of late stage disease (DUKES C & D) prior to any reduction in mortality and can therefore be used as an early indicator of effectiveness. Because screening will result in the detection of a large number of early stage cases, and hence a reduction in the proportion of late stage disease, it is preferable to monitor rates of late stage disease. The ability to do this will depend on the completeness of stage information that ideally should be available for a sufficiently lengthy period immediately prior to the introduction of the screening programme, to allow trends to be studied.

Projected mortality based on stage-adjusted cancer incidence.

Models have been developed to use prognostic information provided by Dukes stage and age at diagnosis to predict cancer mortality.

3.4.4 CRC mortality rates

As discussed above, it will be several years before the impact of population screening on CRC mortality becomes observable, and many more years before the full effect is achieved. The timing of a reduction depends on the natural history of the disease, and the ‘lead time’ due to screening (i. e. the time by which screening advances the date of diagnosis) as well as on the time taken to cover the target population. It will also depend on the quality of screening.

Methods to evaluate the impact of screening on CRC mortality include analyses of population trends, cohort studies (aggregated or individual-based) and case-control studies.

Population trends

Mortality from CRC has been decreasing in many European countries since the mid 1990’s [20]. Analyses of the routinely produced age-gender specific population rates over time will be subject to limitations due to the dilution of the effect of screening from deaths occurring in cases diagnosed prior to the introduction of screening, and/or at an age below which invitations begin. This can be overcome by use of refined CRC mortality rates in which such deaths are excluded. However, the rates will also be confounded by other factors such as cohort effects on underlying incidence, and by the effects of improvements in treatment and/or the stage of diagnosis of symptomatic disease on survival and mortality. Thus whilst a lack of any reduction in population mortality rates several years after the introduction of a screening programme should be a cause for concern, it is difficult to use such trends to quantify the effect, and attempts to do so should take account of the factors discussed above.

Cohort studies

In some settings, the introduction of population screening will have been phased in such a way as to facilitate comparisons of populations invited at different time points. Such a model has been used in Finland (see Ch. 2 [26], Sect. 2.6.4). In the absence of such a system, comparisons can be made between geographical areas (regions invited/not invited to screening) or between the same population in different time periods before and after the introduction of screening. Both types of comparison are liable to possible bias due to underlying differences in the risk in the populations/time-periods. This may – under certain circumstances – be compensated for by including also a comparison group from geographic areas where no invitational program existed from before the introduction of screening. Cohort studies using aggregated data need estimates of incidence in order to avoid dilution effect discussed above.

These biases can be avoided by individual-based cohort studies in which deaths and cancer registrations are linked to screening histories.

Case-control studies

Case control studies that compare ‘exposure’ (i. e. ‘screening’) between cases (deaths from CRC) and controls are an attractive alternative to cohort studies in terms of cost and effort. However, careful consideration of the design issues is necessary to avoid a number of potential biases [19]. The major problem with such studies is that of selection bias, due to different levels of underlying risk in participants and non-participants with screening. Methods to adjust for this are required both to estimate the mortality benefit in those actually screened, and the ‘impact’ on the population invited for screening.

Conclusions

In a multidisciplinary process, wide consensus has been achieved on a comprehensive package of evidence-based recommendations for quality assurance in evaluation and interpretation of outcomes in colorectal cancer screening. Following these recommendations has the potential to enhance the control of colorectal cancer in Europe and elsewhere through improvement in the quality and effectiveness of screening programmes and services.

Disclaimer

The views expressed in this document are those of the authors. Neither the European Commission nor any person acting on its behalf can be held responsible for any use that may be made of the information in this document.

Competing interests: Dr Brenner’s research institute (German Cancer Research Center, DKFZ) has received significant research support for previously and currently running studies on colorectal cancer detection from the company Eiken Chemicals (less than 40 000 €).
The following companies have provided the DKFZ with faecal occult blood tests free of charge for previously and currently running evaluation studies: ulti med, Ahrensburg, Germany; DIMA, Gottingen, Germany; Beckman Coulter, Krefeld, Germany; CAREdiagnostica; Voerde, Germany; Preventis, Bensheim, Germany; Quidel, San Diego, California. The total value of the non-monetary support is less than 100 000 €.

Acknowledgements

The comments and suggestions contributed by Josep Espinàs, Spain; are gratefully acknowledged.

The comments and suggestions received from the following reviewers are gratefully acknowledged:

Jack Cuzick, United Kingdom, and Elsebeth Lynge, Denmark.

The comments and suggestions received from consultation of the European Cancer Network are gratefully acknowledged.

The production of the Guidelines was supported by the European Union through the EU Public Health Programme, (grant agreement no. 2005317: Development of European Guidelines for Quality Assurance of Colorectal Cancer Screening). Partner institutions: Oxford University Cancer Screening Research Unit, Cancer Epidemiology Unit, University of Oxford, Oxford, United Kingdom; Unit of Cancer Epidemiology, Centre for Cancer Epidemiology and Prevention (CPO) and S. Giovanni University Hospital, Turin, Italy; Public Association for Healthy People, Budapest, Hungary; European Cancer Patient Coalition (ECPC), Utrecht, Netherlands; Quality Assurance Group, Section of Early Detection and Prevention, International Agency for Research on Cancer, Lyon, France.

Financial support was also received through the Public Affairs Committee of the United European Gastroenterology Federation, and from a cooperative agreement between the American Cancer Society and the Division of Cancer Prevention and Control at the Centers for Disease Control and Prevention.

¹ Sect (superscript) after each recommendation in the list refers the reader to the section/s of the Guidelines dealing with the respective recommendation.*
Rec (superscript) throughout the chapter refers to the number of the recommendation dealt with in the preceding text.*
* The first digit of the section numbers and recommendation numbers refers to the respective chapter in the guidelines. For Chapters 1 and 2 see: [23] [26]; for Chapters 4 to 10 see: [1] [2] [15] [33] [41] [42] [44] respectively.

² The process variables related to performance of follow-up colonoscopy as a result of a positive FOBT test are the same as for follow-up colonoscopy as a result of a positive FS or CS screening examination. They are therefore described in Section 3.2.3.2 (“referral to surgery or tertiary endoscopy”, “severe complications requiring hospitalisation”, “30-day mortality”).

³ In rare cases in which follow-up colonoscopy is not possible, other follow-up examinations may be performed. Those patients should be included in the group referred to follow-up CS but should also be counted separately.

⁴ See previous footnote on follow-up colonoscopy.

⁵ In screening programmes the use of the term “advanced adenoma” has developed and is sometimes used to categorise adenomas for management. In the present context an advanced adenoma is one that is either ≥ 10 mm or contains high-grade mucosal neoplasia or a villous component (Ch. 7 [33]).

⁶ Where possible, these should be separated into invitations at the routine screening interval defined by the screening policy, and subsequent invitations at irregular intervals, i. e. those who have been screened at least once who do not respond to an invitation to routine re-screening and are invited in a subsequent organisational screening round [or attend a subsequent screening more than a defined time frame after the previous test].

⁷ equal to the defined screening interval or reporting period, e. g. 12 months in the case of yearly reporting.

⁸ equal to the defined screening interval or reporting period

• References

• 1 Atkin W, Valori R, Kuipers EJ et al. European guidelines for quality assurance in colorectal cancer screening and diagnosis: First edition. Colonoscopic surveillance following adenoma removal. Endoscopy 2012; 44: SE151-SE163
• 2 Austoker J, Giordano L, Hewitson P et al. European guidelines for quality assurance in colorectal cancer screening and diagnosis: First edition. Communication. Endoscopy 2012; 44: SE164-SE185
• 3 Brotherstone H, Vance M, Edwards R et al. Uptake of population-based flexible sigmoidoscopy screening for colorectal cancer: a nurse-led feasibility study. J Med Screen 2007; 14: 76-80
• 4 Chrissidis T, Saliangas K, Economou A et al. Mass screening for colorectal cancer: compliance in Almopea Region. Tech Coloproctol 2004; 8: s193-s195
• 5 Council of the European Union. Council Recommendation of 2 December 2003 on cancer screening (2003/878/EC). Off J Eur Union; 2003: 34-38
• 6 Crotta S, Castiglione G, Grazzini G et al. Feasibility study of colorectal cancer screening by immunochemical faecal occult blood testing: results in a northern Italian community. Eur J Gastroenterol Hepatol 2004; 16: 33-37
• 7 Day NE, Williams DR, Khaw KT. Breast cancer screening programmes: the development of a monitoring and evaluation system. Br J Cancer 1989; 59: 954-958
• 8 Denis B, Ruetsch M, Strentz P et al. Short term outcomes of the first round of a pilot colorectal cancer screening programme with guaiac based faecal occult blood test. Gut 2007; 56: 1579-1584
• 9 Faivre J, Arveux P, Milan C et al. Participation in mass screening for colorectal cancer: results of screening and rescreening from the Burgundy study. Eur J Cancer Prev 1991; 1: 49-55
• 10 Federici A, Marinacci C, Mangia M et al. Is the type of test used for mass colorectal cancer screening a determinant of compliance? A cluster-randomized controlled trial comparing fecal occult blood testing with flexible sigmoidoscopy. Cancer Detect Prev 2006; 30: 347-353
• 11 Fenocchi E, Martinez L, Tolve J et al. Screening for colorectal cancer in Uruguay with an immunochemical faecal occult blood test. Eur J Cancer Prev 2006; 15: 384-390
• 12 Ferlay J, Shin HR, Bray F et al. GLOBOCAN 2008 v1.2, Cancer Incidence and Mortality Worldwide: IARC CancerBase No. 10. [Internet] Lyon, France: International Agency for Research on Cancer; 2010 Available from: http://globocan.iarc.fr Accessed on 05/04/2012
• 13 Gondal G, Grotmol T, Hofstad B et al. The Norwegian Colorectal Cancer Prevention (NORCCAP) screening study: baseline findings and implementations for clinical work-up in age groups 50-64 years. Scand J Gastroenterol 2003; 38: 635-642
• 14 Grazzini G, Castiglione G, Ciabattoni C et al. Colorectal cancer screening programme by faecal occult blood test in Tuscany: first round results. Eur J Cancer Prev 2004; 13: 19-26
• 15 Halloran S, Launoy G, Zappa M. European guidelines for quality assurance in colorectal cancer screening and diagnosis: First edition. Faecal Occult Blood Testing. Endoscopy 2012; 44: SE65-SE87
• 16 Hardcastle JD, Chamberlain JO, Robinson MH et al. Randomised controlled trial of faecal-occult-blood screening for colorectal cancer. Lancet 1996; 348: 1472-1477
• 17 Hart AR, Glover N, Howick-Baker J et al. An industry based approach to colorectal cancer screening in an asymptomatic population. Postgrad Med J 2003; 79: 646-649
• 18 Hol L, van Leerdam ME, van Ballegooijen M et al. Screening for colorectal cancer: randomised trial comparing guaiac-based and immunochemical faecal occult blood testing and flexible sigmoidoscopy. Gut 2010; 59: 62-68
• 19 Hosek RS, Flanders WD, Sasco AJ. Bias in case-control studies of screening effectiveness. Am J Epidemiol 1996; 143: 193-201
• 20 Karim-Kos HE, de Vries E, Soerjomataram I et al. Recent trends of cancer in Europe: a combined approach of incidence, survival and mortality for 17 cancer sites since the 1990s. Eur J Cancer 2008; 44: 1345-1389
• 21 Kewenter J, Brevinge H, Engaras B et al. Results of screening, rescreening, and follow-up in a prospective randomized study for detection of colorectal cancer by fecal occult blood testing. Results for 68,308 subjects.. Scand J Gastroenterol 1994; 29: 468-473
• 22 Kronborg O, Fenger C, Olsen J et al. Randomised study of screening for colorectal cancer with faecal-occult-blood test. Lancet 1996; 348: 1467-1471
• 23 Landsorp-Vogelaar I, von Karsa L. European guidelines for quality assurance in colorectal cancer screening and diagnosis: First edition 1. Introduction. Endoscopy 2012; 44: SE15-SE30
• 24 Lieberman DA, Weiss DG, Bond JH et al. Use of colonoscopy to screen asymptomatic adults for colorectal cancer. Veterans Affairs Cooperative Study Group 380.. N Engl J Med 2000; 343: 162-168
• 25 Malila N, Oivanen T, Malminiemi O et al. Test, episode, and programme sensitivities of screening for colorectal cancer as a public health policy in Finland: experimental design. BMJ 2008; 337: 2261
• 26 Malila N, Senore C, Armaroli P. European guidelines for quality assurance in colorectal cancer screening and diagnosis: First edition. Organisation. Endoscopy 2012; 44: SE31-SE48
• 27 Mandel JS, Bond JH, Church TR et al. Reducing mortality from colorectal cancer by screening for fecal occult blood. Minnesota Colon Cancer Control Study.. N Engl J Med 1993; 328: 1365-1371
• 28 Minozzi S, Armaroli P, Banzi R et al. European guidelines for quality assurance in colorectal cancer screening and diagnosis – First edition. Appendix 1: Systematic evidence review. 2010 http://bookshop.europa.eu/en/european-guidelines-for-quality-assurance-in-colorectal-cancer-screening-and-diagnosis-pbND3210390/ Accessed 11/2/2012
• 29 Minozzi S, Armaroli P, Segnan N. European guidelines for quality assurance in colorectal cancer screening and diagnosis: First edition. Principles of evidence assessment and methods for reaching recommendations. Endoscopy 2012; 44: SE9-SE14
• 30 Moss SM, Hardcastle JD, Coleman DA et al. Interval cancers in a randomized controlled trial of screening for colorectal cancer using a faecal occult blood test. Int J Epidemiol 1999; 28: 386-390
• 31 NHS. Bowel Screening Programme Clinical Standards. Scotland: NHS Quality Improvement; 2007 http://www.bowelscreening.scot.nhs.uk/wp-content/uploads/2007/06/bowelsc_stnf_feb07.pdf Accessed 11/2/2012
• 32 Peris M, Espinas JA, Munoz L et al. Lessons learnt from a population-based pilot programme for colorectal cancer screening in Catalonia (Spain). J Med Screen 2007; 14: 81-86
• 33 Quirke P, Risio M, Lambert R et al. European guidelines for quality assurance in colorectal cancer screening and diagnosis: First edition. Quality assurance in pathology in colorectal cancer screening and diagnosis. Endoscopy 2012; 44: SE116-SE130
• 34 Regula J, Rupinski M, Kraszewska E et al. Colonoscopy in colorectal-cancer screening for detection of advanced neoplasia. N Engl J Med 2006; 355: 1863-1872
• 35 Saito H. Colorectal cancer screening using immunochemical faecal occult blood testing in Japan. J Med Screen 2006; 13 (Suppl. 01) 6-S7
• 36 Schoenfeld P, Cash B, Flood A et al. Colonoscopic screening of average-risk women for colorectal neoplasia. N Engl J Med 2005; 352: 2061-2068
• 37 Segnan N, Patnick J, von Karsa L (eds.) European guidelines for quality assurance in colorectal cancer screening and diagnosis – First edition. Luxembourg: European Commission, Publications Office of the European Union; 2010
• 38 Segnan N, Senore C, Andreoni B et al. Randomized trial of different screening strategies for colorectal cancer: patient response and detection rates. J Natl Cancer Inst 2005; 97: 347-357
• 39 Segnan N, Senore C, Andreoni B et al. Baseline findings of the Italian multicenter randomized controlled trial of "once-only sigmoidoscopy"--SCORE. J Natl Cancer Inst 2002; 94: 1763-1772
• 40 Segnan N, Senore C, Andreoni B et al. Comparing attendance and detection rate of colonoscopy with sigmoidoscopy and FIT for colorectal cancer screening. Gastroenterology 2007; 132: 2304-2312
• 41 Steele RJC, Pox C, Kuipers EJ et al. European guidelines for quality assurance in colorectal cancer screening and diagnosis: First edition. Management of lesions detected in colorectal cancer screening. Endoscopy 2012; 44: SE31-SE48
• 42 Steele RJC, Rey J-F, Lambert R. European guidelines for quality assurance in colorectal cancer screening and diagnosis: First edition. Professional requirements and training. Endoscopy 2012; 44: SE15-SE30
• 43 UK Flexible Sigmoidoscopy Screening Trial Investigators. Single flexible sigmoidoscopy screening to prevent colorectal cancer: baseline findings of a UK multicentre randomised trial. Lancet 2002; 359: 1291-1300
• 44 Valori R, Rey J-F, Atkin W et al. European guidelines for quality assurance in colorectal cancer screening and diagnosis: First edition. Quality assurance in endoscopy in colorectal screening and diagnosis. Endoscopy 2012; 44: SE88-SE105
• 45 Viiala CH, Olynyk JK. Outcomes after 10 years of a community-based flexible sigmoidoscopy screening program for colorectal carcinoma. Med J Aust 2007; 187: 274-277
• 46 von Karsa L, Anttila A, Ronco G et al. Cancer Screening in the European Union. Report on the implementation of the Council Recommendation on Cancer Screening – First Report. Luxembourg: European Commission; 2008 http://ec.europa.eu/health/archive/ph_determinants/genetics/documents/cancer_screening.pdf Accessed 11/2/2012
• 47 von Karsa L, Lignini TA, Patnick J et al. The dimensions of the CRC problem. Best Pract Res Clin Gastroenterol 2010; 24: 381-396
• 48 Weissfeld JL, Schoen RE, Pinsky PF et al. Flexible sigmoidoscopy in the PLCO cancer screening trial: results from the baseline screening examination of a randomized trial. J Natl Cancer Inst 2005; 97: 989-997
• 49 Weller D, Coleman D, Robertson R et al. The UK colorectal cancer screening pilot: results of the second round of screening in England. Br J Cancer 2007; 97: 1601-1605
• 50 Wilson JM, Jungner YG. Principles and practice of mass screening for disease. World Health Organization; 1968 http://whqlibdoc.who.int/php/WHO_PHP_34.pdf Accessed 11/2/2012
• 51 Zappa M, Castiglione G, Paci E et al. Measuring interval cancers in population-based screening using different assays of fecal occult blood testing: the District of Florence experience. Int J Cancer 2001; 92: 151-154
• 52 Zorzi M, Falcini F, Fedato C et al. Screening for colorectal cancer in Italy: 2006 survey. Epidemiol Prev 2008; 32: 55-68