Download PDF
Open Access
CC BY-NC-ND 4.0 · Endosc Int Open 2022; 10(04): E549-E557
DOI: 10.1055/a-1766-5259
Original article

Cumulative sum learning curves guiding multicenter multidisciplinary quality improvement of EUS-guided tissue acquisition of solid pancreatic lesions

Authors

  • Hannah M. Schutz‡

     1   Department of Gastroenterology and Hepatology, Reinier de Graaf Gasthuis, Delft, the Netherlands

  • Rutger Quispel‡

     1   Department of Gastroenterology and Hepatology, Reinier de Graaf Gasthuis, Delft, the Netherlands

  • Bart J. Veldt

     1   Department of Gastroenterology and Hepatology, Reinier de Graaf Gasthuis, Delft, the Netherlands

  • Frank M.M. Smedts

     2   Department of Cyto- and Histopathology, Reinier de Graaf Gasthuis, Delft, the Netherlands

  • Marie-Paule G.F. Anten

     3   Department of Gastroenterology and Hepatology, Franciscus Gasthuis en Vlietland, Rotterdam, the Netherlands

  • Klaas J. Hoogduin

     4   Department of Cyto- and Histopathology, Pathan B.V., Rotterdam, the Netherlands

  • Pieter Honkoop

     5   Department of Gastroenterology and Hepatology, Albert Schweitzer Ziekenhuis, Dordrecht, the Netherlands

  • Francien H. van Nederveen

     6   Department of Cyto- and Histopathology, PAL Dordrecht, Dordrecht, the Netherlands

  • Lieke Hol

     7   Department of Gastroenterology and Hepatology, Maasstad Ziekenhuis, Rotterdam, the Netherlands

  • Mike Kliffen

     8   Department of Cyto- and Histopathology, Maasstad Ziekenhuis, Rotterdam, the Netherlands

  • Claire E. Fitzpatrick

     9   Department of Gastroenterology and Hepatology, IJsselland Ziekenhuis, Capelle aan den Ijssel, the Netherlands

  • Nicole S. Erler

    10   Department of Biostatistics, Erasmus Medical Center, Rotterdam, the Netherlands

  • Marco J. Bruno

    11   Department of Gastroenterology and Hepatology, Erasmus Medical Center, Rotterdam, the Netherlands

  • Lydi M.J.W. van Driel

    11   Department of Gastroenterology and Hepatology, Erasmus Medical Center, Rotterdam, the Netherlands
    • on behalf of QUEST
Further Information
Also available at
 
 PDF Download Permissions and Reprints

Abstract

Background and study aims In this study, we evaluated the performance of community hospitals involved in the Dutch quality in endosonography team regarding yield of endoscopic ultrasound (EUS)-guided tissue acquisition (TA) of solid pancreatic lesions using cumulative sum (CUSUM) learning curves. The aims were to assess trends in quality over time and explore potential benefits of CUSUM as a feedback-tool.

Patients and methods All consecutive EUS-guided TA procedures for solid pancreatic lesions were registered in five community hospitals between 2015 and  2018. CUSUM learning curves were plotted for overall performance and for performance per center. The American Society of Gastrointestinal Endoscopy-defined key performance indicators, rate of adequate sample (RAS), and diagnostic yield of malignancy (DYM) were used for this purpose. Feedback regarding performance was provided on multiple occasions at regional interest group meetings during the study period.

Results A total of 431 EUS-guided TA procedures in 403 patients were included in this study. The overall and per center CUSUM curves for RAS improved over time. CUSUM curves for DYM revealed gradual improvement, reaching the predefined performance target (70 %) overall, and in three of five contributing centers in 2018. Analysis of a sudden downslope development in the CUSUM curve of DYM in one center revealed temporary absence of a senior cytopathologist to have had a temporary negative impact on performance.

Conclusions CUSUM-derived learning curves allow for assessment of best practices by comparison among peers in a multidisciplinary multicenter quality improvement initiative and proved to be a valuable and easy-to-interpret means to evaluate EUS performance over time.


 

Introduction

Endoscopic ultrasound (EUS)-guided tissue acquisition (TA) is first choice for establishing a tissue diagnosis in suspected pancreatic cancer [1]. The increasing use of neoadjuvant chemotherapy for pancreatic carcinoma, and the fact that neoadjuvant treatments require pathological confirmation of the diagnosis, have rendered quality of EUS-guided TA of solid pancreatic lesions ever more important [2] [3]. Proficiency in EUS-guided TA can only be reached in centers in which all its aspects, including TA, tissue handling, microscopic assessment and reporting, are safeguarded. Feedback on performance is key to improving quality [4].

In 2015, the American Society of Gastrointestinal Endoscopy (ASGE) defined the following key performance indicators (KPIs) for EUS-guided TA in solid pancreatic lesions: rate of adequate sample (RAS) with a performance target of 85 %, diagnostic yield of malignancy (DYM) with a performance target of 70 %, and sensitivity for malignancy (SFM) with a performance target of 85 % [5]. RAS mainly reflects the quality of the process within the endoscopy suite (TA, preparation of smears, including transport to the cytopathology lab), whereas DYM and SFM reflect the quality of the entire process, including patient selection, specimen preparation, microscopic assessment and reporting.

Currently, quality control for the yield of EUS-guided TA is not customary or required for centers performing EUS-guided TA. Quality measurements for EUS-guided TA procedures were previously described as a monitoring tool during the development of academic or regional EUS programs [6] [7] [8]. Wani et al. used CUSUM curves to describe the development of competence in advanced endoscopy trainees performing both EUS and ERCP [9] [10] [11] [12] [13]. CUSUM curves reflect development of quality delivered in time relative to predefined performance targets.

In 2015 the Dutch Quality in Endosonography Team (QUEST) was founded. This is a regional EUS interest group, consisting of endosonographers and pathologists from five community hospitals in the Netherlands. QUEST aims to improve performance of EUS-guided TA by providing feedback on KPIs of individual centers based on a prospective registration of consecutive EUS-guided TA procedures of solid pancreatic lesions. This has led to improvements in RAS (80 % to 95 %), DYM (28 % to 64 %), and SFM (63 % to 84 %) comparing the results of an initial retrospective analysis of yield of EUS-guided TA to the first 21 months of prospective registration [14].

This study evaluated the use of CUSUM curves to monitor performance of contributing centers regarding the yield of EUS-guided TA of solid pancreatic lesions. Using this tool, we aimed to assess trends in KPIs over time, and explore potential benefits of CUSUM curves as a feedback-tool.


 

Patients and methods

This was a prospective, multicenter, quality improvement study of consecutive EUS-guided TA procedures on solid pancreatic lesions conducted in five community hospitals in the Netherlands. The local medical ethics committee (METC Zuidwest Holland 17–038) approved the study protocol. Informed consent was obtained from all patients. The study is registered in the Dutch trial registry (NTR) with trial number NL9470.

Study population and data collection

All patients aged 18 and older with a solid pancreatic lesion with high suspicion of malignancy who underwent an EUS-guided TA procedure were eligible for this study. Primary outcome parameters were CUSUM-derived learning curves with RAS and DYM as input parameters. RAS was defined as proportion of procedures yielding specimen sufficient for cytopathological and/or histopathological analysis. DYM was defined as the proportion of procedures yielding a “suspicious for malignancy” or a “malignant” diagnosis. The secondary outcome parameter was SFM. SFM was defined as the total of true positives (“suspected malignancy” or “malignancy” based on EUS-guided TA with a malignancy as final diagnosis) divided by all patients with a final diagnosis of malignancy.

Collected data on EUS-guided TA procedures included: patient demographics, localization of the pancreatic mass, hospital, endosonographer, pathologist, needle diameter ( < 22-gauge or 22-gauge), type of needle (fine-needle aspiration [FNA]/fine-needle biopsy [FNB]), number of passes, use of suction (slow withdrawal of stylet or vacuum suction), availability of rapid on-site specimen evaluation (ROSE), and the result of the cytopathological and/or histopathological evaluation of the EUS-guided TA specimen. Based on current practice guidelines and previous experience of our group, endosonographers were advised to perform at least three passes with FNA needles or at least two passes with FNB needles (unless ROSE detected sufficient material for diagnosis earlier), and to use vacuum suction [14] [15]. All other techniques and materials used were at the discretion of the local clinicians and according to local availability of equipment and hospital standards.

The results of cytopathological and/or histopathological evaluation were classified as follows: non-diagnostic, benign, atypical, suspicious for malignancy, and malignant. Neuroendocrine tumors were classified as malignant. For the purpose of this study “suspicious for malignancy” and “malignant” were both considered malignant. All types of pancreatic and periampullary malignancies were considered a malignant reference standard. The gold standard for a malignant diagnosis was based on either histopathological diagnosis after surgical resection or progression of disease compatible with malignancy during a minimum of 12 months of follow-up.


 

Feedback on performance

Regional interest group meetings were organized three times a year. Prior to meetings, all contributors received data regarding the performance of their individual center accompanied by anonymized benchmark data from the other centers. At the regional interest group meetings, the results of prospective registration, best practices, guidelines, and difficult cases were discussed. Until 2017, feedback on performance overall and per center was provided as RAS, DYM, and SFM (proportions). From 2018 onward, visual feedback by means of CUSUM curves of RAS and DYM was also provided. At meetings all data (numbers and CUSUM curves) were presented (in an anonymized fashion) and subsequently discussed. Participating endosonographers and pathologists were invited to reflect on changes in directions of the curves provided. Significant changes in the direction of the curve were subjected to further analysis, of which, the results were discussed separately with the practitioners from the centers involved, prior to the next general meeting. At a subsequent meeting, the results of these analyses were presented and discussed, with emphasis on potential learning opportunities for all participants. All gastroenterologists and pathologists involved had completed their training at least 3 years before the start of this study [14].


 

Statistics

Cumulative sum analysis (CUSUM)

Each EUS procedure was scored as a success (adequate sample/malignant outcome) or failure (inadequate sample/non-malignant outcome). Each success is rewarded with adding score s, each failure results in subtraction of (1 – s). Each procedure is a dot in the learning curve that is created by a plot of the cumulative sum of all cases in chronological order.

The acceptable rates (P0) and unacceptable rates (P1) were defined based on the ASGE KPIs and a previous publication by Eltoum et al. [16]. For inadequate samples, we designated 10% as acceptable (P0) and 15 % as unacceptable (P1) rates. For a nonmalignant outcome of the EUS, the P0 was defined as 25 % and the P1 as 30 %.


 

Decision limits

Two decision limits (h1 and h0) were calculated. The decision limits are calculated based on type I (α) and type II (β) errors. A type I error is the risk of rejection of a true null hypothesis and a type II error is the risk of non-rejection of a false null hypothesis. The formulas that are used to calculate h0 and h1 were previously described [16]. The meaning of the decision limits in relation to the curve can be explained as follows: [17] [18]

  1. If the learning curve crosses the upper decision limit, the failure rate is within the preset acceptable range and it reflects high quality.

  2. If the learning curve crosses the lower decision limit, the failure rate is above the preset unacceptable rates and an intervention is needed.

  3. If the learning curve remains between the two decision limits, the performance is within the preset acceptable range.


 

CUSUM charts

CUSUM charts were constructed using Excel. Each success (adequate sample/malignant outcome) contributes to an upward slope of the CUSUM curve. Each inadequate sample will contribute to a downward slope of the CUSUM curve. A downslope curve means that the key performance indicator is not met. A horizontal curve indicates that quality is up to standards. An upslope curve signifies quality is above the predefined key performance indicator threshold.


 

Multivariable analysis

To investigate the association of RAS and DYM with procedure characteristics, we fitted logistic mixed models. Given the limited number of inadequate samples, only two parameters (suction: yes/no and ROSE: yes/no) could be included in the RAS model.

The model for the DYM included the variables suction type (no, slow withdrawal of stylet or vacuum), ROSE, number of passes (continuous), needle size (< 22-gauge, 22-gauge) and needle type (FNA or FNB). In both models we used endoscopist specific (random) intercepts to take into account that samples obtained by the same endoscopist may not be independent. The model for DYM also included a pathologist specific (random) intercept. Both models were fitted in the Bayesian framework, which allowed us to include observations for which some of the covariates were missing. We used normal priors with mean 0 and standard deviation 100 for all regression coefficients. The Bayesian models were fitted using Markov chain Monte Carlo, with the help of the freely available and widely used “JAGS” software [19] that uses Gibbs sampling and provides a wide range of samplers to sample from full-conditional distributions that do not have a closed form. Results are presented as posterior mean and 95 % confidence interval (CI). Calculations were performed in R version 4.0.2 (2020–06–22) (R Core Team 2020) and the package JointAI 1.0.0.9000 [20]. Missing observations were imputed during the analysis.


 

 

 

Results

From January 2015 until December 2018, 431 EUS-guided TA procedures on solid pancreatic lesions in 403 individual patients were included. The median age of the patients was 68 years (range 27–88), and 51 % were men. During follow-up, a pancreatic or periampullary malignancy (reference standard) was diagnosed in 87 % of all cases. Per hospital, two to four endosonographers were involved in these procedures. A wide range of eight to sixteen pathologists per hospital were involved ([Table 1]).

Table 1

Characteristics of the participating patients and hospitals.

Total cohort

(n = 403)

A
(n = 79)

B
(n = 88)

C
(n = 81)

D
(n = 94)

E
(n = 61)

Sex male, n (%)

206 (51 %)

43 (54 %)

42 (48 %)

40 (49 %)

54 (57 %)

27 (44 %)

Median age in years (range)

68 (27–88)

70 (42–86)

68 (43–86)

68 (27–87)

67 (33–88)

68 (35–88)

Reference standard malignant, n (%)

351 (87 %)

69 (87 %)

77 (88 %)

68 (84 %)

81 (86 %)

56 (92 %)

Number of endoscopists involved

15

 2

4

2

3

 4

Number of pathologists involved

39

16

8

8

8

14

Rate of adequate sample overall and per hospital

A total of 399 of 431 procedures yielded an adequate sample. Hence, RAS was 93 % for the complete cohort (range 86 %–99% among individual hospitals). The ASGE-defined KPI of RAS ≥ 85 % was met overall and in each of the individual hospitals ([Table 2]). This can also be appreciated from the upslope direction of the overall learning curve drawn for this parameter (Supplementary Fig. 1). The RAS learning curves of the individual hospitals indicate adequate and stable quality (curves between the decision limits) in Hospitals A, B, and E, and adequate and improving quality in Hospitals C and D (Supplementary Fig. 2, Supplementary Fig. 3, Supplementary Fig. 4, Supplementary Fig. 5, Supplementary Fig. 6).

Table 2

Values of RAS, DYM and SFM for the complete cohort and per hospital.

Hospital

No. of procedures

RAS

DYM

SFM

A

 87

 75 (86 %)

 53 (61 %)

68 %

B

 91

 82 (90 %)

 57 (63 %)

71 %

C

 90

 87 (97 %)

 59 (66 %)

79 %

D

100

 99 (99 %)

 75 (75 %)

87 %

E

 63

 56 (89 %)

 41 (65 %)

73 %

Total cohort

431

399 (93 %)

285 (66 %)

76 %

Italics: equal or above ASGE performance target.

RAS, rate of adequate sample; DYM, diagnostic yield of malignancy; SFM, sensitivity for malignancy.


 

Diagnostic yield of malignancy overall and per hospital

A total of 285 of 431 procedures yielded a malignant diagnosis. Therefore, the overall DYM was 66 % (ranging from 61 %–75 % in the individual hospitals). This is below the KPI of DYM ≥ 70 % ([Table 2]). The overall learning curve of this parameter has a downslope direction (crossing the lower decision limit) until January 2018 ([Fig. 1a]). From this point onward, the curve has a more horizontal direction between the newly constructed decision limits, indicating an adequate and stable quality throughout 2018 ([Fig. 1a] and [Fig. 1b]).

Zoom
Fig. 1 DYM CUSUM learning curve of the complete cohort. a January 2015 to December 2018. b January 2018 to December 2018. DYM, diagnostic yield of malignancy.

In only one of the contributing hospitals (Hospital D) the KPI of DYM ≥ 70 % was met overall ([Table 2]). However, the learning curves of the individual hospitals for this parameter developed from an initial downslope (Hospitals B and E) or horizontal direction (Hospitals C and D) into a horizontal (Hospitals B, C, and E) or an upslope direction (Hospital D) ([Fig. 2a], [Fig. 3a], Supplementary Fig. 7a, Supplementary Fig. 8a, Supplementary Fig. 9a). This indicates a gradual improvement in these centers up to an adequate quality level in 2018.

Zoom
Fig. 2 DYM CUSUM curve of hospital a January 2015 to December 2018. b January 2018 to December 2018. Black arrow marks the decrease in evaluating pathologists from nine to three. DYM, diagnostic yield of malignancy.
Zoom
Fig. 3 DYM CUSUM curve of hospital C. a January 2015 to December 2018. b October 2017 to December 2018. Black arrows mark the temporarily absence of one experienced cytopathologist. DYM, diagnostic yield of malignancy.

The CUSUM curve for Hospital B started with a downward slope and in January 2018, the curve suddenly improved to a horizontal slope ([Fig. 2a] and [Fig. 2b]).

The curve of Hospital C initially showed a stable and adequate quality until May 2017. From this point onward there was a remarkable short and sharp downslope development of the curve, which again developed in a more horizontal direction from September 2017 onward ([Fig. 3a] and [Fig. 3b]). This indicates a 4-month episode during which a significantly lower number of malignant diagnoses were made. During these 4 months, a high proportion of specimens with atypia (40 %) was graded in comparison to the episodes prior to May 2017 (4 %) and from September 2017 onward (11 %) (Supplementary Table 1). The 4-month episode coincided with the temporary absence of the most experienced cytopathologist in this center, who had been involved in all cytopathological evaluations of pancreatic lesions in the previous years in this hospital.


 

Sensitivity for malignancy overall and per hospital

The overall SFM for the contributing hospitals throughout the 4 years of this study was 76 %, ranging from 68 % to 87 % among different hospitals. The KPI of SFM ≥ 85 % was not met in four of five contributing hospitals. The developments in the learning curves regarding DYM suggest improvement in quality in the majority of these centers. In 2018, the final year of this study, the overall SFM was 85 %, ranging from 69 % to 96 % among the centers. In this year, the KPI of SFM ≥ 85 % was met in three of five centers (Supplementary Table 2).


 

FNB versus FNA needles

A total of 282 FNA procedures and 127 FNB procedures were performed. The outcome of FNA and FNB procedures was similar (Supplementary Table 3) The use of FNB needles did not increase over time.


 

Multivariable analysis

Nine observations for which all covariates were missing were excluded from the analysis. Missing values in the remaining 422 observations were imputed (missing values: suction type 4.7 %, needle brand 2.8 %, number of passes 2.1 %, needle size 1.7 %, needle type 0.9 %, ROSE 0.2 %, and suction 0.2 %). The use of any type of suction and the presence of ROSE were positively associated with RAS, with odds ratios of 3.2, 95 % CI (1.1–7.8) and 2.8, 95 % CI (1.1–8.4), respectively ([Table 3]). There was no clear evidence that any of the covariates considered was associated with DYM ([Table 3]).

Table 3

Odds ratios and corresponding 95 % CIs for the logistic mixed models for RAS and DYM.

RAS

DYM

Covariate

OR

95 % CI

Covariate

OR

95 % CI

Use of suction (vacuum and/or slow-withdrawal of stylet)

3.2

1.1 – 7.8

No suction

0.7

0.3 – 1.6

ROSE

2.8

1.1 – 8.4

Vacuumsuction

1.1

0.5 – 2.3

ROSE

1.5

0.9 – 2.4

Number of passes

1

0.8 – 1.4

 < 22G needle (FNA and/or FNB)

1.5

0.4 – 4.9

22G needle (FNA and/or FNB)

0.9

0.6 – 1.5

FNB

1.1

0.7 – 2.1

There were missing values in seven covariates, with a percentage of missing observations

per variable ranging from 0 % to 5 %. These missing observations were imputed during the analysis.

RAS, rate of adequate sample; DYM, diagnostic yield of malignancy; OR, odds ratio; CI, confidence

interval; FNA, fine needle aspiration; FNB, fine needle biopsy; ROSE, rapid on-site evaluation.


 

Feedback and interpretation of curve deflections

During the 4 years of prospective registration, the following changes were reported by contributing practitioners. Hospitals A, D and E requested ROSE on a regular basis, which they did not do before. Hospital A started with ROSE halfway into 2016, Hospital D from January 2018 onward, and Hospital E at beginning of 2016. In Hospitals B and C, there were changes in the number of pathologists involved in EUS-guided TA procedures of the pancreas. In Hospital B, the group of pathologists that reviewed pancreatic samples collected with EUS was downsized from eight to three in January 2018. The most experienced cytopathologist from Hospital C was temporarily absent during a 4-month period in 2017.

The time that the events previously described took place are marked with an arrow in [Fig. 2a], [Fig. 3a], Supplementary Fig. 6, Supplementary Fig. 7a, Supplementary Fig. 8, Supplementary Fig. 8a, and Supplementary Fig. 9a.


 

 

Discussion

This study evaluated the performance of five community hospitals regarding the yield of EUS-guided TA of solid pancreatic lesions using CUSUM curves to assess trends in quality over time and explored potential benefits of CUSUM curves as a feedback tool. Throughout the 4 years of this study, all three ASGE defined KPIs improved. A KPI for RAS ≥ 85 % was met consistently in most of the centers and overall (93 %). A KPI of DYM ≥ 70 % was not met overall throughout the study between 2015 and 2018, but eventually yielded 75 % overall in 2018. Similarly, the KPI for SFM ≥ 85 % was not met overall from 2015 to 2018, but improved to 85 % in 2018. Because not all ASGE-defined KPIs are consistently met in each center, feedback on performance and analyses for potential improvements are indicated and ongoing.

The diagnostic yield of EUS-guided TA for solid pancreatic lesions is considered a benchmark for quality measurements in EUS [1]. However, the majority of studies in which the ASGE-defined KPI are based were performed in tertiary care facilities [21]. Moreover, the majority of publications on EUS-guided TA in solid pancreatic lesions were controlled trials focusing on discrete factors influencing the yield, i. e. different types and diameters of needles, use of suction, the use of ROSE, or the optimal number of passes to perform [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34]. Therefore, when comparing the current study to these previous publications, it cannot be ruled out that differences regarding patient selection may have influenced yield of EUS-guided TA. Nevertheless, questioning the generalizability of the benchmark data may never be an excuse to stop monitoring and improving your performance.

To improve quality of EUS-guided TA, it is necessary to provide feedback on performance. For providing feedback, CUSUM-derived learning curves have several advantages over tables with numbers. First, their interpretation is easy and does not require any knowledge about specific KPI values (a downward trend is not good, a horizontal line is good, and an upward trend is better). Second, they allow determination of best practices and comparison among peers. Third, they provide a more detailed picture of development over time, allowing for focused analysis of performance within specific timeframes [35]. The analysis of the sudden downslope deflection in the DYM curve of Hospital C, coinciding with the 4-month absence of a senior cytopathologist, is an excellent example of this. Analysis of this specific example teaches us how vulnerable the multistep process of EUS-guided TA is, being dependent on each factor or operator involved. Therefore, the discriminating advantage of learning curves for feedback over tables with numbers is that they provide additional learning opportunities.

RAS and DYM are obviously related. However, because CUSUM curves of these variables reflect quality relative to a predefined quality target, they do not necessarily develop in the same direction. An upward RAS curve, therefore, does not mean the DYM curve has to be upward as well. In other words: Having a sample that contains at least a couple of cells from the target organ (adequate sample) does not automatically mean that a pathologist will be confident about the malignant origin of the lesion. This can lead to a RAS above the performance target and a DYM and SFM below the performance target.

Supported by feedback provided by CUSUM analyses, several changes regarding protocols and/or staff involved were made in individual hospitals. In Hospital C today, a pathology report regarding pancreatic cytology or histopathology can only be finalized after consent of a dedicated cytopathologist. Several hospitals implemented routine use of ROSE and the number of pathologists involved was reduced in one of the centers. Although multivariable analysis supports the use of suction and ROSE to be beneficiary to RAS, an overall positive effect of these changes can be assumed. After all, with a RAS of 85 %, the lowest acceptable level according to ASGE definitions, the SFM can never exceed 85 %, and makes DYM ≥ 70 % in patients with solid pancreatic lesions difficult to achieve.

To our knowledge, this is the largest prospective multicenter study of EUS-guided TA of solid pancreatic lesions from community hospitals and the first to implement CUSUM-derived learning curves as a tool for monitoring and improving KPI of these procedures. Previous publications on the use of CUSUM curves in EUS-guided TA investigated performance of either cytopathologists or endoscopy trainees [9] [10] [11] [12] [13] [16]. In contrast to these studies, we used CUSUM curves to evaluate the entire process defining quality and yield of these procedures, including the work of both endosonographers and cytopathologists. Some of the data presented in this study (133 procedures, performed from January 2015 to September 2016) were previously described in the initial publication about this community hospital quality initiative [14]. The current study shows ongoing and persistent improvement in performance and introduces learning curves as a feedback and monitoring tool.

The main limitation of this study is the fact that feedback, either in tables with numbers or as learning curves, was not provided real time. Ideally, CUSUM curves would have been drawn three times a year, enabling contributing centers to respond more quickly to changes in curve directions. Because of logistic challenges and the time-consuming nature of data collection, this could not be realized in the current study. Another limitation is the fact that in the current study, no subtypes of FNB needles were recorded. Recent publications indicate improved outcome with a subtype of FNB needles over FNA needles [36]. The fact that no difference between FNA and FNB was detected in our study may be related to the unclear mix of subtypes of FNB needles used. However, other confounders such as the endosonographer learning curve for a new type of needle or pathologist learning curve for evaluating tissue cores may have been involved.

Future directions

Performing EUS-guided TA comes with the responsibility to measure KPI regarding these procedures. To facilitate this, an automated system is needed allowing EUS-procedural parameters and concomitant pathology reports to be added on regular basis. Subsequently CUSUM curves can be constructed based on KPI data at any point in time, allowing for constant trend analysis thereby providing the fundament for quality improvement. We believe that feedback on KPI is an essential first step for quality improvement. If KPIs are not up to par, this should be followed by a cycle of protocol changes and continued KPI measurements and evaluations (plan-do-check-act cycle), aiming for continuous improvement of quality and life-long learning opportunities for all collaborators.

Changes in protocol are to be tailored and center-specific, depending on KPI measurements and available resources. A measure aiming to increase a low adequate sample rate in a center using 22-gauge FNA needles, three passes and suction, for example could be: 1. The introduction of ROSE; or 2. The introduction of an FNB needle. If the hospital involved does not have its own cytopathology lab, implementation of FNB needles could solve their problem. A measure aiming to increase DYM, with current adequate RAS and high proportions of atypia diagnoses, for example, might be: 1. Reorganization of the workflow in the pathology lab to have all samples evaluated by two cytopathologists instead of seven; 2. Introducing liquid based cytology instead of smears only; or 3. Introducing the use of FNB needles. There is evidence to support that changes made “bottom-up” are more likely to be sustained in comparison to changes implemented “top-down” [37].


 

Conclusions

In conclusion, this prospective multicenter study using CUSUM-derived learning curves for both quality monitoring and feedback demonstrates consistent improvement of KPIs RAS, DYM, and SFM over time. It illustrates the benefits of using learning curves with easy-to-interpret feedback regarding performance of a whole process or its individual components while also allowing comparison with peers. Use of CUSUM curves is an excellent way for responsible staff to monitor and scrutinize their performance and improve the outcome of KPI up to the desired level.


 

 

 

Competing interests

The authors declare that they have no conflict of interest.

Supplementary material


Corresponding author

Hannah M. Schutz
Reinier de Graaf Gasthuis – Gastroenterology and Hepatology
Reiner de Graafweg 5 Delft Zuid
Holland 2625 AD
Netherlands   
Fax: +31152609111   

Publication History

Received: 03 September 2021

Accepted after revision: 03 December 2021

Article published online:
14 April 2022

© 2022. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany


  Permissions and Reprints
Zoom
Fig. 1 DYM CUSUM learning curve of the complete cohort. a January 2015 to December 2018. b January 2018 to December 2018. DYM, diagnostic yield of malignancy.
Zoom
Fig. 2 DYM CUSUM curve of hospital a January 2015 to December 2018. b January 2018 to December 2018. Black arrow marks the decrease in evaluating pathologists from nine to three. DYM, diagnostic yield of malignancy.
Zoom
Fig. 3 DYM CUSUM curve of hospital C. a January 2015 to December 2018. b October 2017 to December 2018. Black arrows mark the temporarily absence of one experienced cytopathologist. DYM, diagnostic yield of malignancy.