Introduction
Advanced gastrointestinal endoscopy (AGIE) encompasses an increasing number of different
procedures for which high complexity is the common denominator. In this context, there
is growing demand for dedicated subspecialty training, namely in endoscopic retrograde
cholangiopancreatography (ERCP) and endoscopic ultrasound (EUS) [1], to deepen the knowledge and acquire the skills needed to achieve competence [2]. Due to the fact that these therapeutic procedures are not only complex but are
also associated with severe complications more frequently in non-expert hands [3], proper training and selection are paramount.
To ensure high-quality training in ERCP/EUS, it is fundamental to determine who should
be trained. Trainee selection should be fair and based on predefined criteria to predict
future performance. However, subjective factors, such as the selector’s personal opinion,
may affect this process. Consequently, training program directors’ (TPDs) personal
values and beliefs involved in selection of a trainee should be better clarified to
improve the selection of the future AGIE workforce.
The current literature reports fellowship selection criteria and methods used in various
medical fields, including gastroenterology [4]
[5]
[6]
[7]
[8], but there is lack of data in the AGIE field. Additionally, there is also little
information about the particular influence of selectors’ personal values and beliefs
on the judgment and decision-making process. For example, there are publications focused
on other medical specialties [9], but there are no specific reports in ERCP/EUS, an area where most guidelines agree
nowadays that both techniques should be taught together at least for therapeutic purposes
[10].
Which tools are used for selecting ERCP/EUS trainees? What do TPDs/experts value,
believe, and consider when judging a potential candidate interested in training in
ERCP/EUS? These questions are still to be answered. Clarifying the factors involved
in this process is important because selectors decide who will be admitted to AGIE
training programs and have an essential role in ensuring that future advanced endoscopists
are skilled enough to provide quality, safe, and effective care. Furthermore, young
physicians applying for these training programs should know what TPDs/experts are
expecting from them.
In this study we aimed:
-
First, to identify TPDs/experts values and beliefs regarding the critical personal
attributes used for selecting AGIE trainees that are likely to excel in ERCP/EUS,
as well as those that may lead to a training disqualification;
-
Second, to compare perspectives between TPDs/experts and trainee opinions regarding
the training selection process and personal characteristics needed to be selected
as a trainee. In addition, to investigate the general approach and satisfaction regarding
the current trainee application process for ERCP/EUS training.
Methods
Study design and participants
We developed an online survey instrument that merged topics from an earlier international
survey (unpublished data). A list of potential trainee characteristics which could
influence selection and training in ERCP/EUS programs was compiled from the answers
to this previous survey and from the literature [4]
[8]
[9]
[11]
[12] and used as a starting point. An invitation was sent by e-mail to TPDs/experts from
a list of 45 European ERCP/EUS training centres. In addition, a request was made to
each TPDs/experts to provide up to three AGIE trainee contacts who could participate
in the survey and an invitation was also sent to them to complete a trainee-specific
form. Participants were contacted to participate in the study between January and
March 2022.
Development and content of survey instrument
Two electronic survey instruments were prepared, one for TPDs/experts and the other
for trainees (Supplementary material, Appendix 1). The available version of Google
forms was used to conduct the survey. Data on individual participants were not collected.
The four-part surveys comprised 50 and 33 questions, for TPDs/experts and trainees,
respectively, structured into the following categories: Section 1, enquiring about
general opinion regarding application process and trainee selection; Section 2, evaluating
the current application process for ERCP/EUS training programs; Section 3, defining
trainee qualities needed to excel in ERCP/EUS; and Section 4, defining trainee characteristics
that could justify disqualification from ERCP/EUS training.
The question format varied and included open-ended, check all that apply, multiple-choice,
yes or no, and 5-point Likert scale (anchors ranging from “strongly agree” to “strongly
disagree”; or from “extremely important” to “not important”) questions. The surveys
also included boxes for free-text comments.
The final survey version was revised and distributed after pilot testing among the
authors.
Survey distribution and collection of data
The online survey was sent to TPDs/experts and trainees via e-mail. The survey was
presented with a short introduction explaining the purpose of the collected information
and acknowledging the contribution of the participants. Participation constituted
voluntary consent to the study. Additionally, a link to the survey was added. Two
direct mailing reminders were sent to non-respondents to maximize participation in
the study. All data per user was automatically recorded into a software database (Microsoft
Excel).
Ethics committee approval was not obtained, since no patient data were collected for
the study.
Answers to the questionnaire were voluntary and individual responses stayed confidential
and were only seen by the investigators analysing the data. Therefore, published data
are described as averages or as totals from the group, no individual responses were
reported and the data is not directly traceable to the contributors.
Study endpoints
The primary outcome was to ascertain TPD/expert values and beliefs regarding the critical
personal attributes used for selection of AGIE trainees (ERCP/EUS programs), including
those that indicate a trainee is likely to be successful and those that may lead to
training disqualification.
The secondary outcomes were to compare perspectives between TPDs/experts and trainees
regarding the selection process and personal characteristics needed to be selected
as a trainee; and to evaluate the general approach and satisfaction regarding current
selection process for AGIE training programs.
Data analysis
A descriptive analysis was conducted on all study variables. Categorical variables
were described through absolute and relative frequencies, and continuous variables
were described using mean and standard deviation, quartiles, minimum and maximum values.
Frequencies of answers were compared between trainees and trainers with Pearson’s
Chi-squared or Fisher’s exact test, depending on expected frequencies. The statistical
significance level was set at 0.05. The analyses were performed with Stata/SE 17.0.
Results
Eighty percent (36 of 45) TPDs/experts answered the questionnaire. Among the trainees,
the response rate was 38.5 % (25 of 65 invited). Trainees were from 17 different departments.
The 18 European countries represented in the survey are shown in [Fig. 1].
Fig. 1 Geographical distribution of both TPDs/experts and trainee respondents to survey
on ERCP/EUS training.
ERCP/EUS trainee selection
The evaluation of TPDs/experts and trainees concerning the current selection process
for ERCP/EUS training is shown in [Fig. 2].
Fig. 2 TPD/expert and trainee opinions regarding current ERCP/EUS trainee selection process.
There is a general agreement between TPDs/experts and trainees regarding the selection
process (P = 0.08; Supplementary material, Appendix 2, Table 1).
Past ERCP/EUS trainee evaluations
The majority of the TPDs/experts were either “very satisfied” (n = 26; 72.2 %) or
“extremely satisfied” (n = 5; 13.2 %) with the trainees who graduated from their fellowship
program in the last five years. Five of them (13.9 %) were “neutral” and none were
“somewhat unsatisfied” or “not at all satisfied.” Those who were “neutral” explained
their reasoning further: “time allowed for the training is limited and is standardized
based on very inadequate criteria at a national level, getting trainees to have an
inadequate period for training.”
Current process for trainee selection in ERCP/EUS training
The application process was conducted through:
-
Individual request, 86.1 % (n = 31)
-
Curriculum vitae (CV), 41.7 % (n = 15)
-
Formal interview, 38.9 % (n = 14)
-
Recommendation letter, 30.6 % (n = 11)
-
Endoscopy society fellowship grant, 27.8 % (n = 10)
-
Motivation letter, 27.8 % (n = 10)
-
Payment of an application fee, 8.3 % (n = 3)
-
Practical evaluation, 8.3 % (n = 3)
-
Theoretical evaluation, 0 cases
A request from another institution (n = 2) was also mentioned as a method for the
application.
Ideal trainee characteristics needed to excel in ERCP/EUS training
The value of the following ERCP/EUS trainee characteristics was rated according to
TPDs/experts and trainee opinions, as shown in [Fig. 3].
Fig. 3 Rating of trainee characteristics regarding their importance to TPDs/experts (left
graphic) and trainees (right graphic).
There is a general agreement between TPDs/experts and trainees regarding the importance
of the abovementioned trainee characteristics to excel in ERCP/EUS (Supplementary
material, Appendix 2, Table 2), with the exception of “honesty”, which is considered
to be slightly more important for trainers than for trainees (e. g. extremely important
rates of 72.2 % versus 44.0 %, respectively; P = 0.032).
Moreover, all participants were asked to select the three most crucial trainee characteristics
from all the ones they had previously considered to be “extremely important” to excel
in ERCP/EUS ([Fig. 4]).
Fig. 4 Trainee characteristics considered to be “extremely important” according to TPDs/experts
(blue columns) and trainee opinion (green columns).
There is a general agreement between TPDs/experts and trainees regarding their opinions
about the characteristics of trainees considered to be “extremely important” (Supplementary
material, Appendix 2, Table 3), the only exception being “to be a team player”, considered
extremely important by eight TPDs/experts but by no trainees (22.2 % versus 0 %, respectively;
P = 0.017).
Definition of technical skills and clinical judgement
The importance of technical skills and clinical judgement, as characteristics for
a trainee to excel in ERCP/EUS training, was defined by both TPDs/experts and trainees
and is shown in [Fig. 5].
Fig. 5 Definitions for “technical skills” according to a TPDs/experts and c trainee opinion; and “clinical judgement” according to b TPDs/experts and d trainee opinion.
None believed that clinical judgement could not be taught ([Fig. 5b] and [Fig. 5d]) and trainees did not consider clinical judgement to be “empathy for the patient”
([Fig. 5d]). One trainee defined technical skills differently, as “Precise maneuvering of endoscope
and equipment, tip control as well as recognizing and acting on scope positions and
situations” and clinical judgement, as “Thoroughly study the individual case and evaluate
the patient. Know the indications of the technique to be applied and the risk-benefit
balance. Establish a therapeutic plan.”
There is general agreement between TPDs/experts and trainees regarding their opinion
about definition of both “clinical judgement” (P = 0.837) and “technical skills” (P = 0.308) (Supplementary material, Appendix 2, Table 4).
Criteria for ERCP/EUS trainee disqualification
Twenty-six TPDs/experts (72.2 %) had identified fellows who performed below the expected
level of competence for ERCP/EUS procedures. This determination was made by:
-
Inadequate performance on specific quality metrics, 51.9 % (n = 14)
-
Inadequate performance on a skills assessment tool, 37 % (n = 10)
-
Inadequate procedure volume, 25.9 % (n = 7)
-
Inadequate verbal attending evaluations, 18.5 % (n = 5)
-
Inadequate fellow self-reporting, 11.1 % (n = 3)
-
Inadequate written attending evaluations, 3.7 % (n = 1)
Other features that were also used to identify trainees performing below the expected
level of competence were: failure to follow institutional protocol/instructions repeatedly,
inadequate clinical judgement or negative input from the trainers and from colleagues,
and overall impression of competence.
Twenty-five TPDs/experts (of 28; 89.3 %) reported that they provided this feedback
directly to their trainees. The following interventions were employed to overcome
these issues:
-
Ensure increased procedure volume in 41.7 % (n = 10)
-
Set up procedure blocks with specific educational focus in 41.7 % (n = 10)
-
Provide didactic theoretical sessions in 16.7 % (n = 4)
-
No specific intervention was set up in 12.5 % (n = 3)
None chose to provide simulator training as a rescue solution for improving the trainee’s
performance. Other interventions were also mentioned to overcome this problem: transfer
to another unit more suitable to the trainee’s situation, suggest training program
interruption, re-explain the principle of training, intensified training of the items
below par, and ensure appropriate supervision and feedback.
In this regard, eight TPDs/experts (of 36; 22.2 %) have disqualified a trainee from
an ERCP/EUS training program, due to: not fulfilling minimum theoretical and technical
capabilities, could not understand the limits of their skills/high complication rate,
lack of honesty, inadequate progression, no adequate technical skills, insufficient
overall involvement and technical level despite corrective measures.
From a list of eight potential reasons to disqualify trainees in an ERCP/EUS training
program, all participants were asked to select the 3 most relevant ones in their opinion
([Fig. 6]).
Fig. 6 Most relevant reasons for trainee disqualification according to TPD/expert opinions
(blue columns) and trainee opinions (yellow columns).
Lack of enthusiasm/interest/motivation was also considered extremely important to
be added to the disqualification reasons (n = 2).
There is general agreement between TPDs/experts and trainees regarding the most relevant
reasons for trainee disqualification (Supplementary material, Appendix 2, Table 5),
with the exception of “lack of skills” (P = 0.006), which, in the opinion of trainees, was not considered to be an important
motive for disqualification, in opposite of TPDs/experts.
Discussion
To our knowledge, this is the first study to provide an insight of the criteria and
perceptions of TPDs/experts regarding the selection of ERCP/EUS trainees. This study
presents a wide range of data gathered from multiple training centers across Europe.
A high response rate (80 %) from the TPDs/experts was obtained, allowing analysis
of practices in a wide range of countries across Europe. However, despite receiving
up to three reminders, the response rate from the trainees was relatively low (38.5 %),
possibly due to a lack of incentive.
Despite a large percentage of TPDs/experts being very (72.2%) or extremely (13.9 %)
satisfied with trainees in their programs, almost half of them felt only moderately
(38.9 %) to slightly (8.3 %) satisfied with the current application process. In addition,
there was a significantly lower rate of extreme satisfaction (2.8 %) with the application
process. The rate of satisfaction with the application process is also shared by the
trainees (P = 0.08). These data raise concerns over the perceived quality of the current process
of selecting candidates for ERCP/EUS training.
As suggested in the literature [13], our results confirm that there is no standardized trainee application process in
these AGIE training programs. The selection process depends greatly on an applicant’s
subjective assessment (e. g., individual application, CV, interview), as in other
medical specialties [4]
[5]
[6]
[7]
[8]. We found noteworthy that “payment of an application fee” is part of the application
process in three centers. We do understand that a proper training involves extra efforts,
not only from Trainer’s point of view, but also from the department and Institution
involved in this process. But at the same time, it may be a constraint to some promising
trainees. A solution could be granting privileges to trainers, endoscopy departments
and hospitals, by institutions or societies, including improving facilities and providing
dedicated time and remuneration for those who truly can and are willing to teach.
Moreover, it was interesting to observe that there is no reported theoretical evaluation
at the start of training, a feature that contrasts with the lack of theoretical knowledge
appearing in trainees with difficulties or even in rare disqualifications. The low
importance of theoretical knowledge in the trainee’s selection process, suggests that
current ERCP/EUS training is based on an apprenticeship model. However, a major endeavor
should be pursued to certify this aspect is acquired as the lack of theoretical knowledge
in this complex area may rapidly appear as problematic.
When asked to rate a series of trainee attributes, there was a high diversity of selection
criteria, but TPDs/experts most highly valued characteristics related to personality
traits, such as “honesty,” “being a team player,” and “self-awareness.” Indeed, these
are considered non-technical skills (NTS) and examples of social and cognitive skills,
that influence quality and safety outcomes, respectively [14]. The importance of such NTS has been acknowledged previously in other areas such
as high-risk industries (e. g., aviation [15]) and healthcare practice (e. g., anesthesiology [16] and surgery [17]). Furthermore, being transferable by nature, NTS is also becoming increasingly recognized
in endoscopy (so-called ENTS) [18] and integrated into gastrointestinal endoscopy training [19] and as an evaluation item in endoscopy competence assessment tools, such as DOPS
[20]. In addition, TPDs/experts highly valued “honesty,” which is a fundamental characteristic
of integrity in the workplace and medical professionalism. Being upfront about one’s
actions and consequences is essential to good medical conduct and effective teamwork.
“Interest in research/academics” was rated as an extremely important feature only
by a minority. Our findings are in line with others [6] that also reported less emphasis on research. It is well recognized that research
is a major element in the development of any medical field [21] and is also needed to provide better care for our patients. However, in the present
context, where trainees are at the beginning stage in AGIE, attaining an acceptable
level of competence is prioritized and pushing the boundaries of knowledge in the
topic is of secondary importance.
“Leadership ability” was comparatively undervalued by TPDs/experts, despite being
considered an ENTS in assessment tools [20]. Of note, some of the characteristics that define good leadership can be found elsewhere
in the list, such as “being a team player,” “being able to make decisions” or “being
calm and having patient temperament,” with higher ratings.
TPDs/experts and trainees agreed regarding the relative importance of these characteristics
to a high degree in most characteristics. Of note, trainees most valued two somewhat
interrelated characteristics, “self-awareness” and “adequate management of complications.”
This may reflect their perception of the complexity and risks of AGIE procedures.
“Technical skills” came up seventh with regard to being considered “extremely important”
by TPDs/experts. Interestingly, even when asked to select only three extremely important
characteristics, technical skills were selected by less than half of the participants.
Clearly, TPDs/experts placed a higher priority on various cognitive or personality
traits, such as “clinical judgement,” “enthusiasm/interest,” and “openness to feedback,”
as hallmarks of a good prospective trainee. On the other hand, AGIE is a demanding
area of endoscopy and requires specific and significant training to reach a level
of competence and life-long training to reach and maintain expertise. In this matter,
technical skills are a crucial characteristic. Nonetheless, when we asked for a definition,
opinions were divided between several options. These options were taken from a previous
questionnaire developed by our group and sent to another list of TPDs/experts (unpublished
data). From the options provided, there were definitions related to purely motor skills
(“precision and hand-eye coordination”) and others integrative of both motor and cognitive
skills (“knowing how to fully use the scope and its degrees of freedom” or “how to
place the scope or device where you want” or “recognizing difficult or unusual situations
and adapting your technique”). Remarkably, a similar number of TPDs/experts chose
an integrative option (36.1 %) over a purely motor skill definition (33.3 %). This
agrees with the general trend towards selecting personality or cognitive-based options
as key attributes to succeed in ERCP/EUS.
“Clinical judgement” was mainly defined by both TPDs/experts and trainees as the “capacity
to evaluate the risks and benefits of the different management alternatives for a
given clinical situation, in a given patient.” This increased agreement between responders
can be attributed to a more discrete distribution of categories and less overlap between
them. Regardless, it is interesting to note that many of the trainee characteristics
highly-rated by both TPDs/experts and trainees needed to excel in ERCP/EUS can be
considered interrelated with the definition of sound clinical judgment. Specifically,
“honesty,” “self-awareness,” and “adequate management of complications” are integral
parts of personalized care for patients, in which the potential benefits and risks
of the procedures, and the recognition of our limitations, need to be weighted.
Although a large proportion of TPDs/experts identified underperforming trainees at
some point, only 22.2 % had to disqualify a trainee. Several reasons were mentioned
as justifying a trainee’s disqualification from an AGIE program, with “disregard patient
welfare” and “lack of work ethic” (dishonesty, misconduct, untrustworthiness) being
the most commonly stated. Of note, in general terms, there was agreement between the
most important characteristics for trainee selection and reasons for disqualification,
according to TPDs/experts. Also, there was general agreement between TPDs/experts
and trainees regarding reasons for disqualification. One notable exception was the
percentage of TPDs/experts and trainees selecting “lack of skills” as a reason for
disqualification (50 % vs 8 %, respectively; P = 0.006). It is possible that this reflects the trainee’s perception that technical
skills should be taught and lack of them cannot be considered a reason for disqualification
from a training program. It may also reflect a lack of self-awareness of the trainee
as well as a lack of clear communication from the trainer.
This study carries an inherent risk of selection bias due to the selective invitation
of certain centers. However, it should be noted that a high number of respondents
was obtained for TPDs/experts and a wide range of countries was included. Another
limitation is related to the questionnaire development process. For example, no validity
or reliability evaluations were performed. In addition, some overlap may exist between
some of the characteristics, which may make the interpretation of some of the results
more difficult. In particular, it is not easy to adequately rank domains or make inferences
between the relative importance of certain characteristics. It should be noted, however,
that other studies [4]
[8] have attempted to evaluate trainee characteristics in other specialties using similarly
formulated questions or using similar Likert rating scales. The scale was rated up
to 5 points, instead of 10 points used in the others studies [4], to reduce constraints in the data evaluation. In addition, the questions meant
to select the 3 most crucial trainee characteristics from all the ones they considered
to be extremely important to excel in ERCP/EUS or to be disqualified in training,
had the specific purpose of ensuring that the most important characteristics would
be captured in a context where potentially all could be very important. Another limitation
is the fact that ERCP and EUS were not considered separately. Indeed, there might
be variation between the personality of an EUS endoscopist and an ERCP endoscopist,
as, traditionally, EUS is more of a diagnostic tool and ERCP is a therapeutic procedure.
Nonetheless, ERCP is increasingly being linked to simultaneous EUS training as these
techniques are very complimentary when approaching patients with hepatobiliary diseases
[10]. In this sense, it makes sense to evaluate ERCP and EUS trainee characteristics
together. Finally, no definitive assumptions can be made about the true value of these
characteristics with regard to outcomes as no correlations between TPD/expert opinion
on ideal trainee characteristics and performance after training were evaluated. Therefore,
as is in other specialties [11], future studies may address this research issue specifically in AGIE.
Conclusions
In summary, these results highlight the importance of ENTS to excel in AGIE, and specifically
in ERCP/EUS. As highly regarded by TPDs/experts, the process of selection and evaluation
of an ERCP/EUS trainee should take into consideration, apart from knowledge and skills,
professional attitudes not directly related to the technical aspect itself. Ultimately,
it is the set of effective teamwork, self-awareness, openness to feedback and self-improvement,
enthusiasm and interest, clinical judgement, technical skills and work ethics that
will enable an advanced endoscopist to perform high-quality and safe endoscopy.
