Keywords
microsurgical suturing - microsurgical training - surgical skills assessment - resident
training - Kern's model - distributed practice
In ophthalmology, microsurgical suturing is essential for procedures such as open
globe repair, corneal transplant suturing, and strabismus surgery.[1] Yet restrictions on resident training hours and advances in sutureless ophthalmic
procedures may limit opportunities for residents to practice microsurgical suturing.[2]
[3]
[4]
[5] While the Accreditation Council for Graduate Medical Education requires residency
training programs to have a microsurgical skills development resource (wet laboratory
or simulators), research shows that residents may find value in (additional) microsurgical
skills training courses.[2]
[6]
The purpose of our study was to design, implement, and evaluate a faculty-guided,
self-directed, distributed (e.g., practice interspersed with periods of rest) microsurgical
skills pilot course for ophthalmology residents based on Kern's six-step approach
to curriculum development. We hoped to enhance microsurgical suturing skills acquisition
and retention among ophthalmology residents. Kern's model for curriculum development
was selected as its purpose is to “provide a practical, theoretically sound approach
to developing, implementing, evaluating and continually improving educational experiences
in medicine.”[7] Kern's model was initially sculpted by educators who have developed over 100 medical
curricula for topics ranging from clinical reasoning to surgical skills assessment.[7] The model utilizes a six-step approach ([Table 1]) that emphasizes flexibility based on evaluation results, changes in trainee population,
and changes in material (or skills) requiring mastery.[8] No study to date has evaluated the utility of Kern's six-step approach to acquisition
and retention of microsurgical suturing skills.
Table 1
Overview of Kern's six-step approach to curriculum development and how the steps apply
to our training program
|
Step
|
Definition
|
Application of Kern's approach to our pilot curriculum
|
|
1. Problem identification
|
• Identify and characterize the healthcare problem that will be addressed by the
curriculum, how the problem is currently being addressed and how it should ideally
be addressed
|
• Healthcare problem: Based on statements from published literature and direct input from ophthalmology
attendings at Yale University, we have determined that there is a need for enhanced
microsurgical suturing training for ophthalmology residents
• How the problem is being addressed: Annual wet laboratory-based training for ophthalmology residents in addition to
direct surgical experience through the apprenticeship model
• How the problem should ideally be addressed: In addition to our current program, we should implement a microsurgical suturing
course that provides faculty guidance to residents, while allowing for individual
practice time, and evaluation and feedback
|
|
2. Targeted needs assessment
|
• Assess the needs of one's targeted group of learners and their medical institution/learning
environment
|
• Needs of learners and learning environment: Ophthalmology faculty members and the residency program director were interviewed
concerning opportunities for residents to practice microsurgical suturing skills.
Published research has also noted a reduced chance for ophthalmology residents to
practice microsurgical suturing skills in the current climate of reduced training
hours and advances in sutureless procedures
|
|
3. Goals and objectives
|
• Target the curriculum to address the needs of learners
• An end toward which an effort is directed
|
• Primary goals and objectives:
To provide residents with an opportunity to assess their baseline confidence in microsurgical
suturing, and knowledge of sutures and surgical instruments.
To evaluate the effectiveness of a faculty-guided microsurgical suturing training
course for the enhancement of microsurgical suturing skills, using results from pre-
and post-test practical skills assessments
To evaluate the effectiveness of a distributed practical model within a microsurgical
suturing training course
|
|
4. Educational strategies
|
• Chose curriculum content and educational methods that will most likely achieve
the educational objectives
|
• Curriculum content: Our curriculum focuses on components that are essential to enhancing microsurgical
suturing skills. Specifically, we focus on knowledge of sutures and suture material
and practical application of this knowledge to basic microsurgical suturing technique
• Educational methods: Based on prior literature showing the benefits of distributed practice, we decided
to implement a faculty-led teaching course followed by a 30-day period of individual
study and practice. We concluded by providing an additional practice and evaluation
session at the end of the 30-day period, followed by individual and summative feedback
|
|
5. Implementation
|
• Implement the educational intervention and its evaluation
• Components: Obtain political support, identify and procure resources, identify
and address barriers to implementation, introduce the curriculum, administer the curriculum
and refine the curriculum over successive cycles
|
• Obtain political support: Not applicable to this study
• Identify and procure resources: Resources for the curriculum (latex gloves, plastic platforms, suture material and
needles, surgical instruments) were made available through the Yale Department of
Ophthalmology and Visual Science
• Identify and address barrier to implementation: There were no specific barriers to implementation
• Introduce the curriculum: Residents were notified that a pilot curriculum was to be conducted within their
current training course
• Administer the curriculum: The course was administered over a 1-month period
• Refine the curriculum over successive cycles: We plan to revise the pilot curriculum based on evaluation results and eventually
formally integrate the course within the resident training program. We also plan to
create an online module that can be applied to any residency program requiring trainees
to master basic microsurgical suturing knowledge and skills
|
|
6. Evaluation and feedback
|
• Assess the performance of individuals and the curriculum. The purpose of the evaluation
may be formative (provide ongoing feedback) or summative (provide a final grade or
evaluation)
|
• Evaluation of individuals: Each resident was evaluated independently based on their scores on both the written
knowledge assessment and the practical skills assessment
• Evaluation of the curriculum: The curriculum as a whole was evaluated by de-identifying and pooling data from
all resident classes and assessing results of survey data, written knowledge assessments
and practical skills assessments both before and after the administration of the course
|
We emphasized principles of faculty-guided, self-directed practice, and distributed
practice within the framework of Kern's six-step approach. Through integrating these
principles into a curriculum model, we propose a novel method for implementing and
evaluating a microsurgical suturing skills program that is generalizable to all ophthalmology
residents, throughout all stages of training.
Methods
Curriculum Design and Test Population
The six steps of Kern's model are as follows: (1) Problem identification and general
needs assessment; (2) targeted needs assessment; (3) goals and objectives; (4) educational
strategies; (5) implementation; and (6) feedback and evaluation.[8] For specific definitions of each step and how we apply each step to the design of
our curriculum, see [Table 1].
A total of 15 ophthalmology residents (5 postgraduate year 2 [PGY-2], 5 PGY-3, and
5 PGY-4) participated in the pilot curriculum. All residents took part in the survey,
written knowledge assessment, and practical skills assessment. The pilot curriculum
was run from March through August of 2017 at the Yale University Department of Ophthalmology
and Visual Science. Of note, “pre-test” refers to any time period before the teaching
session and 30-day practice period, whereas “post-test” refers to the time period
after the teaching session and 30-day practice period.
Pre-Test Survey and Pre-Test Written Knowledge Assessment
Prior to the pre-test practical skills assessment and faculty-guided teaching session,
residents were given the following: a pre-test Likert-scale survey (1 through 5 with
5 listed as “strongly agree” and 1 listed as “strongly disagree”) and a pre-test written
knowledge assessment. The survey was designed as a self-assessment for residents to
evaluate their practical abilities and knowledge of various components of microsurgical
suturing before the teaching session and practice period. Residents' opinions on the
utility of practicing suturing were also assessed.
The pre-test written knowledge assessment was designed to evaluate residents' knowledge
of needles, suture material, and their clinical applications. Examples of both the
pre-test survey and pre-written knowledge assessment are provided as [supplemental materials].
Pre-Test Practical Skills Assessment and Faculty-Guided Teaching Session
Two ophthalmology attendings (CCT and JHC), including the residency program director
(JHC), scheduled a 1-hour faculty-guided teaching session for each class of ophthalmology
residents. Before the teaching session, residents' microsurgical suturing skills were
evaluated through a pre-test practical skills assessment. Components of the practical
skills assessment included the following: (a) loading the needle; (b) 1–1-1–1 square
knot; (c) 3–1-1 forehand; d. 3–1-1 backhand; (e)1–1-1 slip knot; (f) closure running
suture; (g) closure running locking suture. Residents used taut latex, a low-fidelity
model (e.g., does not mimic real tissue), to practice their sutures as it is a cost-effective
method and research has shown that “surgical skills training on low-fidelity…models
appears to be as effective as high-fidelity model training for the acquisition of
technical skill among novice surgeons.”[9]
During the pre-test practical skills assessment, residents were evaluated independently
by the ophthalmology attendings based on the Global Rating Scale of Operative Performance
([Table 2]).[10] Of note, economy of movement, confidence of movement, respect for tissue, and overall
performance were graded as separate categories. Knowledge and precision of technique
were based on each of the listed components of the practical skills assessment (e.g.,
skill in loading the needle, completing a 1–1-1–1 square knot). After the pre-test
practical assessment, residents participated in the faculty-guided teaching session
that consisted of a practice period with individual guidance from each ophthalmology
faculty member. Residents were then given a 30-day period of self-directed practice
where they were encouraged to utilize faculty instructions from the practice session
and online resources from the University of Iowa EyeRounds Web site: (http://webeye.ophth.uiowa.edu/eyeforum/tutorials/Iowa-OWL/suture/suturing-and-tying-techniques.htm).
Table 2
Template for faculty scoring of practical microsurgical suturing skills using the
global rating scale of operative performance
|
Score
|
1
|
2
|
3
|
4
|
5
|
|
Economy of movement (time and motion)
|
Many unnecessary moves
|
|
Efficient time/motion but some unnecessary moves
|
|
Economy of movement and maximum efficiency
|
|
Confidence of movement (instrument handling) and use of nondominant hand
|
Repeatedly makes tentative or awkward moves with instruments
|
|
Competent use of instruments although occasionally stiff or awkward
|
|
Fluid moves with instruments with no awkwardness
|
|
Respect for tissue
|
Frequently used unnecessary force on tissues or caused damage by inappropriate use
of instruments
|
|
Careful handling of tissue but occasionally caused inadvertent damage
|
|
Consistently handled tissues appropriately with minimal damage
|
|
Knowledge and precision of technique
|
Imprecise, wrong technique. Required specific instruction at most steps
|
|
Careful technique with occasional errors. Knew all important steps
|
|
Fluent, secure and correct technique in all stages of suturing. Familiar with all
steps
|
|
Overall performance
|
Unable to perform independently
|
|
Competent. Could perform with minimal assistance
|
|
Superior, able to perform independently with confidence
|
Post-Test Survey and Post-Test Written Knowledge Assessment
After the self-directed practice period, residents returned to the ophthalmology department
for a post-test survey and post-test written knowledge assessment. The post-test survey
gave residents the opportunity to re-evaluate their practical skills and knowledge
of microsurgical suturing. The post-test written knowledge assessment gave residents
the opportunity to demonstrate their retention of specific details of microsurgical
suturing instruments and suture materials learned from the faculty-guided teaching
session and 30-day practice period.
Post-Test Practical Skills Assessment and Feedback
Following the post-test survey and post-test written knowledge assessment, residents
were again evaluated on each component of the practical skills assessment. The two
ophthalmology attendings who initially evaluated their pre-test microsurgical skills
graded residents' performance using the same Global Rating Scale of Operative Performance.
The attendings then gave the residents another practice period with individual feedback
on how the residents performed relative to their initial practical skills assessment
and on how they can continue to improve. All feedback given was formative, or designed
to be ongoing throughout the remainder of the residents' training period. A final
competency assessment was therefore not included in the evaluation.
The study was determined to be exempt by the Yale University Institutional Review
Board.
Statistical Analysis
All data were collected by one investigator (ACR) and entered into Excel Version 15.21.1.
Scoring of written knowledge assessments was performed by one investigator (ACR) using
an answer key provided by the residency program director. Practical skills assessments
were graded independently by each faculty member. Descriptive statistics and paired
t-tests were used as appropriate. Cohen's kappa statistic was calculated to evaluate
inter-rater reliability between the two attendings on the practical skills assessments.
Results
Resident Self-Assessment Survey
Results are reported as mean ± standard deviation with results from pre-test surveys
reported first followed by results from post-test surveys. PGY-2 residents felt significantly
more confident on the following items after the course: “I know the names of the instruments”
(2 ± 0.0 vs. 3.4 ± 0.5; p < 0.01); “I know the proper way to hold instruments” (2.8 ± 0.4 vs. 4 ± 0.0; p < 0.01); and “I know the proper way to handle instruments” (2.6 ± 0.5 vs. 3.8 ± 0.4;
p < 0.01). PGY-3 residents did not feel significantly more confident on any of the
survey items. After the course, they felt significantly less confident on the item,
“I know how to suture” (3.8 ± 0.8 vs. 3.0 ± 1.0; p < 0.05). PGY-4 residents felt significantly more confident on the item, “I know the
different types of sutures” after the course (3.4 ± 0.5 vs. 4.2 ± 0.4; p < 0.05).
When all years of training were combined, residents felt more confident in the following
areas after the course: “I know the names of the instruments” (2.5 ± 0.7 vs. 3.2 ± 0.8;
p < 0.01); “I know the proper way to hold instruments” (2.9 ± 0.5 vs. 3.7 ± 0.8; p < 0.01); and “I know the different types of needles” (3.0 ± 0.6 vs. 3.6 ± 0.5; p < 0.05). Residents overall felt significantly less confident on the following items
after the course: “I feel like a suturing teaching session will help me better understand
how to suture,” “I feel like I have a better understanding of how to suture” (4.8 ± 0.4
vs. 4.2 ± 0.4; p < 0.001) and “I feel that practicing suturing will make me better” (4.9 ± 0.3 vs.
4.5 ± 0.5; p < 0.05) ([Fig. 1)].
Fig. 1 Survey results from before and after the microsurgical suturing course and 30-day
practice period for postgraduate year 2 (PGY-2) (A), PGY-3 (B), PGY-4 (C), and all resident years combined (D). Note: *(p < 0.05); **(p < 0.01); ***(p < 0.001). Asterisks correspond to levels of statistical significance based on paired
t-test results, as seen on the figure.
Resident Written Knowledge Assessment
Results are reported as median percent correct out of 12 items, with corresponding
first and third interquartile range. Median percent correct for the pre-test assessment
is listed first, followed by the median percent correct for the post-test assessment.
All resident classes scored significantly higher on their written knowledge assessment
after the teaching session and practice period. PGY-2 (42.3% [34.6, 42.3] vs. 65.4%
[65.4, 69.2]; p <0.05); PGY-3 (50.0% [46.1, 53.9] vs. 65.4% [61.5, 73.1]; p < 0.05); and PGY-4 (57.7% [50.0, 65.4] vs. 80.1% [76.9, 84.6]; p <0.01). When PGY-3 and PGY-4 years were combined, the median knowledge assessment
test score was 51.92% (47.1, 63.5) versus 75.0% (62.5,79.8); p < 0.001 ([Fig. 2)].
Fig. 2 Results of the pre- and post-intervention written knowledge assessments for individual
resident years (A) and postgraduate year 2 (PGY-3) and PGY-4 combined, with PGY-2 for comparison (B). Note: *(p < 0.05); **(p < 0.01); ***(p < 0.001). Asterisks correspond to levels of statistical significance based on paired
t-test results, as seen on the figure.
Faculty-Evaluated Practical Skills Assessment
PGY-2 residents improved in all areas of the practical skills assessment: economy
of movement (1.6 ± 0.6 vs. 2.75 ± 0.5; p < 0.001); confidence of movement (1.6 ± 0.5 vs. 2.7 ± 0.5; p < 0.001); respect for tissue (2.4 ± 0.2 vs. 3.0 ± 0.0; p < 0.01); overall performance (1.7 ± 0.5 vs. 3.0 ± 0.3; p < 0.01); and knowledge and precision of technique (1.6 ± 0.2 vs. 2.9 ± 0.2; p < 0.001). See [Table 2] for a description of the grading scale
PGY-3 residents also improved in all areas of the practical skills assessment: economy
of movement (2.8 ± 0.6 vs. 3.5 ± 0.4; p < 0.05); confidence of movement (2.5 ± 0.4 vs. 3.6 ± 0.6; p < 0.01); respect for tissue (2.7 ± 0.3 vs. 3.6 ± 0.4; p < 0.001); overall performance (2.8 ± 0.5 vs. 3.6 ± 0.6; p < 0.05); and knowledge and precision of technique (2.6 ± 0.2 vs. 3.7 ± 0.6; p < 0.01).
PGY-4 residents improved in the following areas of the practical skills assessment:
economy of movement (3.3 ± 0.4 vs. 4.0 ± 0.4; p < 0.001); confidence of movement (3.5 ± 0.4 vs. 4.3 ± 0.4; p < 0.01); overall performance (3.5 ± 0.3 vs. 4.2 ± 0.4; p < 0.001); and knowledge and precision of technique (3.4 ± 0.3 vs. 4.3 ± 0.3; p < 0.01).
When combined, all residents improved in their practical skills assessment after the
course: economy of movement (2.6 ± 0.9 vs. 3.4 ± 0.7; p < 0.001); confidence of movement (2.5 ± 0.9 vs. 3.6 ± 0.8; p < 0.001); respect for tissue (3.0 ± 0.7 vs. 3.6 ± 0.6; p < 0.001); overall performance (2.6 ± 0.8 vs. 3.6 ± 0.6; p < 0.001); and knowledge and precision of technique (2.5 ± 0.8 vs. 3.6 ± 0.7; p < 0.001) ([Fig. 3)].
Fig. 3 Results of pre- and post-intervention practical skills assessments using the Global
Rating Scale of Operative Performance for postgraduate year 2 (PGY-2) (A), PGY-3 (B), PGY-4 (C), and all resident years combined (D). Note: *(p < 0.05); **(p < 0.01); ***(p < 0.001). Asterisks correspond to levels of statistical significance based on paired
t-test results, as seen on the figure.
Cohen's kappa ranged from 0.06 (–0.2–0.3) to 0.6 (0.2–0.9) for all components of the
practical skills assessment.
Discussion
We applied Kern's six-step approach to curriculum development to a pilot course for
basic microsurgical suturing. We integrated distributed practice and self-directed
practice, which have been shown to improve acquisition and retention of surgical skills.[11]
[12]
[13] After the course, residents improved in their written knowledge and practical skills
assessments. PGY-2 residents felt more confident in their ability to recognize and
use surgical instruments and PGY-4 residents felt more confident in their ability
to identify different types of sutures. PGY-3 residents felt less confident in their
knowledge of microsurgical suturing. All resident classes felt less confident that
practicing suturing will make them better. Our pilot course was useful in improving
practical skills and trainee knowledge base. However, resident lack of confidence
and/or initial cognitive bias may be important concerns to address.
A study of general surgery residents showed that “junior residents in the middle of
clinical training years were most worried about feeling confident enough to perform
procedures independently by the end of training.”[14] These feelings of worry may reflect increased levels of operative responsibility,
while also possibly having to make decisions regarding (fellowship) training.[14] This may reflect similar attitudes of PGY-3 ophthalmology residents and the transition
from PGY-3 to PGY-4 may be a critical stepping point for residents to gain confidence
in both their operative skills.
A reduction in trainee confidence that practicing suturing would help improve their
skills may be explained by the Dunning–Kruger effect, which is based on the theory
that “[novice] individuals do not possess the degree of metacognitive skills necessary
for accurate self-assessment.”[15] Prior to the course, residents may have overinflated their ability to improve their
microsurgical suturing skills through practice alone. After the course, they may have
realized that they also needed to improve on their technical knowledge and may be
in need of guidance from faculty. Future research on resident attitudes toward practicing
microsurgical suturing on their own versus with a faculty mentor may be useful.
PGY-2 residents seemed to gain the most information in their knowledge of how to handle
and identify surgical instruments. This finding suggests that knowledge of handling
instruments in the operating room may be a key area to focus on in subsequent courses.
PGY-4 residents seemed to benefit most from learning to differentiate different types
of sutures, indicating that they may already have a solid grasp on basic suturing
techniques and instrument handling and may require more detailed instruction on use
of specific suture types. Therefore, the course could be improved through tailored
instruction to each resident class. Of note, PGY-3 and PGY-4 residents were combined
when assessing pre- and post-curriculum scores on the written knowledge assessment
for the purposes of evaluating the overall impact of the course on fundamental microsurgical
suturing knowledge among trainees who had at least 1-year of ophthalmology training.
Results supported our findings that the course is beneficial even among more senior
resident groups.
Limitations include use of self-reported survey results, small sample size of trainees
in the pilot program, small number of faculty evaluators, and lack of control for
microsurgical suturing training prior to ophthalmology residency. Cohen's kappa statistic
(“the extent to which the data collected in the study are correct representations
of variables measured”[16]) ranged from 0.06 to 0.6. Any kappa below 0.6 may indicate inadequate agreement
among raters.[16] However, given that the highest kappa statistic is 0.6, we cannot definitively state
that results are not consistent between the two raters. The wide range of kappa scores
may be explained by variation in expectation of resident performance based on PGY
level, differences in faculty training techniques, and/or discrepancies in resident
performance during the practical skills assessment. Additional faculty evaluators
may help mitigate differences in scoring of resident practical skills assessments
in future studies.
Lastly, once established as a formal curriculum within our ophthalmology residency
program, we hope to transition this curriculum to an online module, which would allow
for greater ease of dissemination to other programs that may like to establish their
own resident microsurgical suturing skills course.
Conclusion
Implementation of our pilot course enhances trainee practical and knowledge-based
skill in basic microsurgical suturing. Integration of our model within residency curricula
may help improve acquisition and retention of microsurgical suturing skills. Prior
to the course it may be useful to conduct a needs assessment for each resident class
to address the most pressing concerns during the faculty teaching session.
