Keywords
optics - flipped classroom - problem-based learning - ophthalmology - residency -
medical education
Ophthalmologists must demonstrate competence in clinical optics to attain professional
certification by the American Board of Ophthalmology. Unfortunately, the traditional
lecture-based method of instructing ophthalmology residents in optics has anecdotally
in our experience been perceived by learners and instructors to be both challenging
and of low educational yield. Alternative active learning models such as the flipped
classroom approach may help to address these concerns.[1]
The flipped classroom method involves learners performing independent study outside
of the classroom before attending in-class sessions. During in-class sessions, learners
participate actively by teaching the material covered during independent study to
their peers.[1] Key elements of the flipped classroom approach were first introduced as a “peer
instruction” method in 1991 by Eric Mazur at Harvard University.[2] Over the next few decades, variations in this curricular approach were popularized
and applied to medical education as the “flipped classroom” method, although there
did not exist clear evidence that this pedagogical approach would improve the competency
of healthcare providers. More recently, there has been building evidence that flipped
classroom curricula can be more effective than lecture format in some medical education
settings.[1]
[3] However, meta-analyses thus far comparing the efficacy of flipped classroom and
lecture-based medical curricula have not conclusively weighed in favor of the flipped
classroom methodology.[1]
[3]
[4]
[5] It has been difficult to ascertain the impact of flipped classroom curricula on
learners in part due to variability in both flipped classroom and research methodology
among past studies.[4]
With regard to clinical optics, there have been advocates for an active learning,
problem-focused approach to this subject matter as early as 1913.[6] However, the efficacy of an active educational format applied to clinical optics
has not been directly examined. We aimed to test whether a flipped classroom approach
to clinical optics education can be more effective than traditional lecture-based
didactics in facilitating learner acquisition of clinical optics knowledge and improving
their ability to apply this information. To our knowledge, this is the first investigation
into the implementation of a flipped classroom format to clinical optics in a graduate
medical education setting.
Methods
Ophthalmology resident clinical optics instruction at Oregon Health & Science University
was delivered via traditional lecture-based didactics until a flipped classroom curriculum
was implemented during the 2012 to 2013 academic year and continued thereafter. [Table 1] summarizes the major elements of the two curricula.
Table 1
Comparison of educational curricula
|
Curriculum type
|
Lecture-based curriculum
|
Flipped classroom curriculum
|
|
Focus
|
Instructor-centered
|
Learner-centered
|
|
Learner activities prior to in-class sessions
|
Learners provided with reference books and recommended reading
|
Learners provided with reference books, recommended reading, and online video lecture
series. Learners complete assigned problem sets independently
|
|
Learner and instructor activities during in-class sessions
|
Lectures delivered by faculty instructors via projected slide presentations. Learners
listen to lectures and are allowed to ask questions
|
Learners take turns teaching each other in small groups. Faculty provide oversight
and guidance as needed
|
|
Faculty instructor time commitment and preparation
|
Instructors prepare lecture slide presentations
|
Instructors review predefined problem set and answer key
|
|
Academic years curriculum applied
|
AY 2008–09 to AY 2011–12
|
AY 2012–13 to AY 2015–16
|
|
Timing of in-class sessions
|
Fall (September–October)
|
Winter (January–February)
|
|
Mean number of in-class sessions per year
|
4.5
|
4.25
|
|
Mean number hours dedicated to in-class sessions per year
|
18.0
|
11.75
|
Abbreviation: AY, academic year.
Our lecture-based curriculum consisted of projected slide presentations developed
and delivered by faculty. Residents were expected but not required to prepare for
lecture by reading the appropriate sections from the Basic and Clinical Science Course
(BCSC) reference books. Didactics were delivered in the fall from September to October.
Our flipped classroom approach to optics was structured as follows: 4 sequential problem
sets were constructed, in total consisting of 100 optics questions, 42 of which are
from ophthalmic optics and clinical refraction by Guyton et al.[7] Learners were asked to independently complete the appropriate set of optics questions
prior to each of four mandatory in-class group sessions. Learners were provided with
the following reference materials to utilize during their initial independent study
period—BCSC Section 3: Clinical Optics, Ophthalmic Optics and Clinical Refraction
by David Guyton, Review questions in ophthalmology, and the prerecorded optics video
lecture series by David Hunter.[7]
[8]
[9]
[10] We instructed residents to prepare to teach the material covered in the problem
sets to their peers. At each in-class session, residents took turns working through
the assigned problems and teaching their peers within small groups of four to five
learners organized by year of training. Faculty instructors supervised the discussions,
answered questions, and corrected mistakes, but did not lead the teaching sessions.
In class, flipped classroom sessions were delivered in late winter, from January to
February.
We compared resident performance on the Ophthalmic Knowledge Assessment Program (OKAP)
annual national in-service examination for the 4 years before and after the optics
curriculum change. No other major curricular changes were implemented during this
period of time. All OKAP examination scores from 2009 to 2016 were included with no
exclusions. Data from each year included scores from 15 residents except 2009 (12
residents) and 2014 (14 residents). The scaled scores from the Optics, Refraction, and Contact Lens subsection were examined with scores from the 10 nonoptics subsections serving as
controls. Descriptive statistical analysis was performed using Microsoft Excel (Redmond,
WA). Unpaired, two-tail t-tests were performed on the scores from the 11 subsections of the OKAP examination,
comparing scores before (2009–2012) and after (2013–2016) the optics curriculum change.
Bonferroni correction was applied to adjust the p-value for statistical significance given the 11 subsection comparisons. This analysis
was also repeated for each postgraduate year (PGY) class of residents.
Results
The Optics, Refraction, and Contact Lens mean subsection scores were found to be 50.4 ± 2.3 and 57.3 ± 2.5 (mean ± 95% confidence
interval [CI]) before and after the optics curriculum change, respectively ([Fig. 1]). Optics, Refraction, and Contact Lens was the only subsection to show a statistically significant difference when comparing
scores before and after the optics curriculum change (p = 0.00008, [Table 2]). Descriptive statistical analysis of all 11 OKAP examination subsection scores
before and after the optics curriculum change is shown in [Table 2]. Bonferroni correction of the α to account for the multiple comparisons yielded
a p value of 0.0046 necessary for statistical significance. Subgroup analysis of each
PGY of training showed that PGY-2 and PGY-3 optics scores were higher with flipped
classroom (55.9 ± 4.4 and 59.1 ± 4.1, respectively, mean ± 95% CI) compared with lecture-based
curricula (46.7 ± 4.4 and 51.1 ± 3.7, respectively, mean ± 95% CI). This difference
was statistically significant with p values of 0.0037 for PGY-2 and 0.0043 for PGY-3 residents. There was no statistically
significant difference in optics subsection scores for PGY-4 residents when comparing
flipped classroom and lecture-based curricula (53.3 ± 5.1 and 56.8 ± 4.1, respectively,
mean ± 95% CI; p = 0.25).
Table 2
OKAP subsection performance
|
OKAP subsection
|
Before or after optics curriculum change
|
Mean score
|
Median score
|
SD
|
95% CI
|
Max
|
Min
|
Range
|
n
|
p-Value
|
|
1. General medicine
|
Before
|
49.51
|
49
|
9.1
|
2.4
|
67
|
29
|
38
|
57
|
0.0052
|
|
After
|
54.41
|
57
|
9.4
|
2.5
|
66
|
16
|
50
|
59
|
|
2. Fundamentals and principles of ophthalmology
|
Before
|
52.42
|
53
|
9.1
|
2.4
|
69
|
32
|
37
|
57
|
0.59
|
|
After
|
53.29
|
53
|
8.3
|
2.2
|
68
|
30
|
38
|
59
|
|
3. Optics, refraction, and contact lens
|
Before
|
50.37
|
51
|
8.7
|
2.3
|
67
|
32
|
35
|
57
|
0.000083
|
|
After
|
57.27
|
60
|
9.5
|
2.5
|
74
|
34
|
40
|
59
|
|
4. Ophthalmic pathology and intraocular tumors
|
Before
|
53.72
|
54
|
7.0
|
1.8
|
68
|
39
|
29
|
57
|
0.10
|
|
After
|
55.92
|
57
|
7.4
|
1.9
|
67
|
23
|
44
|
59
|
|
5. Neuro-ophthalmology
|
Before
|
53.30
|
53
|
8.5
|
2.2
|
72
|
28
|
44
|
57
|
0.074
|
|
After
|
56.19
|
57
|
8.8
|
2.3
|
71
|
27
|
44
|
59
|
|
6. Pediatric ophthalmology and strabismus
|
Before
|
52.33
|
54
|
8.5
|
2.3
|
68
|
28
|
40
|
57
|
0.18
|
|
After
|
54.51
|
56
|
8.8
|
2.3
|
67
|
24
|
43
|
59
|
|
7. Orbit, eyelids, and lacrimal system
|
Before
|
54.16
|
54
|
6.6
|
1.8
|
67
|
35
|
32
|
57
|
0.88
|
|
After
|
53.93
|
56
|
9.5
|
2.5
|
67
|
7
|
60
|
59
|
|
8. Cornea, lens, and external disease
|
Before
|
51.74
|
51
|
8.3
|
2.2
|
67
|
25
|
42
|
57
|
0.25
|
|
After
|
53.47
|
54
|
8.0
|
2.1
|
70
|
30
|
40
|
59
|
|
9. Intraocular inflammation and uveitis
|
Before
|
52.02
|
52
|
7.0
|
1.9
|
64
|
23
|
41
|
57
|
0.034
|
|
After
|
55.22
|
56
|
8.9
|
2.3
|
70
|
25
|
45
|
59
|
|
10. Glaucoma
|
Before
|
52.35
|
52
|
9.2
|
2.4
|
68
|
31
|
37
|
57
|
0.040
|
|
After
|
55.97
|
59
|
9.6
|
2.5
|
70
|
20
|
50
|
59
|
|
11. Retina and vitreous
|
Before
|
53.93
|
56
|
7.7
|
2.0
|
68
|
33
|
35
|
57
|
0.71
|
|
After
|
53.34
|
55
|
9.3
|
2.4
|
67
|
18
|
49
|
59
|
Abbreviation: CI, confidence interval; OKAP, Ophthalmic Knowledge Assessment Program;
SD, standard deviation.
Fig. 1 Ophthalmic Knowledge Assessment Program (OKAP) examination scores before and after
implementation of a flipped classroom optics curriculum. Resident OKAP examination
performance for the 4 years before and after changing the optics curriculum from a
lecture-based to flipped classroom curriculum (mean ± 95% CI). See [Table 2] for full-length OKAP subsection titles. Asterisk symbol denotes statistically significant
difference at p = 0.00008. CI, confidence interval.
Discussion
To date, there has been minimal research into the application of alternative educational
methodologies such as the flipped classroom format to ophthalmology education.[11]
[12]
[13] Our data show that transitioning from traditional lectures to a flipped classroom
curriculum for clinical optics was associated with a statistically significant increase
in mean OKAP examination score for the Optics, Refraction, and Contact Lens subsection. This suggests that the flipped classroom format may be more effective
than lectures for facilitating ophthalmology resident clinical optics knowledge and
ability to apply this knowledge in a standardized test setting. Standardized test
scores are relevant, since a written qualifying examination (a standardized test similar
in nature to the OKAP examination) is required for American Board of Ophthalmology
certification, and the board pass rate for graduates is in turn a key metric monitored
by the Accreditation Council for Graduate Medical Education (ACGME) ophthalmology
residency review committee for the purpose of residency program accreditation.[14]
[15]
The flipped classroom format as a form of active learning has been shown to positively
influence learner engagement, motivation, and satisfaction.[4]
[16]
[17]
[18]
[19] Our flipped classroom optics curriculum places learners in a more active role, requiring
them to both complete problem sets independently and teach their colleagues. Of note,
our learners were also given a choice of reference materials to utilize during the
pre-class self-study interval. Trainee learning preferences may differ, with some
favoring prerecorded video lectures and others preferring reference books. While the
average in class time dedicated to our flipped classroom curriculum was less than
that of our lecture-based curriculum, flipped classroom learners were required to
spend additional time outside the classroom to complete problem sets. We did not quantify
the number of hours spent by residents outside of the classroom, but based on our
own experience completing the problem sets, we estimate the total time spent by learners
in and out of the classroom to be roughly similar between the two curricular approaches.
Feedback from learners suggests that the average time spent completing the self-study
portion of the flipped classroom curriculum decreases with each year of experience.
Interestingly, PGY-2 and PGY-3 but not PGY-4 residents exposed to our flipped classroom
curriculum performed significantly better on the OKAP examination than respective
PGY counterparts exposed to a lecture-based optics curriculum. This data suggests
that the flipped classroom format benefited inexperienced optics learners more so
than those with 2 years of pre-existing optics examination experience. Of course,
subgroup analysis is more prone to type 1 statistical error given the smaller sample
size.
Anecdotally, we found our problem-based optics curriculum to be less burdensome and
more enjoyable for instructors. Faculty were no longer required to spend time generating
slide presentations for the optics lecture series. Providing faculty with problem
set answer keys allowed them to efficiently prepare for in-class sessions. Additionally,
faculty time commitment required for in-class educational sessions was less with the
flipped classroom curriculum than with traditional lectures. While a similar group
of optometry and ophthalmology teaching faculty was utilized in this study both before
and after the curriculum change, we did find that the flipped classroom format with
pre-constructed questions and answer key was more conducive to the recruitment of
faculty instructors. The flipped classroom format likely lowers the barrier to participation,
especially for faculty without a clinical practice that focuses on optics. This may
be helpful for residency training programs that struggle to recruit faculty to teach
optics.
This study is subject to several limitations including those associated with a retrospective,
nonrandomized study design based on data from a single institution. A limitation of
our study involves the sampling error that can occur due to the relatively few questions
on any one OKAP examination subsection. As noted in the OKAP user guide: “because
each yearly exam is comprised of a different set of test items, not every clinical
point can be covered in every year. […] Because subtests necessarily must contain
fewer items than the test as a whole, subtest scores are somewhat less reliable and
must be interpreted with more caution.”[20]
Another limitation of our study is that we are unable to separate the contributions
of the different components of our flipped classroom curriculum. The required self-study
problem set, learners as teachers small group discussion format, and the timing of
curriculum prior to testing were all factors that may have contributed to the performance
improvement observed.
While we do not have data to directly test whether the required problem set alone
would be sufficient to improve optics scores, there is extensive literature suggesting
that collaborative learning yields superior educational outcomes relative to individualistic
learning.[19] Additionally, our data suggests that simply providing or improving access to problem
sets is not sufficient to optimize examination performance. OKAP-style practice questions
were made available to all residents throughout the study period, including those
found in review questions in ophthalmology and multiple editions of the BCSC series.[9]
[10] Also of note, ophthoquestions.com, an online resource that could potentially increase
learner access to OKAP-style practice questions, was introduced in September 2011.
Our program has never paid for this online question bank for our trainees and we do
not track those that utilize it; however, ophthoquestions.com company representatives
report that 95% of ophthalmology trainees utilize their services for OKAP preparation.[21] No statistically significant difference was identified when comparing optics subsection
scores before and after the introduction of ophthoquestions.com (and before the introduction
of the flipped classroom curriculum). Similarly, no statistically significant difference
in nonoptics subsection test scores was identified when comparing scores prior to
and after the introduction of ophthoquestions.com. This suggests that improved online
access to optics questions as well as nonoptics questions does not significantly improve
OKAP scores.
Our dataset is not sufficient to fully evaluate whether the temporal shift of optics
curriculum to more closely precede the OKAP test (delivered in March) was influential,
but suggests that the effect, if present, was small. There was no statistically significant
difference between mean subsection exam scores for curricula presented in the fall
versus late winter (51.7 ± 2.0 and 52.8 ± 2.3, respectively, mean ± 95% CI; p = 0.23).
While our research shows that the flipped classroom format can be applied in an advantageous
manner to improve ophthalmology resident optics standardized test scores, further
study will be needed to determine whether this curricular shift translates to an improved
ability to apply optics knowledge to real-world clinical practice. And finally, while
these results may be applicable to the optics portion of the ophthalmology residency
curriculum, it is unclear whether flipped classroom would also be more effective than
traditional lectures for other ophthalmologic areas of study.
Despite these limitations, given the relative advantages of flipped classroom optics
learning over traditional lecture-based curricula, a flipped classroom approach is
a reasonable option for ophthalmology training programs seeking to improve resident
performance in clinical optics.
