Over the past 50 years, the medical field has seen a marked decline in the number
of physicians pursuing careers in clinical research.[1] James Wyngaarden, MD, former director of the National Institutes of Health (NIH),
first noted this decrease in the late 1970s. He called clinical investigators an endangered
species in an atmosphere focused on basic science over patient-oriented research.[2] Years later, Weinreb revisited this issue, underscoring the increasing difficulties
that clinician-scientists face in getting funding while maintaining clinical productivity.[3] In recent decades, attempts to address this shortage have included formalizing research
training during clinical years to ensure that it remains a career-long priority.[4]
Several expert committees—including the NIH Director's Panel on Clinical Research,[5] the Institute of Medicine's Clinical Research Roundtable,[6] and the Association of American Medical Colleges' second Clinical Research Task
Force (CRTFII)[7]—have generated recommendations and funding mechanisms for implementing clinical
research education in the United States. Specific points highlighted by these expert
panels include the need for a structured, didactic curriculum that incorporates a
core knowledge base; practical clinical research experience; mentorship opportunities;
protected research time for mentors; and formal recognition, such as a certificate
or master's degree. All of these panels emphasize the importance of introducing medical
students and physicians to clinical research as early as possible in their training.[5]
[6]
[7]
The creation of the National Eye Institute (NEI) in 1968 was a seminal event for vision
research. The NEI legitimized the field of ophthalmic research, its research budget
growing from $24 million in 1970 to $500 million in 2000 and to $708 million in 2017.[8] Research projects supported by the NEI address the leading causes of blindness and
impaired vision in the United States. The NEI currently funds multiple awards for
mentored clinician-scientists, including the Mentored Clinical Scientist Research
Career Development Award (K08), the Mentored Clinical Scientist Development Program
Award (K12), the Mentored Patient-Oriented Research Career Development Award (K23),
and the Pathway to Independence Award (K99/R00).[9]
The NEI K30 Clinical Research Curriculum Award is now a part of the NIH Clinical and
Translational Science Awards (CTSA) program, which was started in 2006 to strengthen
and support clinical and translational research (CTR). A national consortium of 62
academic health centers are supported to transform and accelerate CTR. Recognizing
the urgent need to increase the pipeline of CTR investigators, the CTSA has emphasized
Research Education and Career Development of junior CTR investigators.[10]
[11]
While there are more than 450 ophthalmology residents in the United States graduating
annually, only 39 K08 and 6 K12 awards were funded by the NEI in 2017.[12] In fact, the number of physician-scientists receiving postdoctoral research training
and career development awards is at an all-time low.[13] In addition, only 2% of ophthalmologists are actively involved in research activities
and this number will likely decline further in the foreseeable future.[14] Thus, physician-scientists are less likely to engage in biomedical research than
in the past years. The number of physicians supported by NIH training and fellowship
grants has also declined, even though the number of applications for funding has increased.[3]
[13] The dearth of clinician-scientists in ophthalmology is particularly critical to
address because of the increase in eye-care demands of the aging U.S. population.
Despite available NEI funding mechanisms, few ophthalmologists become clinician-scientists
due to lack of time to write clinical research grants and limited research training
during medical school, residency, and fellowship programs. However, ample evidence
suggests that exposing students to clinical research during medical school helps increase
their interest in research careers.[13]
[15] Early commitment to the career path of clinician-scientist, especially prior to
or during the resident training phase, is key to retaining the best ophthalmologists
in academic careers. In addition, formal mentoring programs are effective in helping
junior investigators become independent researchers.[16]
In an effort to provide a structured, didactic curriculum with practical, clinical
research experience for students interested in ophthalmology early in their careers,
we developed an 8-week Clinical Vision Research Training and Mentoring Program for
undergraduate and medical students. Wills Eye Hospital, established in 1832 as the
nation's first hospital specializing in eye care, is a global leader in ophthalmology.
U.S. News & World Report currently ranks Wills Eye Hospital as one of the top two ophthalmology hospitals
in the country. The clinical expertise, state-of-the-art diagnostic testing center,
and advanced surgical capabilities make Wills a worldwide referral center, which annually
treats more than 300,000 patients. Subspecialty services include glaucoma, neuroophthalmology,
retina, ocular oncology, pediatrics and ocular genetics, ocular pathology, cornea,
and oculoplastic and orbital surgery. Wills also serves as the Department of Ophthalmology
for Sidney Kimmel Medical College at Thomas Jefferson University and their medical
students rotate through Wills during required and elective ophthalmology courses as
well as clinical clerkships. This article describes the program's aims, course content,
lecture series, and evaluation data during the summer of 2014 and 2015.
Methods
Setting
Program Goals and Objectives
The program was developed, implemented, and evaluated by the Director of the Glaucoma
Research Center at Wills Eye Hospital. Two cohorts of students took part in the Clinical
Vision Research Training and Mentoring during the summers of 2014 and 2015. The full-time,
8-week program offered clinical research training, hands-on research experience, and
mentorship to 34 students from 12 institutions in Pennsylvania, New York, New Jersey,
Ohio, Georgia, and the District of Columbia ([Table 1]). With the largest clinical volume of patients with eye disease in the United States,
Wills offers a unique opportunity for students to gain useful clinical research experience
in ophthalmology.
Table 1
Students' university enrollment (n = 34)
|
Type of student
|
Quantity
|
University represented
|
|
Undergraduate (pre-medicine)
|
4
|
Columbia University
University of Pennsylvania
Emory University
LaSalle College
|
|
Undergraduate (public health)
|
1
|
Temple University
|
|
Graduate (medical school)
|
29
|
Sidney Kimmel Medical College at Thomas Jefferson University
Drexel University College of Medicine
Temple University School of Medicine
Philadelphia College of Osteopathic Medicine
Robert Wood Johnson School of Medicine
George Washington School of Medicine
Northeast Ohio Medical University
|
Source: Wills Eye Clinical Vision Research Training and Mentoring Program (used with
permission, 2017).
The program goals and objectives were to engage students in the following:
-
Actively participate in a vision research project.
-
Conduct vision-related literature reviews.
-
Write a scientific article for a peer-reviewed journal.
-
Receive mentoring from faculty and research staff.
This program gave students a unique introduction to ophthalmology, experience in clinical
research, and the skills necessary to conduct vision research. Students worked with
a multidisciplinary group of research coordinators and managers, biostatisticians,
data managers, and ophthalmologists to advance their knowledge, improve their manuscript
writing skills, and engender positive attitudes about vision research. Mentorship
was a key element of the program, with each student assigned to an investigator and
a member of the research team.
Recruitment and Admissions
The 2014 and 2015 Clinical Vision Research Training and Mentoring Program was directed
by the Department of Research at Wills Eye Hospital. The program was posted on the
Department of Research, Wills Eye Hospital, and Sidney Kimmel Medical College Web
sites. Candidates submitted a curriculum vitae, a letter of recommendation, and a
cover letter to the Department of Research Director in January 2014 and January 2015.
The director, faculty members, research managers, and human resources director conducted
personal interviews, selected students who had an interest in ophthalmology, prior
research experience, and planned to apply to medical school if they were an undergraduate
student. Seventeen undergraduate and medical students were selected and enrolled each
year from a total of 25 applicants in 2014 and 31 applicants in 2015.
Lecture Series
Many students entering the program had only a generalized knowledge of ocular disease
and basic research methodology. To familiarize students with common ophthalmic conditions,
as well as the basic principles of clinical research, an 11-hour lecture series was
developed and presented by Wills Eye Hospital and Sidney Kimmel Medical College faculty.
Lectures discussed glaucoma, optic neuropathy, and other ocular diseases; several
ongoing research studies being conducted at Wills and the Institutional Review Board
(IRB) policies ([Table 2]). Students participated in research projects related to these lectures.
Table 2
Lecture topics
|
Lecture
|
Topic
|
|
1
|
Introduction to ophthalmology
|
|
2
|
Introduction to glaucoma
|
|
3
|
Introduction to neuro-ophthalmology and optic neuropathy
|
|
4
|
Introduction to diabetes and diabetic retinopathy
|
|
5
|
Introduction to age-related macular degeneration
|
|
6
|
Introduction to cataract and refractive surgery
|
|
7
|
Introduction to smoking and eye disease
|
|
8
|
Glaucoma community outreach demonstration project
|
|
9
|
Quality-of-life study for glaucoma
|
|
10
|
Transcorneal electrical stimulation study for optic neuropathy
|
|
11
|
Institutional Review Board policies and procedures
|
Source: Wills Eye Clinical Vision Research Training and Mentoring Program (used with
permission, 2017).
Clinical Research Skills and Experience
The program provided students with a framework for research design, critical analysis,
and the practical skills needed to pursue an academic career in ophthalmology. Prior
to working on any research projects, students completed Collaborative Institutional
Training Initiative (CITI) and electronic medical record (EMR) training.[17] The research staff created a manual and trained the students on the Wills' EMR system.
After demonstrating proficiency, students utilized the EMR to recruit patients for
active clinical trials and extract data for IRB-approved retrospective studies requiring
ocular chart reviews. Students completed CITI training, which taught them how to identify,
recruit, enroll, and retain human subjects in research studies. The CITI training
also familiarized students with the Health Insurance Portability and Accountability
Act (HIPAA) and its protection of patients' privacy. Students had the opportunity
to obtain informed consent from patients during which they explained patients' rights
as well as researchers' expectations in a clinical trial.
We also designed a 10-hour manuscript writing workshop series for enrolled students
([Table 3]). Training modules taught students how to conduct a literature review and use Endnote
X5 software (Thomson Reuters, New York, NY). Students were assigned to an IRB-approved
research project that was near completion and ready for the development of a manuscript.
The students were allotted at least 6 to 8 hours per week for manuscript writing.
Every 2 weeks, students delivered a newly written section of their manuscript, beginning
with the introduction and continuing with the methods, results, and discussion sections.
Several research assistants with writing experience served as peer editors and provided
feedback at each juncture. The program director, research managers, and biostatistician
offered ongoing feedback and reviewed the final manuscripts.
Table 3
Manuscript writing workshop and training schedule
|
Workshop
|
Topic
|
|
1
|
Electronic medical record training
|
|
2
|
Patient recruitment and data collection training
|
|
3
|
Conducting a literature review and EndNote training
|
|
4
|
Writing the introduction
|
|
5
|
Writing the research methods
|
|
6
|
Writing the data analysis
|
|
7
|
Writing the results
|
|
8
|
Writing the discussion
|
|
9
|
Developing an abstract and poster presentation
|
Source: Wills Eye Clinical Vision Research Training and Mentoring Program (used with
permission, 2017).
During the last week of the program, students identified an appropriate peer-reviewed
journal for manuscript submission (based on its impact factor and subject matter)
and formatted their paper accordingly. With supervision from their mentor, who was
usually identified as the corresponding author, they submitted manuscripts. Students
also learned how to prepare an abstract for an ophthalmology-related conference, such
as the Association for Research in Vision and Ophthalmology (ARVO), as well as how
to develop a research poster. If a student's abstract was accepted for a conference,
he or she had the opportunity to present the research findings and were often funded
by the University or a conference research travel grant.
Enrolled students gained hands-on experience by having direct exposure to IRB-approved
research studies. They played an active role in subject recruitment, consenting, data
collection, and data analysis with our biostatistician. Students gained experience
administering baseline research questionnaires and performed visual acuity testing,
visual field testing, visual evoked potential testing, electroretinographies, and
contrast sensitivity testing. They also participated in weekly, department-specific
research meetings with their mentors, research coordinators, and managers. During
these meetings, the research team offered project updates, discussed new studies relevant
to the group's major research topics, and assigned readings to clarify the background
and purpose of the clinical research studies underway ([Table 4]).
Table 4
Students' responsibilities
|
• Recruit patients for research studies
|
|
• Perform electronic medical records review
|
|
• Develop a mentoring relationship with a member of the Wills Eye research staff
|
|
• Work closely with mentors and trainers to complete assignments
|
|
• Attend lectures and workshops
|
|
• Review scientific literature to develop a manuscript
|
|
• Write a manuscript about the assigned research study
|
|
• Identify peer-reviewed journals for manuscript submission
|
|
• Format final manuscript for submission to a peer-reviewed journal
|
|
• Complete deadlines for assignments related to the manuscript
|
Source: Wills Eye Clinical Vision Research Training and Mentoring Program (used with
permission, 2017).
Mentorship and Core Attitudes
Mentorship was also a significant component of the clinical research program and a
strong influence on students' attitudes toward vision research.[16] Each student was teamed up with a research mentor, with whom they met with several
times during the summer. This relationship offered frequent contact, performance feedback,
insight into diagnostic techniques and interpretation, and, in many cases, the potential
for continuing research after the summer. Many students who engaged in clinical research
at Wills Eye return independently during the academic year to follow up on projects.
Often, these students are able to complete research manuscripts from the data they
collected and receive publication credit for their contributions.
Students were encouraged to view their research projects in the context of patient
experience. Therefore, clinical and surgical shadowing opportunities were offered.
The degree of clinical participation varied by specialty, and students observed their
mentor in the subspecialty clinic, operating room, or the Wills Eye Emergency Room.
They learned the meaning of the data variables collected and analyzed, and consequently
learned how to associate these values with patients' ocular conditions and treatments.
Students had the opportunity to obtain informed consent from patients during which
they explained patients' rights as well as researchers' expectations in a clinical
trial. It is our hope that regular involvement with patients will reinforce the translational
aspects of research and keep students mindful of the importance that patient-centered
research can have on vision care.
Pre- and Posttest Statistical Analysis
A pre- and posttest was administered to all students on the first and last days of
the 8-week program. These tests consisted of 40 multiple choice questions within 11
topic areas and 3 to 4 questions were asked about each topic ([Table 2]). The test consisted of paired questions, so that the two tests were similar in
difficulty and subject matter. A paired-sample t-test was conducted to compare scores using SAS 9.4 software (Statistical Analysis
System [SAS] Institute, Cary, NC). This test was used to determine whether the mean
difference between the two sets of observation is zero. The null cutoff p-value for detecting statistical significance was ≤0.05.
Satisfaction Survey
The students completed an anonymous 10-question survey of their satisfaction with
the overall program, including lecture series, manuscript writing workshop series,
pre- and posttests, and mentor/trainer experience. Possible responses were on a 5-point
Likert scale, with answers varying from strongly disagree to strongly agree. Means
and standard deviations were assessed for each question.
Results
Evaluation
Students scored statistically significantly higher on the posttest (mean [M] = 79.78%,
standard deviation [SD] = 6.25) than on the pretest (M = 71.43%, SD = 8.43); p< 0.001. The largest improvement in scores was seen on the glaucoma, community outreach
for glaucoma, vision and quality of life, transcorneal electrical stimulation, and
IRB/consenting test sections (p < 0.001; [Table 5]).
Table 5
Pre- and posttest results: percentage of correct items (n = 34)
|
Covariate
|
Statistics
|
Pretest
n = 34
|
Posttest
n = 34
|
Parametric p-value[a]
|
|
General ophthalmology
|
n
|
34
|
34
|
0.239
|
|
Mean (SD)
|
84.71 (19.73)
|
79.41 (18.08)
|
|
Median
|
90
|
80
|
|
Min, Max
|
40, 100
|
40, 100
|
|
Glaucoma
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
52.94 (28.23)
|
84.71 (18.46)
|
|
Median
|
60
|
80
|
|
Min, Max
|
0, 100
|
40, 100
|
|
Child vision screening
|
n
|
34
|
34
|
0.021
|
|
Mean (SD)
|
78.68 (23.14)
|
88.97 (15.31)
|
|
Median
|
75
|
100
|
|
Min, Max
|
0, 100
|
50, 100
|
|
Community outreach for glaucoma
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
71.57 (32.96)
|
95.1 (11.98)
|
|
Median
|
83.33
|
100
|
|
Min, Max
|
0, 100
|
66.67, 100
|
|
Transcorneal electrical stimulation
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
52.21 (22.5)
|
78.68 (21.44)
|
|
Median
|
50
|
75
|
|
Min, Max
|
0, 75
|
25, 100
|
|
IRB/Consenting
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
64.71 (25.87)
|
88.24 (19.9)
|
|
Median
|
66.67
|
100
|
|
Min, Max
|
0, 100
|
33.33, 100
|
|
Statistics
|
n
|
34
|
34
|
0.148
|
|
Mean (SD)
|
69.61 (28.86)
|
58.82 (27.29)
|
|
Median
|
66.67
|
66.67
|
|
Min, Max
|
0, 100
|
0, 100
|
|
Diabetes and diabetic retinopathy
|
n
|
34
|
34
|
0.513
|
|
Mean (SD)
|
80 (17.75)
|
82.35 (14.58)
|
|
Median
|
80
|
80
|
|
Min, Max
|
20, 100
|
60, 100
|
|
Electronic medical records
|
n
|
34
|
34
|
0.074
|
|
Mean (SD)
|
92.35 (23.1)
|
79.41 (33.21)
|
|
Median
|
100
|
100
|
|
Min, Max
|
0, 100
|
0, 100
|
|
Smoking cessation
|
n
|
34
|
34
|
0.228
|
|
Mean (SD)
|
62.75 (17.91)
|
69.61 (22.27)
|
|
Median
|
66.67
|
66.67
|
|
Min, Max
|
0, 100
|
33.33, 100
|
|
Quality of life/SPARCS
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
86.76 (19.69)
|
68.38 (21.59)
|
|
Median
|
100
|
75
|
|
Min, Max
|
25, 100
|
25, 100
|
|
Overall score
|
n
|
34
|
34
|
<0.001
|
|
Mean (SD)
|
71.43 (8.43)
|
79.78 (6.25)
|
|
Median
|
74.5
|
80
|
|
Min, Max
|
52.5, 82.5
|
64, 90
|
Abbreviations: IRB, Institutional Review Board; Max, maximum; Min, minimum; SD, standard
deviation; SPARCS, Spaeth/Richman Contrast Sensitivity test.
a The parametric p-value is calculated by a paired t-test. Bold values indicate statistically significant.
Students strongly agreed that they were satisfied with the overall program and that
the lecture series enhanced their learning (M = 4.6, using 5-point Likert scale).
The lecture series (M = 4.6), the manuscript writing workshop (M = 4.6), and the editorial
mechanisms for completing their manuscripts (M = 4.5) all received very favorable
reviews ([Table 6]). We received several personal “thank you” notes via email and handwritten notes
and have provided an example from a medical student:
Table 6
Program evaluations (n = 34)
|
Evaluation questions
|
M (SD)
|
|
I was satisfied with the program
|
4.6 (0.51)
|
|
Program goals and objectives were clear
|
4.3 (0.63)
|
|
Program was organized in a manner consistent with its stated goals
|
4.3 (0.68)
|
|
Pre- and posttest questions were fair
|
4.2 (0.82)
|
|
Adequate review mechanisms were provided for manuscript development
|
4.5 (0.56)
|
|
The course materials were valuable for learning
|
4.4 (0.73)
|
|
The lecture series and pre-residency lectures enhanced my learning
|
4.6 (0.60)
|
|
I was satisfied with the lecture series overall
|
4.3 (0.71)
|
|
The manuscript workshop series enhanced my learning
|
4.0 (0.80)
|
|
I was satisfied with the manuscript workshop series overall
|
4.6 (0.88)
|
|
The abstract and poster development sessions enhanced my learning
|
4.0 (0.79)
|
|
I was satisfied with the editorial/feedback process to prepare the manuscript
|
4.4 (0.67)
|
|
I was satisfied with the program supervision
|
4.5 (0.61)
|
Abbreviations: M, mean; SD, standard deviation.
Note: Scores range from 1 to 5 (1 = strongly disagree, 5 = strongly agree)
Source: Wills Eye Clinical Vision Research Training and Mentoring Program (used with
permission, 2017).
Thank you for the opportunity to participate in the Vision Research Training and Mentoring
Program this summer. It was a tremendous privilege to be a part of the wonderful research
community at Wills Eye Hospital and especially in the Department of Research. I am
grateful to have been a part of a program where students are given such trust and
independence in their work. I treasure my experience this summer and look forward
to continuing work at Wills in the future.
– Daniel Kim, MS2 at Sidney Kimmel Medical College
Discussion
The Clinical Vision Research Training and Mentoring Program provided an evidence-based
foundation in research methods and manuscript development for students pursuing degrees
in medicine and public health. Students emerge with knowledge and skills to conduct
vision research and understand basic ophthalmology. They also benefit from the support
of a mentor in guiding their future career ambitions. The key strength of the Wills'
approach is the vast amount of clinical vision research exposure that students gain
over the course of the 8-week program. Students participate in all levels of research,
from recruiting patients and gathering and analyzing data to preparing and submitting
a manuscript. We hypothesize that this clinical research exposure enhanced students'
core knowledge of ophthalmology and vision research skills.
Students in medical schools with traditional rigorous curricula often prefer to undertake
research during the summer. Harvard University's Summer Program in Clinical Effectiveness
fosters students' interest in research and provides tools for careers in academic
medicine by teaching clinical research design and methodology.[18] Studies at the University of Tennessee and Vanderbilt University have examined the
impact of student research fellowship programs over 25 years.[15] These programs offered mentored summer research experiences for first- and second-year
medical students. A significant percentage of students performed further research,
with one-third to one-half of graduates going on to pursue careers in academic medicine.[15] However, these summer research programs tend to be available only between the first
and second years of medical school due to clinical rotation schedules.
The Wills Eye Program was more flexible and served not only medical students between
their first and second years, but also has been abbreviated to a 1-month vision research
elective format—without the lecture series and workshops—for medical students during
their third or fourth year. Though created for medical students interested in applying
to an ophthalmology residency, the program is easily adaptable for other specialties
and may encourage research in any medical specialty. Students who decided to pursue
another specialty after taking the program were able to apply basic research skills
to any clinical research project.
Given that medical education programs leading to the MD/DO degree in the United States
and Canada are accredited by the Liaison Committee on Medical Education (LCME), it
is important to consider the recent update in LCME research requirements.[19] The LCME has created standards outlined in Functions and Structure of a Medical School, which medical schools must meet to achieve and maintain accreditation.[19] According to Standards IS-13 and IS-14, an institution that offers a medical education
program should make available sufficient opportunities for medical students to participate
in research and other scholarly activities of its faculty and encourage and support
medical student participation.[19] Medical schools are required to describe the opportunities available for medical
student participation in research, including the time periods when students may do
so, the average number of students in the base year who were involved in each type
of program, and the funding sources that are available to support student participation.
The Wills Vision Research Training and Mentoring Program serves as a model for medical
schools to successfully meet these LCME standards in the departments of ophthalmology,
as well as other clinical and surgical departments.
Study Limitations
In structuring the 8-week program, we had to sacrifice some of the depth and continuity
of a typical clinical-research program. Since a research project can span months or
years from concept to completion, 2 months may not allow students enough time to participate
in the entire process. To address this limitation, we incorporated lectures and workshops
that examined various research projects at different stages. Of the students enrolled
in the 2014 program, six students chose to continue working with their research mentors
and several undergraduate students returned the following summer prior to beginning
medical school or after their first year of medical school. As this robust program
was only recently established, no longitudinal data are yet available.
