Keywords
ophthalmology - applicant - residency - medical student - digital media - decision-making
- information gathering
Every year, hundreds of medical students dedicate significant time and energy to apply
for ophthalmology residency using the San Francisco Residency and Fellowship Matching
Services (SF Match), representing an important milestone that shapes the trajectory
of their future careers. The recruitment process is expensive for both applicants
and residency programs. Applicants spend an average of $5,704 each application cycle.[1] Residency programs review an increasing number of applications each year despite
funding cuts for graduate medical education from the Center for Medicare and Medicaid
Services.[1] The average expense for each interview, including faculty, staff, and administrative
time, is estimated to reach $1,042. This excludes recruitment costs such as student
tours, marketing, and review of noninterviewed candidates.[2]
[3]
The application cycle itself spans an entire year and involves three distinct stages
for applicants: deciding where to apply, interview, and, ultimately, rank. Prior literature
exploring applicant decision-making have focused on financial and interview scheduling
considerations,[1]
[4] but have not explored the decision-making process at each stage. Deciding where
to apply versus interview and rank are distinct decisions with widely variable levels
of commitment. For example, applicants often decide where to apply with limited firsthand
knowledge of most programs, while a decision to rank is often informed by additional,
multifaceted considerations after visiting and interviewing at a program.
Online digital resources are increasingly used by applicants to gather information
and make decisions.[5] These resources include residency program Web sites as well as databases and interactive
platforms such as Twitter, YouTube, Reddit, Doximity, and the FREIDA residency program
database. Which resources are most important and the types of content applicants find
most useful at each stage of the decision-making process have not been explored. These
questions are particularly important in light of the coronavirus disease 2019 (COVID-19)
pandemic, which rendered all interviews virtual.[6] As a result, both applicants and residency programs rely exclusively on digital
communication to make appropriate match decisions. In addition, while the match rate
has remained relatively stable at 74 to 78%, the mean number of applications per person
has risen from 48 in 2008 to 80 in 2021.[7]
[8] This indicates that medical students are applying to a greater number of programs
with which they arguably have little direct experience.
The purpose of this study is to evaluate the resources that applicants use at each
stage of the residency application process: deciding where to apply, interview, and
rank. A secondary aim is to identify the types of digital content applicants find
most useful in their decision-making process. These findings can aid residency program
leaders in recruiting applicants who are a good fit for their program.
Materials and Methods
This study was conducted in accordance with the tenets of the Declaration of Helsinki
and Good Clinical Practice Guidelines, and designated exempt by the University of
California, San Francisco (UCSF), Institutional Review Board. The study is a cross-sectional
survey of 2019 to 2020 and 2020 to 2021 ophthalmology residency applicants who applied
to the UCSF program. After obtaining informed consent, participants were queried via
anonymous questionnaire through the Qualtrics (Qualtrics XM) software. Given that
no similar studies had been conducted for ophthalmology,[9] the questionnaire was created after extensive literature review and input from ophthalmology
program directors (PDs) and directors of medical student education at various institutions,
and validated by current ophthalmology residents at UCSF.
A total of 14 content topics (19 items) were included in the questionnaire ([Appendix A], available online only), which consisted of four sections inquiring about demographic
information, match outcomes, how students gathered resources when learning about programs,
and how students made decisions at each stage of the application process: deciding
where to (1) apply, (2) interview, and (3) rank. Specific questions pertained to the
importance applicants placed on digital mediums such as social media, online databases,
and residency program Web sites compared with more traditionally surveyed considerations
such as program structure and faculty when making their residency application decisions.[1]
[10] For instance, applicants were queried about the information they sought on residency
program Web sites, how they engaged with various digital platforms, and the content
topics they felt were most important to include when learning about programs.
The questionnaire remained open for completion between July 12, 2020 to August 31,
2020 and from February 11, 2021 to March 31, 2021, with three email reminders during
both periods. Responses were collected anonymously, written consent was obtained at
the start of the questionnaire, and the invitation to participate in the study was
sent to the applicants only after the completion of each ophthalmology residency match
cycle to ensure that applicants' participation status would not affect residency program
rank lists. Statistical analysis and data visualizations were conducted using RStudio
Team (2020) and Microsoft Excel Version 16.47.1 (Microsoft), with p< 0.05 considered statistically significant.
Results
Demographics
Of 870 invited participants, 136 responses were received, for a response rate of 15.6%
([Table 1]). The mean (standard deviation [SD]) age of applicants was 27.7 (2.56) years. Most
respondents were single (n= 98 [72.1%]) and had no children (n=125 [91.2%]). The mean (SD) number of ophthalmology residency program applications
per respondent was 74.8 (22.4), with a range of 30 to 120 programs. In addition to
these numerical values, write-in responses (n=8) included students who expressed the need for having an application cap moving
forward; one respondent commented that they and their peers generally took every interview
offered and “were not in any position to turn down any sort of opportunity.” Among
respondents, 63.2% (n=86) were first-time applicants, 3.7% (n=5) were not first-time applicants, and 33.1% (n=45) chose not to respond. Additionally, 61.8% (n=84) matched to an ophthalmology residency program, 5.1% (n=7) did not match, and 33.1% (n=45) did not respond. Additional demographics broken down by application cycle (2019–2020
vs. 2020–2021) are shown in [Table 1].
Table 1
Applicant demographics
|
Question
|
2019–2020 Applicant (n=63)
|
2020–2021 Applicant (n=73)
|
|
Gender identity
|
|
|
|
Male
|
34 (54.0%)
|
44 (60.3%)
|
|
Female
|
29 (46.0%)
|
29 (39.7%)
|
|
Nonbinary
|
0 (0%)
|
0 (0%)
|
|
Prefer to self-identify
|
0 (0%)
|
0 (0%)
|
|
Prefer not to state
|
1 (1.6%)
|
0 (0%)
|
|
Ethnicity
|
|
|
|
American Indian or Alaska Native
|
1 (1.6%)
|
1 (1.4%)
|
|
Asian
|
25 (39.7%)
|
23 (31.5%)
|
|
Black or African American
|
0 (0%)
|
8 (11.0%)
|
|
Hispanic or Latinx
|
5 (7.9%)
|
6 (8.2%)
|
|
Native Hawaiian or Pacific Islander
|
0 (0%)
|
0 (0%)
|
|
White
|
30 (47.6%)
|
35 (47.9%)
|
|
Prefer to self-describe
|
3 (4.8%)
|
1 (1.4%)
|
|
Prefer not to state
|
1 (1.6%)
|
1 (1.4%)
|
|
Age (y), mean±SD
|
27.3±2.03
|
28.0±2.91
|
|
Marital status
|
|
|
|
Single
|
47 (74.6%)
|
51 (69.9%)
|
|
Married
|
15 (23.8%)
|
20 (27.4%)
|
|
Divorced
|
0 (0%)
|
1 (1.4%)
|
|
Separated
|
0 (0%)
|
0 (0%)
|
|
Widowed
|
0 (0%)
|
0 (0%)
|
|
Prefer not to state
|
1 (1.6%)
|
1 (1.4%)
|
|
Applicant with children?
|
|
|
|
Yes
|
3 (4.8%)
|
8 (11.0%)
|
|
No
|
59 (93.7%)
|
65 (89.0%)
|
|
Prefer not to state
|
1 (1.6%)
|
0 (0%)
|
|
Location of medical school
|
|
|
|
U.S. - West
|
8 (12.7%)
|
12 (16.4%)
|
|
U.S. - Southwest
|
6 (9.5%)
|
6 (8.2%)
|
|
U.S. - Southeast
|
8 (12.7%)
|
15 (20.5%)
|
|
U.S. - Midwest
|
17 (27.0%)
|
17 (23.3%)
|
|
U.S. - Northeast
|
21 (33.3%)
|
21 (28.8%)
|
|
International
|
3 (4.8%)
|
2 (2.7%)
|
|
Location of home
|
|
|
|
U.S. - West
|
20 (31.7%)
|
21 (28.8%)
|
|
U.S. - Southwest
|
7 (11.1%)
|
6 (8.2%)
|
|
U.S. - Southeast
|
7 (11.1%)
|
9 (12.3%)
|
|
U.S. - Midwest
|
10 (15.9%)
|
14 (19.2%)
|
|
U.S. - Northeast
|
14 (22.2%)
|
22 (30.1%)
|
|
International
|
5 (7.9%)
|
1 (1.4%)
|
|
Number of residency programs applied, mean±SD
|
74.3±22.2
|
75.2±22.8
|
|
First-time applicant?
|
|
|
|
Yes
|
39 (61.9%)
|
47 (64.4%)
|
|
No
|
1 (1.6%)
|
4 (5.5%)
|
|
No response
|
23 (36.5%)
|
22 (30.1%)
|
|
Matched?
|
|
|
|
Yes
|
37 (58.7%)
|
47 (64.4%)
|
|
No
|
3 (4.8%)
|
4 (5.5%)
|
|
No response
|
23 (36.5%)
|
22 (30.1%)
|
Abbreviation: SD, standard deviation.
Note: The ethnicity groupings were presented according to the official choices available
to applicants under the Association of American Medical Colleges (AAMC), and applicants
were able to select as many groups as they self-identified with. Results are displayed
based on whether the respondent was an applicant from the 2019–2020 or the 2020–2021
interview cycle.
Factors Affecting Applicant Decisions to Apply to Ophthalmology Residency Programs
When asked to rank 12 information sources when learning about ophthalmology residency programs ([Table 2]), the most important factors were current residents at the program, PDs at the program,
faculty or faculty interviewers, and program Web sites. Factors such as direct personal
experience with a program (e.g., away rotations) and ranking platforms (e.g., U.S.
News and World Report, SF Match program directory) had the greatest variance in the
evaluation of importance. Free-text responses (n=13) included geography or family preference (n=4 [30.8%]), mentor opinions (n=2 [15.4%]), alumni careers (n=2 [15.4%]), representation of fellowships (n=1 [7.7%]), and webinars hosted during the virtual application cycle (n=1 [7.7%]).
Table 2
Most important resources in learning about ophthalmology residency programs
|
Factor
|
Mean score±SDV
|
|
Current residents at the program
|
3.34±2.23
|
|
Program director(s) at the program
|
4.94±2.32
|
|
Faculty or faculty interviewers at the program
|
5.02±2.36
|
|
Residency program Web site
|
5.03±2.61
|
|
Direct personal experience with a program (e.g., away rotations)
|
5.86±4.57
|
|
Ophthalmology faculty at your medical school
|
6.02±2.90
|
|
Ranking platforms (U.S. News and World Report, SF Match program directory)
|
6.06±3.18
|
|
Online forums (Reddit, Student Doctor Network, Doximity, etc.)
|
6.66±2.85
|
|
Ophthalmology residents at your medical school
|
7.25±2.60
|
|
Classmates applying to ophthalmology from your medical school
|
8.36±2.61
|
|
Residency program social media pages (e.g., Twitter)
|
8.61±2.32
|
|
Other
|
10.84±3.12
|
Abbreviations: SDV, standard deviation of volume; SF Match, San Francisco Residency
and Fellowship Matching Services.
Note: Respondents (n=136) ranked a total of 12 information sources in order of importance (1=most important,
12=least important).
The most important resources when deciding to apply were ranking platforms, direct personal experience with programs, and digital platforms
([Fig. 1]). Of secondary importance were faculty, career advisors, residents, and PDs. Free-text
responses (n=10) had a common theme of either location (n=4 [40.0%]) or “applied to all programs” (n=2 [20.0%]).
Fig. 1 Most important resources in deciding to apply. Respondents (n=136) determined the importance of 13 resources on influencing their decision to apply
to a particular residency program, using a 5-point Likert scale (1=extremely important,
5=not at all important). Resources are ordered in decreasing importance. The black
dots indicate the mean numerical rating from respondents. PD, program director.
Factors Affecting Applicant Decisions to Interview at Ophthalmology Residency Programs
When deciding to interview at an ophthalmology program, the most important resources were ranking platforms,
direct personal experience with programs, residency program outreach/communications,
and digital platforms ([Fig. 2]). Similar to above, of secondary importance were faculty, career advisors, residents,
and PDs. Free-text responses (n=11) included whether they were invited to interview (n=5 [45.5%]), program location (n=2 [18.2%]), and ease of interview scheduling (n=1 [9.1%]).
Fig. 2 Most important resources in deciding to interview. Respondents (n=136) determined the importance of 13 resources on influencing their decision to interview
at a particular residency program, using a 5-point Likert scale (1=extremely important,
5=not at all important). Resources are ordered in decreasing importance. The black
dots indicate the mean numerical rating from respondents.
Factors Affecting Applicant Decisions to Rank Ophthalmology Residency Programs
When deciding to rank an ophthalmology program, the most important resources were direct personal experiences
with a program, ranking platforms, residency program outreach/communications, and
residents at the ophthalmology programs ([Fig. 3]). Resources such as PDs, other faculty, and digital platforms were less important
in making the decision to rank. Free-text responses (n=12) had common themes of interview experience (n=4 [33.3%]), location/cost of living (n=4 [33.3%]), and program culture (n=3 [25.0%]).
Fig. 3 Most important resources in deciding to rank. Respondents (n=136) determined the importance of 13 resources on influencing their decision to rank
a particular residency program, using a 5-point Likert scale (1=extremely important,
5=not at all important). Resources are ordered in decreasing importance. The black
dots indicate the mean numerical rating from respondents.
How Applicants Used Digital Media during the Ophthalmology Residency Application Process
When asked to rate program Web site content topics by importance, respondents noted
the number of residents accepted per year, resident alumni job or fellowship placement,
the current resident listing, curriculum and didactics, call schedule, and rotation
schedule as “extremely important” ([Fig. 4]). Resident benefits (e.g., salary, housing, vacation), research opportunities, extracurricular
opportunities, comprehensive faculty listing, media, published research projects by
residents, message from the PD/chair/faculty, and published research projects by faculty
were all felt to be “very important” to include on ophthalmology program Web sites,
in decreasing order of importance. Notably, on average, applicants felt that all 14
content topics were at least moderately important to display on program Web sites.
Fig. 4 Content most important to applicants on ophthalmology residency program Web sites.
Respondents (n=136) ranked a total of 14 content topics on a 5-point Likert scale (1=extremely important,
5=not at all important). Content topics are ordered in decreasing importance. The
black dots indicate the mean numerical rating from respondents.
When queried about how they engaged with digital platforms ([Table 3]), all respondents (n=103 [100.0%]) reported engaging with residency program Web sites, while the majority
also engaged with emails from residency programs (n=88 [85.4%]), Doximity (n=82 [79.6%]), Reddit (n=64 [62.1%]), Instagram (n=59 [57.3%]), the FREIDA residency program database (n=55 [53.4%]), and YouTube (n=53 [51.5%]). Platforms such as Student Doctor Network, the Texas STAR survey, Twitter,
American Medical Association resources, and Facebook were less popular, but these
digital platforms still received engagement from at least 25% of survey respondents.
Among all resources, passive engagement such as reading and watching videos was the
most popular form of use, ranging from 91.5 to 100.0%, while active engagement such
as sharing, commenting, and asking questions varied depending on the platform. Write-in
responses (n=11) included the OphthoMatch Google spreadsheet (n=10 [90.9%]) and webinars hosted by the residency program (n=1 [9.1%]). OphthoMatch is an applicant-managed, shared spreadsheet tracking applicant
statistics, interview offers, postinterview reviews, impressions of programs from
away rotations or interviews, and advice from applicants in prior years. Active engagement
was most commonly noted for the OphthoMatch Google spreadsheet (n=8 [80.0%]), Twitter (n=9 [26.5%]), and emails from residency programs (n=18 [20.5%]).
Table 3
Applicant engagement with digital platforms
|
Resource
|
Total engagement
|
Passive engagement
|
Active engagement
|
|
Residency program Web sites
|
103 (100.0%)
|
102 (99.0%)
|
4 (3.9%)
|
|
Emails from residency programs
|
88 (85.4%)
|
84 (95.5%)
|
18 (20.5%)
|
|
Doximity
|
82 (79.6%)
|
82 (100.0%)
|
2 (2.4%)
|
|
Reddit
|
64 (62.1%)
|
61 (95.3%)
|
11 (17.2%)
|
|
Instagram
|
59 (57.3%)
|
54 (91.5%)
|
9 (15.3%)
|
|
FREIDA residency program database
|
55 (53.4%)
|
55 (100.0%)
|
3 (5.5%)
|
|
YouTube
|
53 (51.5%)
|
53 (100.0%)
|
0 (0.0%)
|
|
Student Doctor Network (SDN)
|
50 (48.5%)
|
50 (100.0%)
|
4 (8.0%)
|
|
Texas STAR Survey
|
36 (35.0%)
|
36 (100.0%)
|
0 (0.0%)
|
|
Twitter
|
34 (33.0%)
|
34 (100.0%)
|
9 (26.5%)
|
|
American Medical Association (AMA) resources
|
29 (28.2%)
|
29 (100.0%)
|
1 (3.4%)
|
|
Facebook
|
26 (25.2%)
|
26 (100.0%)
|
2 (7.7%)
|
|
Other
|
12 (11.7%)
|
12 (100.0%)
|
9 (75.0%)
|
Note: Respondents (n=103) interacted with 13 digital resources when learning about ophthalmology residency
programs. Results are ordered by most to least amount of total engagement (n, %). Percentages for passive engagement (reading) and active engagement (sharing,
commenting, asking questions) were calculated as a fraction of the total engagement
for each digital platform.
Discussion
The annual ophthalmology residency match requires considerable participation and investment
from faculty, residents, program coordinators, and other staff. In recent years, digital
platforms such as program Web sites and online forums have played an increasingly
important role in this process, given that the average applicant applies to 80 residency
programs.[7] The digitization of information has been further accelerated by the implementation
of virtual interviews during the COVID-19 pandemic. The majority of applicants and
PDs now indicate that a virtual or hybrid (virtual/in-person) interview format will
likely continue for future application cycles; a recent cross-sectional study found
that if given a choice regarding the future direction for interviews, 73.4% of applicants
would prefer to hold interviews virtually or were unsure.[6] Therefore, understanding the resources that applicants use to decide where to apply,
interview, and rank is critical to conveying relevant information and fostering effective
matches for both programs and applicants.
One of our goals was to understand whether applicants used different resources and
factors to make decisions at each of the three stages of the application process.
This distinction has not been examined previously, and respondents to this study indeed
noted that surveys often fail to distinguish between factors that lead one to apply
to, versus interview or rank, a program, which are very different decisions for the
applicant.[1]
[4]
Our analyses demonstrate the importance of digital platforms for applicant decisions
on where to apply and interview, but not necessarily rank. Digital platforms were
rated as more important resources than faculty, career advisors, residents, and PDs
during these first two stages of the application process. However, these same platforms
became far less important for the final rank list decisions. These findings imply
digital platforms are one of the most high-yield resources that PDs should utilize
early in the application cycle to target applicant recruitment, as online content
most directly impacts whether an individual will apply to the program. As a result,
it may be beneficial to time digital platform interactions and content (e.g., emails,
videos) to the summer before the residency application deadline.
Among all digital platforms, ophthalmology program Web sites were the single most
important resource for applicants, used by 100% of respondents. The three most important
content topics that applicants sought on Web sites were the number of residents accepted,
alumni job placement, and current resident listings. Although ophthalmology residency
program Web sites have not been studied in detail, a 2018 study of urology residency
program Web sites showed that only 76% included the number and names of current residents,
while only 39% listed alumni names, fellowships, and jobs.[11] A 2020 analysis of physical medicine and rehabilitation residency program Web sites
found similar rates of current resident listings at 71.3%.[9] The ease of navigating to such information on program Web sites is important but
often lacking.[12] Our survey demonstrates the importance of resident listings and alumni placements,
which should be made clear on program Web sites. The inclusion of other topics such
as curriculum, call schedule, benefits, and extracurricular opportunities are also
highly valuable to applicants seeking to learn about ophthalmology residency programs.
A significant proportion of respondents engaged with other digital platforms such
as Doximity, Reddit, Instagram, the FREIDA residency program database, and YouTube.
The vast majority (91.5–100%) of applicants who used digital platforms passively engaged
with the content, reading posts and text written by others. In contrast, few applicants
(0–26.5%) actively engaged with digital platforms, including sharing information or
asking questions. These findings suggest that while an ophthalmology program's articles
and posts online may appear to have low levels of engagement (e.g., few “likes” and
comments), the true readership of this content is likely to be much higher. Thus,
it remains important for residency programs to maintain a presence on these digital
platforms despite seemingly low rates of reader interaction.
More generally, applicants felt that the most important resources in learning about
ophthalmology residency programs were current residents at the program, followed by
PDs and faculty. Many of the most important factors in formulating applicant rank
lists were subjective factors such as the perceived happiness of residents and faculty,
interview experiences, and geographic location. These results are in line with studies
in other specialties that point to the importance of resident happiness and other
subjective features.[10]
[13]
[14]
[15] A well-organized interview day involving extensive interaction with ophthalmology
faculty and trainees can provide applicants a critical window into program culture
and camaraderie and allow them to evaluate their own fit within the program.
There are several limitations to our study. The 872 applicants invited to participate
in this study represented 66.5% of the 1,312 participants in the 2019 to 2020 and
2020 to 2021 ophthalmology residency match. While they constitute a majority of the
entire applicant pool, only individuals who applied to the UCSF ophthalmology residency
program were invited to participate in the study, which represents a selection bias.
However, while limited demographic data are available through the SF Match Ophthalmology
Residency Match Summary Report, the demographics of our respondents (including gender,
race, and geographic location) are similar to those of applicants reported in recent
studies, supporting the external validity of our study.[1]
[6] Our survey only captured responses from 7 of 179 (5.1%) unmatched applicants, although
a sizeable proportion of respondents (33.1%) did not indicate whether they had matched
or were first-time applicants. Furthermore, we did not assess how many interviews
each respondent received, which impacts how interviews were chosen and how rank lists
were developed. Despite the assurance that all responses were anonymous, applicants
may have been concerned that their answers may influence future applications, which
has been shown in previous studies in other specialties.[10] Applicant access to faculty mentors at home programs was also not assessed.
Importantly, the COVID-19 pandemic may have affected results in ways that the survey
was unable to assess for the 2020 to 2021 application cycle. For example, students
were unable to participate in away rotations, which may have impacted the degree of
their direct personal experience with programs. Additionally, as some medical schools
adjusted their clinical curricula, students may have had more free time to peruse
digital media when making residency application decisions. Furthermore, the survey
questions did not assess whether student engagement with digital media differed between
residencies or whether students viewed all digital media available to each program.
Finally, this study is subject to both recall bias (applicants completed the survey
after they had matched) and memory bias (questionnaire was administered at two different
times in the calendar year for the two application cycles).
In conclusion, ophthalmology residency applicants consult a multitude of digital resources
and consider numerous factors when deciding where to apply, interview, and rank. Given
an increasing number of program applications each year and greater reliance on virtual
components, it is critical for residency programs to dedicate attention to the information
they share online. Digital platforms such as residency program Web sites and online
forums should be utilized early in the application cycle, as the vast majority of
applicants consult these resources when deciding where to apply. Personal interactions
with residency programs through interviews and emails are critical late in the application
cycle when applicants craft their rank lists. In the advancing digital era, understanding
applicants' decision-making is critical to fostering effective matches for both programs
and future residents.
