Keywords
electronic health record - documentation time - talking time - examination time -
time-motion - ophthalmology trainee - paper chart - burnout
Electronic health record (EHR) use has become commonplace in residency training programs
across the United States. While much focus has been placed on the effects of EHR use
on attending physician workload, how EHR use impacts residency training is not as
well studied.[1]
[2] Efficiency in navigating and documenting in the EHR is of paramount importance as
work-hour restrictions are enforced across training programs and as more evidence
emerges connecting EHR use to resident burnout.[3]
[4] This discussion is increasingly relevant as more studies examine EHR use and ophthalmology,
with recent data suggesting that the complexity and time associated with using an
EHR are especially concerning to ophthalmologists.[5]
[6]
Several studies have investigated the impact of an EHR on residents in specialties
including otolaryngology, emergency medicine, internal medicine, pediatrics, psychiatry,
orthopaedics, and general surgery.[7]
[8]
[9]
[10]
[11]
[12]
[13] Many of these studies focused their attention on how residents and fellows spent
their time caring for patients in an inpatient setting, while some focused on the
effect of implementation of a note template within the EHR.[14]
[15] There are few studies investigating ophthalmology trainee workflow as it relates
to an EHR, but these focus on how the presence of an ophthalmology trainee affects
overall clinic flow in which an EHR is present or on the implementation of a template
within the EHR.[16]
[17]
The purpose of this study was to examine the ways in which ophthalmology residents
and fellows (“trainees”) utilize the EHR within patient encounters while in an outpatient
clinic setting. The EHR is a powerful tool that affects how trainees learn and practice
ophthalmology and how they interact with patients. To understand the impact of the
EHR on this training, we first need to understand how the EHR is used. Understanding
how trainees utilize the EHR by evaluating the time dedicated to EHR use compared
with other clinical activities may shed light on how to better optimize EHRs for both
future trainees and physicians in general. Our study is unique in that it collected
distinct time-motion data following trainees before and after implementation of an
EHR, providing a look into how trainees spend their time in clinic on a second-to-second
basis. Detailed information about time expenditure within individual patient encounters
is important because clinical activities play a large role in trainee education. Analyzing
this data allows for a granular analysis of how much time trainees spend on a variety
of clinical tasks, providing us with an opportunity to optimize trainee education
and patient care. The University of California San Diego (UCSD) Shiley Eye Institute
and Viterbi Family Department of Ophthalmology recently underwent a transition from
paper charts to an EHR for documenting outpatient clinical encounters. By conducting
time-motion analyses of paper-based clinical workflows and of early post-EHR implementation
workflows of ophthalmology trainees in outpatient clinical encounters, we are better
able to understand how an EHR impacts ophthalmology trainee time expenditure in clinic.
Methods
The UCSD Shiley Eye Institute and Viterbi Family Department of Ophthalmology is an
academic ophthalmology department that transitioned from paper charts to an EHR (Epic
Kaleidoscope; Epic Systems, Verona, WI) for outpatient clinic encounters for half
of its faculty in September 2018. The department had already previously implemented
the EHR for patient registration and scheduling as well as in the surgical suite.
This study adhered to the tenets of the Declaration of Helsinki and was approved by
the UCSD Institutional Review Board with waiver of documented consent and HIPAA exemption.
This was a prospective time-motion study of outpatient encounters in the clinics of
attending ophthalmologists who involved trainees in direct patient care. In the typical
workflow of an attending clinic, the trainee performs an examination and assessment
of the patient prior to the attending performing an examination and assessment of
the patient. Time-motion observations were performed 2 to 3 weeks prior to EHR implementation
in September 2018 and again 5 to 6 weeks after EHR implementation in November 2018,
after all temporary onsite support staff involved in the initial implementation efforts
were no longer regularly present in clinic. Time-motion data were collected by SLB,
HEG, and trained student observers using a standardized electronic iPad-based data
entry tool (Numbers; Apple, Inc, Cupertino, CA). A templated spreadsheet tool containing
easy-to-use dropdown menus of time-stamped, observable activities was designed for
this study to reduce interobserver variability and lag time when documenting activities
(i.e., no time spent checking a watch or writing down the time). All observers underwent
a 2-hour didactic orientation before the start of data collection during the pre-EHR
phase, and an additional 2-hour training refresher session was conducted prior to
data collection in the post-EHR phase. A pilot study was performed prior to formal
data collection during seven half-day clinical training sessions. Parallel observations
were performed between the corresponding author (SLB) and observers during the pilot
phase to verify consistency, as measured by intraclass correlation coefficient exceeding
0.8 (calculated with the icc function in the psych
[18] package in R). Data collected during the pilot training sessions were not included
in the analysis. Observers minimized interactions with residents and fellows, attending
ophthalmologists, other ancillary staff, and patients. Clinical providers were instructed
to limit any interaction with observers.
Seven ophthalmology residents and four ophthalmology fellows were observed in the
clinics of attending ophthalmologists from six divisions (comprehensive, cornea, glaucoma,
pediatrics, retina, and oculoplastics). In the preimplementation study phase (documentation
on paper), time-motion observations were conducted during 13 half-day clinic sessions.
In the postimplementation study phase (documentation on Epic), time-motion observations
were conducted for six half-day clinic sessions. The inclusion criterion was any ophthalmology
resident or fellow currently working in an outpatient attending clinic at the UCSD
Shiley Eye Institute. Of note, all residents and fellows included in the study population
underwent a comprehensive, 4-hour long training session led by a UCSD Epic analyst
that was specific to Epic Kaleidoscope, during which they learned how to navigate,
operate, and practice using a “play” environment simulating real documentation and
charting. During the training session, each trainee also had the opportunity to work
with the analyst to develop customization and personalization features. Additionally,
during the first week of implementation, trainees received support in clinic from
Epic Kaleidoscope analysts on site, a centralized implementation command center accessible
via telephone, and UCSD staff support. Trainees in this study did not include medical
students. Due to changes in rotation schedules between the preimplementation study
phase and the postimplementation study phase, each participating trainee may not have
been observed in both study phases. Because the time spent on documentation, examination,
and other activities was expected to vary substantially by subspecialty clinic based
on differing clinical workflows, the decision was made to observe trainees based on
specific clinics before and after EHR implementation, rather than by individual trainees,
since individual trainees rotated to different subspecialty clinics between the two
phases of observation. Demographic information such as age, self-reported gender,
self-reported ethnicity, and primary language were recorded for each patient seen,
as well as visit type (new patient, routine follow-up, or postoperative visit within
the 90-day global period), and whether the patient was dilated. The number of exam
rooms, technicians, and patients (“clinic volume”) were recorded for each clinic session.
Total time spent by the resident or fellow with the patient, time spent documenting,
time spent examining, and time spent talking with the patient were documented. The
protocol for collecting time-motion data was based on previously published methods.[19] Documenting was broadly defined to include reviewing notes and images, writing notes,
and writing orders or prescriptions. Documenting on EHR included any use of electronics
to document, including the use of a desktop computer, tablet, or mobile phone app.
Whenever talking with the patient occurred at the same time as another activity (such
as reviewing notes or examining the patient), the nontalking activity was recorded.
Time spent performing procedures, talking with other trainees or staff, talking with
attending ophthalmologists, and waiting for patients to be ready were also recorded.
Descriptive statistics for ophthalmology residents and fellows, patients, and timing
outcomes were generated in aggregate and also by subspecialty. To examine the effects
of factors related to ophthalmology trainees, patients, and encounters on timing requirements,
linear mixed effects models were used with ophthalmology trainees and patients as
random effects. A separate linear mixed effects model was created for the subset of
EHR encounter data alone to determine whether or not prior months of EHR experience
or postgraduate year (PGY) training level had a significant impact on EHR use time
in clinic. Random intercept and random slope models were evaluated. Covariates included
patient's age, gender, ethnicity, language, visit type, dilation status, clinic volume,
and number of available technicians and exam rooms. Statistical significance was defined
as p < 0.05. Analyses were conducted in R[20] using the lme4
[21] and lmerTest
[19] packages.
Results
Seven ophthalmology residents and four ophthalmology fellows were observed, with mean
(standard deviation [SD]) PGY level of training of 3.7 (1.2). Trainee demographics
are depicted in [Table 1]. The same trainees were not necessarily observed during both the preimplementation
phase and the postimplementation phase due to changes in rotation schedules between
the two phases of observation. All 11 (100%) trainees had prior experience with an
EHR, with a mean (SD) of 50.1 (22.7) months. The average length of prior EHR experience
among the preimplementation trainee cohort was 51.75 (23.56) months, whereas average
prior EHR experience among the postimplementation trainee cohort was 44.75 (19.97)
months (p = 0.61), which was not a statistically significant difference. There were also no
significant differences in trainee age (33.1 vs. 33.0 years, p = 0.97) or PGY training level (3.75 vs. 3.25, p = 0.65) between the two study phases.
Table 1
Characteristics of ophthalmology trainees and patients included in time-motion analyses
of outpatient encounters in 2018
|
Characteristic
|
Ophthalmology trainees
(n = 11)
|
Ophthalmology patients
(n = 156)
|
|
Mean age[a] (SD, range)
|
32.8 (4.6, 27–42)
|
61.7 (20.6, 1–96)
|
|
Sex
|
|
|
|
Female
|
4 (36.4%)
|
88 (56.4%)[e]
|
|
Male
|
7 (63.6%)
|
66 (42.3%)
|
|
Race/ethnicity[b]
|
|
|
|
White
|
4 (36.3%)
|
101 (64.7%)
|
|
Black or African American
|
0 (0%)
|
5 (3.2%)
|
|
Hispanic
|
0 (0%)
|
16 (10.3%)
|
|
Asian
|
5 (45.5%)
|
23 (14.7%)
|
|
Other race or mixed race
|
2 (18.2%)
|
8 (5.1%)
|
|
Primary language[c]
|
|
|
|
English
|
8 (72.7%)
|
141 (90.4%)
|
|
Spanish
|
0 (0%)
|
5 (3.2%)
|
|
Other
|
3 (27.3%)
|
8 (5.1%)
|
|
Subspecialty[d]
|
|
|
|
Comprehensive
|
2 (16.7%)
|
23 (14.7%)
|
|
Cornea
|
2 (16.7%)
|
14 (9.0%)
|
|
Glaucoma
|
2 (16.7%)
|
24 (15.4%)
|
|
Oculoplastics
|
3 (25%)
|
56 (35.9%)
|
|
Pediatrics
|
1 (8.3%)
|
10 (6.4%)
|
|
Retina
|
2 (16.7%)
|
29 (18.6%)
|
Abbreviation: SD, standard deviation.
a Age in years at the time of the observed clinical encounter.
b Race based on self-report for ophthalmology trainees, and for patients based on self-reported
identification in the electronic registration system.
c Primary language for patients based on language patient used during the clinical
encounter.
d Subspecialty for patients indicates the subspecialty of the patient's attending ophthalmologist
in the observed clinical encounter.
e Percentages may not add up to 100% due to missing data.
The demographic information of the 156 patients whose encounters were observed are
depicted in [Table 1]. The time requirements for different clinical activities before and after EHR implementation
are depicted in [Table 2]. Tasks considered as “documentation” were similar on paper charting and EHR charting,
including reviewing and writing progress notes and procedure notes, performing medication
reconciliation, acquiring and reviewing imaging results, and ordering prescriptions.
Time spent by the trainee that is not reflected in [Table 2] represents time that was spent on other miscellaneous activities such as talking
with the attending ophthalmologist or staff members about the patient, assisting with
procedures, or completing administrative tasks.
Table 2
Time spent on patient encounter by ophthalmology trainees ∼2 weeks before and ∼6 weeks
after EHR implementation
|
Before EHR implementation
|
After EHR implementation
|
|
Minutes, mean (SD)
|
Percentage of total time[c]
|
Minutes, mean (SD)
|
Percentage of total time[c]
|
|
Documentation time[a]
|
5.4 (3.5)
|
48%
|
6.8 (4.7)
|
57%[*]
|
|
Examination time[a]
|
3.4 (3.3)
|
28%
|
2.9 (2.5)
|
24%
|
|
Talking time[a]
|
2.8 (2.8)
|
24%
|
2.0 (1.6)[*]
|
18%[*]
|
|
Total time[b]
|
11.6 (6.5)
|
|
11.8 (6.9)
|
|
Abbreviations: EHR, electronic health record; SD, standard deviation.
* Indicates p < 0.05.
a For activities occurring simultaneously (such as examining while talking to the patient),
the nontalking activity was recorded.
b Total time spent by the trainee on the patient encounter during the clinic session.
This includes all time spent on the patient's care during the observed clinic session
and was not limited to time in the clinic room with the patient.
c Percentages do not add up to 100%, as other activities may have been performed for
patient care, including performing procedures and talking with attendings or with
staff about the patient. Percentage time for each activity was calculated per patient
and then averaged, but since not all activities were performed for each patient, percentages
do not add to 100%.
Total time expenditure per patient was not significantly changed after EHR implementation
(+0.17 minutes, 95% confidence interval [CI] for the difference in means: –2.78, 2.45;
p = 0.90). Similarly, documentation time did not change significantly after EHR implementation
in absolute terms (+1.42 minutes, 95% CI: –3.13, 0.29; p = 0.10). However, the proportion of time spent on documentation was significantly
increased on the EHR (48% on paper to 57% on EHR; 95% CI: 2.17, 15.83; p = 0.011). Examination time was not significantly changed after EHR implementation
in absolute terms (–0.49 minutes, 95% CI: –0.55, 1.52; p = 0.35) or in proportional terms (–4%, 95% CI: –8.86, 2.65; p = 0.28). Time spent exclusively talking with each patient was significantly less
after EHR implementation in both absolute terms (–0.77 minutes, 95% CI: –0.04, –1.50;
p = 0.04) and proportional terms (–6%, 95% CI: –0.87, –10.92; p = 0.022).
After EHR implementation, trainees spent a significantly larger proportion of the
total time per patient inside the clinic room (84 vs. 70%, 95% CI: –22.40, –6.40,
p < 0.01), demonstrating a trend of performing more clinical documentation in the clinic
room with the patient present rather than documenting in a paper chart in the hallway
between encounters, as average time spent documenting in the hallway decreased from
3.32 minutes prior to EHR implementation to 2.42 minutes after EHR implementation.
Across all observed patient encounters, increasing PGY training level was associated
with significantly less total time expenditure per patient (1.3 minutes less per additional
year of training, p = 0.004), and dilation was associated with significantly greater total time expenditure
per patient (increase of 3.29 minutes with dilation, p = 0.002). However, neither PGY training level, patient dilation status, nor other
patient demographics (age, gender, ethnicity) significantly influenced time required
for documentation across all observed encounters.
Encounters observed after EHR implementation were evaluated separately to assess whether
prior EHR experience in other settings (e.g., medical school) and PGY training level
influenced total time or documentation time per patient. Neither prior months of EHR
experience nor PGY level had a significant influence on total time or documentation
time required per patient.
Discussion
Understanding trainee time expenditures during patient encounters is critical for
evaluating how EHR implementation affects clinical workflows in academic centers.
Additionally, because trainees shoulder a substantial burden of clinical care documentation,[3] understanding these time requirements can also shed light on the trainee experience.
This study analyzed the time requirements of various clinical activities during outpatient
clinical encounters conducted by ophthalmology trainees before and after EHR implementation.
Key findings from this study were that overall time spent per patient encounter did
not change at 5 to 6 weeks post-EHR implementation, the proportion of time spent documenting
with the EHR increased, and time spent exclusively talking with the patient during
the encounter decreased after the EHR was implemented.
While EHR use is sometimes thought to lengthen clinical encounters among ophthalmologists
and the field of medicine as a whole, studies examining the impact of EHR implementation
on ophthalmology practices have demonstrated a mixed impact on patient volume and
the time physicians spend in clinic.[5]
[22]
[23]
[24]
[25]
[26]
[27]
[28] In our study, the total time that residents and fellows spent per patient encounter
did not change with EHR implementation. Average time spent on a patient encounter
during a paper chart-based encounter was 11.6 minutes, compared with 11.8 minutes
spent on a patient encounter when using the EHR. Furthermore, this equivalent total
time expenditure on patients between paper and electronic documentation occurred only
5 to 6 weeks after EHR implementation, when presumably trainees are still learning
how to use the EHR. This lack of change in total time expenditure is similar to that
found by Victores et al when examining the workflow of otolaryngology residents on
clinic days.[12] In their study, overall efficiency was not affected by the implementation of an
EHR, but time spent in clinic was shifted from direct patient care to indirect patient
care (defined as documenting and reviewing the medical record and results). This is
similar to our study's finding that proportionally more time was spent documenting
the clinical encounter when using the EHR versus the paper chart ([Table 2]). This could have also been affected by the fact that the EHR was newly implemented
in the observed clinics. Therefore, many of the observed patients were “new to the
EHR” even if they were not “new patients” to the clinic per se, because their clinical
data had not yet been migrated to the EHR. It is possible that this initial data migration
would have taken more time compared with subsequent visits. This represents an area
of future study.
Data collection revealed that trainees spent statistically significantly less time
talking exclusively with the patient during an encounter that was charted using the
EHR compared with using paper documentation (2.0 vs. 2.8 minutes, p = 0.04). Consistent with previously published methods for conducting time-motion
studies in ophthalmology,[19]
[29] we only recorded the trainee as “talking” if no other activity was being performed
simultaneously (such as documenting, examining, or performing procedures). If the
trainee was engaged in multitasking, only the nontalking activity was recorded. For
instance, if the trainee was talking to the patient while also typing in the EHR,
“documenting” was the recorded activity. The decrease in time spent talking exclusively
with the patient but the proportional increase in time spent documenting could suggest
that trainees were engaged in more frequent multitasking (e.g., talking while documenting
simultaneously) after EHR implementation, but further studies recording multitasking
data may shed light on the distribution of time during an encounter.
Of note, the proportion of total time spent with the patient in the clinic room itself
increased from 70 to 84% after EHR implementation. This may be accounted for by the
fact that paper chart review was previously performed outside of the clinic room in
the hallway before EHR implementation, whereas after EHR implementation chart review
was conducted on a desktop computer inside the clinic room. For years, studies have
shown that longer ambulatory visits with time spent with the provider are associated
with increased patient satisfaction.[30]
[31] In addition, the patient's estimation of the visit length also plays a role in satisfaction
with their provider.[30] Taking these two factors into account, it is interesting to consider the impact
of the EHR increasing trainee time spent with the patient while also possibly influencing
the patient's and trainee's perceived time spent with each other. However, while there
is an objective increase in face-to-face time with the patient if the trainee is in
the exam room for a longer period of time, studies show that what occurs during that
time is also important. Marmor et al demonstrate that EHR usage may negatively influence
a patient's perception of a physician's communication skills and overall satisfaction
of care.[32] Additionally, the perceived increase in time with a patient while performing data
collection through the EHR may also influence trainees, leading them to believe they
require less history from the patient themself.[32]
Another important consideration when examining increased proportional documentation
time is the impact that this increase has on trainees' perceptions of how they spend
their time. Using the EHR, it may be that trainees had access to a larger amount of
existing progress notes, laboratory results, and operative notes when using an EHR
spanning the whole health system and encompassing multiple specialties than they had
when they were reviewing patients' medical information in the paper chart, which only
included information from ophthalmology encounters. It is possible that the increase
in observed documentation time could be increasing trainee time spent multitasking
and thus increasing cognitive burden on trainees who routinely experience significant
task interruptions and task switching.[33]
[34] EHR use time has been identified as the strongest predictor of burnout among residents,
above factors such as sleep and exercise,[3] and the Massachusetts Medical Society recently named EHR reform as one of the three
main ways to address physician burnout.[35] Despite our study and others finding equivalent encounter time expenditure while
using the EHR as compared with paper-based documentation, it could be that the reported
impact of the EHR on trainee burnout rates can be explained by the cognitive burden
of task fragmentation created by interacting with the EHR environment.[36]
[37] While there may be an educational benefit to having increased information available
to the trainee in the form of the EHR, this benefit needs to be weighed against factors
that increase the cognitive burden of the EHR.
The number of months of prior EHR experience and PGY level of training did not significantly
affect total time or documentation time per patient during the post-EHR implementation
phase of this study. All trainees observed had extensive prior EHR experience, as
the least experienced EHR user had previously used an EHR for 12 months. Therefore,
we were not able to evaluate how time demands of an EHR would affect novice EHR users
with less than 12 months of experience. Rodriguez Torres et al also examined the relationship
between a trainee's level of experience and their EHR use, demonstrating that the
type and format of an EHR play a more significant role in an ophthalmology trainee
following institutional documentation guidelines than does level of training.[17] While compliance with documentation was the measured metric in their study rather
than timing, it is interesting to note that the design of the EHR played a more important
role in documentation compliance than did years of training. Trainees in our study
had previously used a variety of EHR platforms, and future studies may explore the
role of different EHR platforms on trainee education.
An increasing emphasis has been placed on standardizing EHR use and exposure from
the start of medical education.[38]
[39]
[40] Most medical schools and academic medical centers provide medical students and residents
with EHR access,[41] but training in these settings may be varied and can result in a wide range of documentation
styles and outcomes.[42] How different methods of EHR training impact educational outcomes and efficiency
(whether for trainees or for practicing physicians) may be worthwhile future directions
of study.
The UCSD ophthalmology department's paper-to-EHR transition may be a rare occurrence
as EHRs become universally adopted, but EHR-to-EHR transitions and updates to existing
systems may mimic some of the resulting outcomes observed in our study. While potentially
less dramatic than paper-to-EHR transitions, EHR-to-EHR transitions may still entail
major changes in clinical information systems and clinical workflows, with potential
impacts on trainee experience and education. Our results may also be used to start
establishing benchmarks to study other problems. As previously discussed, EHR use
time has been identified as an important factor contributing to trainee burnout.[3]
[35] Perhaps observed impacts on clinic flow and resident burnout rates would be minimized
with more frequent training and optimization sessions that would reduce documentation
burden for trainees. It is thus important to continue examining how ophthalmology
trainees utilize the EHR, how the EHR affects residency education, and how EHR-specific
training may optimize EHR use for trainees.
In summary, this study evaluated the time demands of outpatient clinical encounters
of ophthalmology trainees at an academic ophthalmology clinic that recently underwent
EHR implementation. Unlike other studies that followed trainees throughout their day
during various inpatient, outpatient, and didactic settings and collected broad aggregated
timing data on different clinical activities, our study collected detailed timing
data on activities performed within individual patient encounters on a second-to-second basis, allowing us to understand
how ophthalmology trainees spend their time in clinic with individual patients before
and after EHR implementation. We observed that the EHR had little effect on overall
time spent with patients but that it did change how trainees divided that time between
various clinical activities. As EHRs have been widely adopted among academic centers,
it is important to understand how health information technology may influence trainee
education so as to improve patient care, learning opportunities, and documentation
optimization for ophthalmology trainees.
