Keywords electronic health records - ambulatory care - workload
Background and Significance
Background and Significance
Office-based physician adoption of electronic health records (EHRs) has more than
quadrupled in the past 15 years with the establishment of the Meaningful Use Program.
As of 2017, 86% of office-based physicians had adopted an EHR.[1 ]
[2 ] The digitization of EHRs, as it was intended, may have led to advances in medical
practice such as potential for reduction in hospital mortality and potentially preventable
hospital admissions.[3 ]
[4 ] However, the use of clinical documentation to satisfy demands peripheral to direct
patient care such as to meet billing, quality, and compliance requirements is a potential
source of frustration among physicians, an unintended consequence of EHR adoption.[5 ]
[6 ]
[7 ]
[8 ]
[9 ] Clinicians in the United States spend significantly more time per day actively using
the EHR and performing four clinical activities: notes, orders, in-basket messages,
and clinical review, a reflection of this documentation burden.[10 ]
[11 ]
Time spent on EHR has been identified as an important unit of measure of attenuated
efficiency in use of health care provider clinical time, which is a high-cost and
limited resource and may be associated with reduced patient satisfaction.[12 ]
[13 ] Time motion studies and EHR log records have indicated an increase in time allocated
for desktop medicine with physicians spending nearly 2 hours in the EHR and on other
desk work for every hour of direct patient care.[11 ]
[14 ]
[15 ]
[16 ]
[17 ]
[18 ] This perhaps contributes to overflow of EHR work into the physician's nonclinical
and personal time, negatively affecting work–life balance.[19 ]
There is accumulating evidence indicating the association of EHR use after work hours
with occupational distress including burnout.[20 ]
[21 ]
[22 ]
[23 ]
[24 ] These studies are based on either physician perception of time spent in EHR through
surveys which may be prone to bias or by utilizing vendor-defined EHR use measures
which often rely on proprietary algorithms that may not take into account variation
in physician's schedules which may underestimate time spent on the EHR outside of
scheduled clinic time.[12 ]
[25 ] A variety of novel EHR metrics have been proposed to evaluate the impact of the
EHR on physician experience.[12 ] Our team developed and refined a nonproprietary EHR use algorithm to track the number
of hours a physician spends logged into the EHR and calculates the Clinician Logged-in
Outside Clinic (CLOC) time, the number of hours spent by a physician on the EHR outside
of allocated time for patient care. An earlier iteration of CLOC time differed by
including only evening, weekend, and vacation time and discounted “shoulder” time
before and after scheduled time. Even so, this metric was noted to be significantly
correlated with self-reported time and was also significantly associated with satisfaction
with workload in the EHR and satisfaction with subjective report of amount of time
spent in the EHR after clinic hours.[23 ]
Objective
The objective of our study was to measure the association between CLOC metrics and
validated measures of physician burnout and professional fulfillment.
Methods
Study Population
Stanford Health Care (SHC) is a large academic adult health care organization with
an academic teaching hospital, based in Palo Alto, California, United States, and
over 70 academic community-based ambulatory practices across Northern California called
the University Healthcare Alliance (UHA). Physicians from adult outpatient Internal
Medicine, Neurology, Dermatology, Hematology, Oncology, Rheumatology, and Endocrinology
departments were eligible for inclusion in the present study as they are primarily
ambulatory and nonproceduralist specialties, capturing a homogenous group of physicians.
Those physicians in these specialty disciplines who had active status in EHR, logged
more than 8 hours of scheduled clinic time per week, and answered the annual wellness
survey administered in Spring 2019 were included in the analysis.
CLOC Metrics
The algorithm tracks the amount of time that a physician is logged into the EHR. Allocated
appointment time was defined as time specified for patient care including scheduled
and add-on appointment slots (total appointment time allocated to all patients). All
time logged into the EHR system outside of allocated appointment time is considered
CLOC time including time spent in the EHR immediately before and after a physician's
first and last allocated patient slot, weekends, vacations, canceled schedule for
meetings or conferences, and weekdays without clinical commitment ([Fig. 1 ] and Appendix A ). CLOC metrics take into account inactive time, i.e., idle time without mouse clicks
and keyboard strokes. It has been suggested that both active and idle times may impact
the physician and the quality of their time at home.[12 ] Similarly, inactive time may represent review of notes/laboratory values without
keyboard or mouse activity or interruptions at work and may contribute to longer time
to task completion and add to frustration. In our study, physicians log out from the
EHR manually or after workstation inactivity of 20 minutes. The CLOC ratio was defined
as the ratio of CLOC time to allocated appointment time.
Fig. 1 Summary of time metrics. *CLOC time: time spent on the EHR outside of allocated appointment
time. CLOC, Clinician Logged-in Outside Clinic; EHR, electronic health record.
Calculations were done for each week from January 1 through March 31, 2019—the 3 months
(13 weeks) period immediately before the survey administration began. Face and content
validity was established through end-user testing and feedback from the analytics
team as recommended by Sinsky et al.[12 ] A study utilizing a prior iteration of the CLOC metric noted that CLOC time was
correlated with self-reported time spent in the EHR after clinic hours, establishing
construct validity.[23 ] Criterion validity of the metric was established based on prior literature suggesting
associations between time spent on EHR after hours and satisfaction with amount of
time spent on EHR and burnout.[22 ]
[23 ]
[24 ]
Survey Data
As previously described, data were collected from the 2019 Stanford physician wellness
surveys, administered in April 2019, which incorporates the Professional Fulfillment
Index (PFI) instrument to measure physician well-being including burnout and professional
fulfillment.[26 ] This instrument was developed by national experts in physician well-being and these
scoring procedures have been validated against other burnout and patient safety metrics
(Appendix B ).[27 ]
Each PFI item was scored using the associated 5-point Likert scale (range: 0–4). The
work exhaustion and interpersonal disengagement domain scores were calculated by averaging
the item scores of all items within each corresponding domain (range: 0–4). Burnout
and professional fulfillment scale scores were calculated by averaging the item scores
of all items within each corresponding scale (range: 0 to 4). These scores were later
standardized to 0–10 for ease of interpretation (Appendix B ).[27 ]
Data Linkage
The survey and the EHR use datasets were provided to an independent, institutionally
approved third party administrator, SullivanLuallin Group (San Diego, California),
without access to personnel or other employment records. The third-party administrator
then paired the EHR data with survey data and subsequently removed all identifying
information before returning an anonymous dataset set to our team for analysis. The
study statistician did not have the ability to identify any specific physician or
division/work unit through their analysis. All aspects of the study were reviewed
by the Stanford University Institutional Review Board and deemed exempt because they
involved retrospective analysis of administratively collected data using a completely
anonymized dataset.
Statistical Methods
Pearson's correlation and multiple linear regression controlling for specialty were
used to explore the relationship between CLOC metrics and wellness scores. Observations
with missing data were removed from the analysis. All p -values were two-sided, and p <0.05 was considered significant. Statistical analyses were performed using R statistical
software (version 3.6.0; R Foundation for Statistical Computing, Vienna, Austria).
Results
The wellness survey response rate for academic hospital-based ambulatory care physicians
(at SHC) was 66.8% and for academic community-based practice physicians (at UHA) was
97.2%. A total of 621 academic hospital-based ambulatory care physicians (at SHC)
and 235 academic community-based practice physicians (at UHA) had available survey
and CLOC metrics data. Of these, 97 and 108 physicians at SHC and UHA respectively
belonged to our selected specialties, had active status in EHR and logged more than
8 hours of scheduled clinic time per week, and were included in the analysis. Sample
demographics are provided in [Table 1 ]. The majority of physicians practiced general Internal Medicine. While 38.5% physicians
had high professional fulfillment (a professional fulfillment score of 7.5 or greater
on a scale of 0–10), 32.2% had symptoms of burnout (score of 3.325 or greater on a
scale of 0–10). On average, physicians spent an average of 12.3 (standard deviation
[SD]: 8.14) CLOC hours per week with an average of 1.83 (SD: 2.73) CLOC hours per
week on weekends. The average CLOC ratio was 0.71 (SD: 0.49) indicating that for every
1 hour of allocated appointment time physicians spent an additional 43 minutes logged
in to the EHR.
Table 1
Summary statistics
Sample demographics
Provider specialty
All (n = 205), n (%)
SHC (n = 97), n (%)
UHA (n = 108), n (%)
Dermatology
17 (8.29%)
14 (14.43%)
3 (2.78%)
Endocrinology
10 (4.88%)
4 (4.12%)
6 (5.56%)
General Internal Medicine
138 (67.32%)
46 (47.42%)
92 (85.19%)
Hematology and Oncology
9 (4.39%)
9 (9.28%)
0 (0%)
Neurology
21 (10.24%)
20 (20.62%)
1 (0.93%)
Rheumatology
10 (4.88%)
4 (4.12%)
6 (5.56%)
Summary of wellness scores
Variables
Mean (SD)
Burnout (range 0–10)
2.79 (2.07)
Work exhaustion (range 0–10)
3.65 (2.37)
Interpersonal disengagement (range 0–10)
2.21 (2.10)
Professional fulfillment (range 0–10)
6.55 (2.05)
Summary of CLOC metrics
Variables
Mean (SD)
Total CLOC time[a ] (hours per week)
12.35 (8.14)
CLOC ratio[b ]
0.71 (0.49)
CLOC time weekends (hours per week)[c ]
1.83 (2.73)
Summary of other metrics
Variables
Mean (SD)
Allocated appointment time[d ] (hours per week)
22.59 (8.38)
Scheduled patient time (hours per week)
19.36 (7.04)
Add-on patient time (hours per week)
0.08 (0.19)
With patient time (hours per week)
19.43 (7.04)
Without patient time (hours per week)
3.15 (2.37)
No show time (hours per week)
0.89 (0.58)
Canceled time (hours per week)
0.00 (0.01)
Left without seen time (hours per week)
0.05 (0.08)
Abbreviations: CLOC, Clinician Logged-in Outside Clinic; SD, standard deviation; SHC,
Stanford Health Care; UHA, University Healthcare Alliance.
a Total CLOC time: time spent on the EHR outside of allocated appointment time.
b CLOC ratio: ratio of CLOC time to allocated appointment time.
c CLOC time weekends: time spent on EHR outside of allocated appointment time on weekends.
d Allocated appointment time: time specified for patient care including scheduled and
add-on appointment slots.
[Table 2 ] reports the relationships between CLOC metrics and burnout, the two domains of burnout,
i.e., work exhaustion and interpersonal disengagement, and professional fulfillment.
A larger CLOC ratio was significantly associated with higher work exhaustion (Pearson's
r = 0.14; p = 0.04). We conducted a subanalysis for the largest specialty group in our study,
Internal Medicine, and found no significant associations between CLOC time and CLOC
ratio and wellness measures ([Table 3 ]). Regression analysis adjusting for specialty did not find statistically significant
association between CLOC time or CLOC ratio and wellness measures.
Table 2
Correlations for CLOC metrics and wellness measures
Variables
Burnout
(rho)
Work exhaustion (rho)
Interpersonal disengagement
(rho)
Professional fulfillment
(rho)
Total CLOC time[a ]
0.01
0.07
−0.03
0.08
CLOC ratio[b ]
0.10
0.14[c ]
0.06
0.02
Abbreviation: CLOC, Clinician Logged-in Outside Clinic.
a Total CLOC time: time spent on the EHR outside of allocated appointment time.
b CLOC ratio: ratio of CLOC time to allocated appointment time.
c
p < 0.05.
Table 3
Correlations for CLOC metrics and wellness measures for the internal medicine group
Variables
Burnout
(rho)
Work exhaustion
(rho)
Interpersonal disengagement
(rho)
Professional fulfillment
(rho)
Total CLOC time[a ]
0.05
0.08
0.02
0.06
CLOC ratio[b ]
0.06
0.07
0.04
0.03
Abbreviation: CLOC, Clinician Logged-in Outside Clinic.
a Total CLOC time: time spent on the EHR outside of allocated appointment time.
b CLOC ratio: ratio of CLOC time to allocated appointment time.
We conducted exploratory analysis looking at the time of day and week (e.g., 6–9 p.m.,
9 p.m. to 12 a.m.; 12 a.m. to 3 a.m.; 3 a.m. to 6 a.m., weekend time) the CLOC time
was recorded; with the exception of a counterintuitive association between CLOC time
on weekends and professional fulfillment that was opposite of hypothesized direction;
no other associations were identified.
Discussion
Ours is the first study to examine the relationship between a nonproprietary, transparent
EHR use duration metric developed with clinical insight and validation and validated
measures of burnout and professional fulfillment. Although it is challenging to determine
an ideal number of CLOC hours given individual variation in EHR use patterns, documentation
preferences,[28 ] and schedules, our study establishes an association between CLOC ratio and work
exhaustion. In general, this would suggest that a lower CLOC ratio, ratio of total
CLOC time to allocated appointment time, is more favorable which could suggest that
efforts to reduce the CLOC ratio may be worthwhile and that it could be used as an
improvement metric for efforts to tackle work exhaustion. Our finding of CLOC ratio
being significantly correlated with work exhaustion is consistent with association
in recent literature.[20 ] However, the magnitude of effect of this correlation was small, which may suggest
that other aspects of EHR use may be more likely to be drivers of physician burnout
than time spent on the EHR alone.[29 ]
[30 ]
[31 ]
[32 ] Moreover, the evolving body of evidence suggests that the impact of EHRs, while
being an important source of frustration that needs to be addressed, does not appear
to be a dominant factor driving the high rates of occupational burnout in physicians.[20 ]
[30 ]
[33 ]
[34 ]
We observed a statistically significant correlation between CLOC ratio and the work
exhaustion domain of burnout, but not the interpersonal disengagement domain of burnout
or professional fulfillment. Our results indicate that the physicians who spend more
time on the EHR outside of allocated appointment time are also more exhausted at work,
physically and emotionally. It may very well be that physicians who are exhausted
at work due to other factors spend more time on the EHR outside allocated appointment
time.
Interpersonal disengagement specifically assesses empathy and connectedness with others,
particularly patients and colleagues.[27 ] Our finding that the physicians with larger CLOC ratios are not disengaged or burned
out despite being exhausted may be reflective of well-established personal resources
aiding self-efficacy.[35 ]
[36 ] Since Stanford is an academic medical center, these findings may also be a result
of greater variation in work tasks (e.g., patient care, education, research, administrative
leadership, etc.) and dedicated time for documentation available to academic physicians
and the support resources available. Moreover, substantial variation exists in timing
of EHR use among individual providers, more so in the academic setting, to the effect
that some providers prefer documenting outside allocated appointment time during available
administrative or research slots during daytime ([Table 4 ]). We noted that CLOC time during daytime (8 a.m. to 6 p.m.) was greater than CLOC
time during after-hours (6 p.m. to 8 a.m.) across all specialties, emphasizing the
importance of capturing these times as work outside work by acknowledging variation
in physician schedules especially in an academic setting ([Table 4 ]). Majority of the work being done between the hours of 8 a.m. and 6 p.m. may have
contributed to the lack of significant correlation between CLOC ratio and interpersonal
disengagement and burnout. This finding may support incorporating more dedicated time
for documentation into physicians' schedules. Exhaustion, however, has been shown
to predict future disengagement, which is cause for concern.[37 ] Further studies are needed to assess the longitudinal relationship between CLOC
ratio, work exhaustion, interpersonal disengagement, and overall burnout.
Table 4
Correlations for allocated appointment time and CLOC time across specialties
Dermatology
Endocrinology
General internal medicine
Hematology and oncology
Neurology
Rheumatology
N
17
10
138
9
21
10
Allocated appointment time[a ]
(mean/SD) (h/wk)
16.46 (5.23)
22.85 (8.47)
24.39 (8.58)
16.25 (4.25)
18.89 (5.61)
21.32 (8.38)
Total CLOC time[b ]
(mean (SD))
(h/wk)
11.94 (6.49)
14.41 (13.26)
12.25 (8.41)
11.05 (6.58)
12.30 (5.55)
13.67 (7.69)
CLOC ratio[c ]
(mean (SD))
0.98 (0.65)
0.76 (0.60)
0.65 (0.48)
0.92 (0.51)
0.79 (0.34)
0.75 (0.38)
CLOC time 8 a.m. to 6 p.m. M-F (mean/SD)
(h/wk)
7.47 (2.59)
7.33 (4.01)
7.04 (3.72)
6.92 (3.85)
9.17 (3.68)
6.55 (2.40)
CLOC time 6 p.m. to 8 a.m. M-F
(mean/SD)
(h/wk)
2.68 (2.98)
4.03 (4.24)
3.34 (3.85)
2.66 (2.61)
2.17 (1.94)
4.87 (4.49)
CLOC Time weekends (mean/SD)
(h/wk)
1.78 (2.43)
3.04 (6.18)
1.87 (2.62)
1.47 (1.30)
0.95 (1.43)
2.52 (2.43)
Abbreviations: CLOC, Clinician Logged-in Outside Clinic; SD, standard deviation.
a
p < 0.001.
b Total CLOC time: time spent on the EHR outside of allocated appointment time.
c CLOC ratio: ratio of CLOC time to allocated appointment time.
Our results are in alignment with a recent study that reported that clinicians (physicians
and nurse practitioners) in the top two quartiles of vendor-defined EHR time after
hours on scheduled clinic days had a significantly greater odds of high exhaustion.[20 ] On the other hand, Adler-Milstein et al did not find any significant associations
between minutes active on unscheduled days per clinical full-time equivalent and either
cynicism or emotional exhaustion. In contrast to common vendor-defined use measures,
the CLOC metrics fulfill many of the recommendations around use of EHR use measures
toward research as recommended by Sinsky et al.[12 ]
A feature of the CLOC metrics is that it takes into consideration variation in physicians'
schedules. For instance, a physician may choose to work certain hours of the day,
e.g., 7 a.m. to 2 p.m., as it better suits their work–life balance. In this case,
any work done on the EHR after 2 p.m. would be considered as work outside work (WOW)
by the CLOC metric. On the other hand, most vendor-defined work outside work metrics
may consider time spent on EHR after a certain time, e.g., 7 p.m., as WOW, which may
underestimate work outside of work. Another instance of variability in physician schedules
could be seen when a physician has long time intervals between consecutive appointment
slots (e.g., schedule split between morning and afternoon sessions) which they may
use to complete charting. The CLOC metrics capture time spent on the EHR during these
periods between appointments as WOW, whereas vendor-defined WOW metrics may miss this
work done on the EHR in between appointments which we believe also contributes to
EHR burden. We noted that CLOC time during daytime (8 a.m. to 6 p.m.) was greater
than CLOC time during after-hours (6 p.m. to 8 a.m.) across all specialties, emphasizing
the importance of capturing these times as work outside work by acknowledging variation
in physician schedules especially in an academic setting ([Table 4 ]). CLOC metrics also consider work done during shoulder time (the period of time,
usually around 30 minutes, immediately before and after a physician's first and last
allocated patient slot) as work outside work. As an example, a physician with 3 hours
of allocated appointment time per day will have a relatively longer shoulder time
than a physician with 8 hours of allocated appointment time. Any time spent on the
EHR during shoulder time for a clinician with a smaller clinic volume would be considered
significant. By including time spent on the EHR during shoulder time, CLOC metrics
capture this EHR work outside work which may be missed in some vendor-defined metrics
leading to underestimation of work outside of work. Moreover, CLOC metrics include
inactive EHR time as it has been suggested that both active and idle times may impact
the physician and the quality of their time at home.[12 ] Similarly, inactive time may represent review of notes/laboratory values without
keyboard or mouse activity or interruptions at work and may contribute to longer time
to task completion and add to frustration.
Our finding of allocated appointment time having a significant positive correlation
with professional fulfillment and a significant inverse correlation with burnout,
work exhaustion, and interpersonal disengagement is unexpected as independent relationship
between work hours and burnout has been described in the literature ([Table 5 ]).[38 ] Our results could be interpreted as having more time dedicated to clinical work
in an academic medical center being protective against work exhaustion, interpersonal
disengagement, and overall burnout as has been suggested in previous studies with
statements such as “seeing the patient is the joy.”[39 ]
[40 ] This could also be explained by the reasoning that physicians who enjoy practicing
take on higher clinic volumes.
Table 5
Correlations for other metrics of interest and wellness measures
Variables
Burnout
(rho)
Work exhaustion (rho)
Interpersonal disengagement
(rho)
Professional fulfillment (rho)
Allocated appointment time
−0.19[a ]
−0.15[a ]
−0.19[a ]
0.17[a ]
Scheduled patient time
−0.19[a ]
−0.15[a ]
−0.20[a ]
0.18[a ]
With patient time
−0.19[a ]
−0.15[a ]
−0.20[a ]
0.18[a ]
a p < 0.05.
Our study has inherent limitations. Although the study used a chronologic approach
that evaluated nonproprietary CLOC metrics over the 3 months immediately prior to
assessment of burnout and professional fulfillment, causality for specific dimensions
cannot be established. The study included physicians affiliated with a single clinical
organization which creates a sample population that may be different from other institutions
owing to differences in structure of administration and service, although the inclusion
of academic and community-based practices may increase generalizability of our findings.
Since we extracted burnout and professional fulfillment scores from surveys, nonrespondent
bias is an inherent limitation of our study. CLOC metrics may overestimate WOW as
they include inactive EHR time and time spent interacting with other applications
simultaneously while active on the EHR. CLOC metrics include inactive time as opposed
to vendor-provided metrics which may discount inactive time (the threshold of inactivity
used by Epic for calculations of vendor-defined metrics at our institution is 5 seconds).
We believe that inactive time may represent review of notes/laboratory values without
keyboard or mouse activity or interruptions at work/home and may contribute to longer
time to task completion and add to frustration as supported by the recommendations
for core EHR use metrics by Sinsky et al.[12 ] CLOC metric also does not account for hours spent on inpatient service and other
nonclinical work in the EHR (quality improvement or other administrative work, chart
review for research, etc.).
Conclusion
Our study emphasizes the need to measure, report, and compare work outside work hours
at scale. We introduce the CLOC metrics, some of which, like vendor-defined metrics,
appear to be objective EHR activity-based markers of work exhaustion, but not professional
fulfillment or burnout. This is consistent with emerging data that show that if there
is an association between markers of EHR use time with burnout or professional fulfillment,
it appears to be a small effect. The CLOC metrics also have the potential added benefit
of transparency in calculation over proprietary metrics supporting broader implementation,
interpretation, and longitudinal analysis.[41 ] CLOC may potentially open options for comparative research involving metrics associated
with EHR use duration data and physician wellness both within and across institutions
and EHRs.
Clinical Relevance Statement
Clinical Relevance Statement
The CLOC metrics are potential objective EHR activity-based markers associated with
physician work exhaustion. Institutions may be able to leverage CLOC metrics for comparative
and longitudinal research involving metrics associated with EHR use duration data
and physician wellness. Targeting interventions at the individual level for providers
with higher CLOC ratio may be beneficial in decreasing work exhaustion.
Multiple Choice Questions
Multiple Choice Questions
Our study suggests that CLOC ratio, ratio of CLOC time (time spent on EHR outside
of allocated appointment time) to allocated appointment time in the outpatient setting,
is correlated with physician:
Correct Answer: The correct answer is option a. Work exhaustion is the correct choice. Although it
is challenging to determine an ideal number of CLOC hours given individual variation
in EHR use patterns, documentation preferences, and schedules, our study establishes
an association between CLOC ratio and work exhaustion. Our finding is consistent with
association in recent literature. In general, this would suggest a lower CLOC ratio
is more favorable, which could suggest that efforts to reduce the CLOC ratio may be
worthwhile and that it could be used as an improvement metric for efforts to tackle
work exhaustion.
Clinician Logged-in Outside Clinic (CLOC) metrics take into account:
Correct Answer: The correct answer is option d. All of the above is the correct choice. In contrast
to common vendor-defined use measures, the CLOC metrics fulfill many of the recommendations
around use of EHR use measures toward research as recommended by Sinsky et al.[12 ] It takes into consideration variations in physician's schedules, takes into account
inactive time, i.e., idle time without mouse clicks and keyboard strokes, and includes
work done immediately before and after a physician's first and last allocated patient
slot (i.e., “shoulder time”), which prevents underestimation of work outside of work.
Appendix A Definitions of Clinician Logged-In Outside Clinic (CLOC) Time and Other
Metrics
Allocated appointment time: time specified for patient care including scheduled and
add-on appointment slots, defined as below (total appointment time allocated to all
patients). All hours of a physician's schedule outside of scheduled appointment slots
was defined as unavailable. Allocated appointment time only captures times for those
providers who have slots under their name.
Total CLOC time: time spent logged-in to the EHR outside of allocated appointment
time either physically inside or outside the clinic and includes time spent in the
EHR immediately before and after a physician's first and last allocated patient slot,
time in between patient slots, weekends, vacations, canceled schedule for meetings
or conferences, and weekdays without clinical commitment.
CLOC time weekends: time spent on EHR outside of allocated appointment time on weekends.
CLOC ratio: ratio of total CLOC time to allocated appointment time.
Scheduled patient time: time specified for patient care when a patient was scheduled
for an appointment (includes with and without patient time, defined as below).
Add-on patient time: time specified for patient care when a patient was seen during
a slot marked as unavailable (includes with patient time, defined as below).
With patient time: time specified for patient care when a patient was seen during
a scheduled or add-on appointment.
Without patient time: time specified for patient care when a patient was scheduled
for an appointment but not seen including no show, canceled, and left without seen
time (defined as below).
No show time: time specified for patient care when the patient did not show up.
Canceled time: time specified for patient care when the patient canceled and did not
reschedule.
Left without seen time: time specified for patient care when a patient checked in
but left before being seen.
Appendix B Survey Prompts Pertaining to Burnout and Professional Fulfillment
Copyright 2016 Board of Trustees of the Leland Stanford Junior University. All rights
reserved. Nonprofit organizations are permitted to use this survey instrument without
modification for research or program evaluation exclusively. An electronic version
of the survey is available by contacting wellmdcenter@stanford.edu. Any other use
of this survey is granted by express written permission of the Stanford WellMD Center
by contacting wellmdcenter@stanford.edu.
