Novel Nonproprietary Measures of Ambulatory Electronic Health Record Use Associated with Physician Work Exhaustion

Abstract

Background Accumulating evidence indicates an association between physician electronic health record (EHR) use after work hours and occupational distress including burnout. These studies are based on either physician perception of time spent in EHR through surveys which may be prone to bias or by utilizing vendor-defined EHR use measures which often rely on proprietary algorithms that may not take into account variation in physician's schedules which may underestimate time spent on the EHR outside of scheduled clinic time. The Stanford team developed and refined a nonproprietary EHR use algorithm to track the number of hours a physician spends logged into the EHR and calculates the Clinician Logged-in Outside Clinic (CLOC) time, the number of hours spent by a physician on the EHR outside of allocated time for patient care.

Objective The objective of our study was to measure the association between CLOC metrics and validated measures of physician burnout and professional fulfillment.

Methods Physicians from adult outpatient Internal Medicine, Neurology, Dermatology, Hematology, Oncology, Rheumatology, and Endocrinology departments who logged more than 8 hours of scheduled clinic time per week and answered the annual wellness survey administered in Spring 2019 were included in the analysis.

Results We observed a statistically significant positive correlation between CLOC ratio (defined as the ratio of CLOC time to allocated time for patient care) and work exhaustion (Pearson's r = 0.14; p = 0.04), but not interpersonal disengagement, burnout, or professional fulfillment.

Conclusion The CLOC metrics are potential objective EHR activity-based markers associated with physician work exhaustion. Our results suggest that the impact of time spent on EHR, while associated with exhaustion, does not appear to be a dominant factor driving the high rates of occupational burnout in physicians.

Keywords

electronic health records - ambulatory care - workload

Background and Significance

Office-based physician adoption of electronic health records (EHRs) has more than quadrupled in the past 15 years with the establishment of the Meaningful Use Program. As of 2017, 86% of office-based physicians had adopted an EHR.[1] [2] The digitization of EHRs, as it was intended, may have led to advances in medical practice such as potential for reduction in hospital mortality and potentially preventable hospital admissions.[3] [4] However, the use of clinical documentation to satisfy demands peripheral to direct patient care such as to meet billing, quality, and compliance requirements is a potential source of frustration among physicians, an unintended consequence of EHR adoption.[5] [6] [7] [8] [9] Clinicians in the United States spend significantly more time per day actively using the EHR and performing four clinical activities: notes, orders, in-basket messages, and clinical review, a reflection of this documentation burden.[10] [11]

Time spent on EHR has been identified as an important unit of measure of attenuated efficiency in use of health care provider clinical time, which is a high-cost and limited resource and may be associated with reduced patient satisfaction.[12] [13] Time motion studies and EHR log records have indicated an increase in time allocated for desktop medicine with physicians spending nearly 2 hours in the EHR and on other desk work for every hour of direct patient care.[11] [14] [15] [16] [17] [18] This perhaps contributes to overflow of EHR work into the physician's nonclinical and personal time, negatively affecting work–life balance.[19]

There is accumulating evidence indicating the association of EHR use after work hours with occupational distress including burnout.[20] [21] [22] [23] [24] These studies are based on either physician perception of time spent in EHR through surveys which may be prone to bias or by utilizing vendor-defined EHR use measures which often rely on proprietary algorithms that may not take into account variation in physician's schedules which may underestimate time spent on the EHR outside of scheduled clinic time.[12] [25] A variety of novel EHR metrics have been proposed to evaluate the impact of the EHR on physician experience.[12] Our team developed and refined a nonproprietary EHR use algorithm to track the number of hours a physician spends logged into the EHR and calculates the Clinician Logged-in Outside Clinic (CLOC) time, the number of hours spent by a physician on the EHR outside of allocated time for patient care. An earlier iteration of CLOC time differed by including only evening, weekend, and vacation time and discounted “shoulder” time before and after scheduled time. Even so, this metric was noted to be significantly correlated with self-reported time and was also significantly associated with satisfaction with workload in the EHR and satisfaction with subjective report of amount of time spent in the EHR after clinic hours.[23]

Objective

The objective of our study was to measure the association between CLOC metrics and validated measures of physician burnout and professional fulfillment.

Methods

Study Population

Stanford Health Care (SHC) is a large academic adult health care organization with an academic teaching hospital, based in Palo Alto, California, United States, and over 70 academic community-based ambulatory practices across Northern California called the University Healthcare Alliance (UHA). Physicians from adult outpatient Internal Medicine, Neurology, Dermatology, Hematology, Oncology, Rheumatology, and Endocrinology departments were eligible for inclusion in the present study as they are primarily ambulatory and nonproceduralist specialties, capturing a homogenous group of physicians. Those physicians in these specialty disciplines who had active status in EHR, logged more than 8 hours of scheduled clinic time per week, and answered the annual wellness survey administered in Spring 2019 were included in the analysis.

CLOC Metrics

The algorithm tracks the amount of time that a physician is logged into the EHR. Allocated appointment time was defined as time specified for patient care including scheduled and add-on appointment slots (total appointment time allocated to all patients). All time logged into the EHR system outside of allocated appointment time is considered CLOC time including time spent in the EHR immediately before and after a physician's first and last allocated patient slot, weekends, vacations, canceled schedule for meetings or conferences, and weekdays without clinical commitment ([Fig. 1] and Appendix A). CLOC metrics take into account inactive time, i.e., idle time without mouse clicks and keyboard strokes. It has been suggested that both active and idle times may impact the physician and the quality of their time at home.[12] Similarly, inactive time may represent review of notes/laboratory values without keyboard or mouse activity or interruptions at work and may contribute to longer time to task completion and add to frustration. In our study, physicians log out from the EHR manually or after workstation inactivity of 20 minutes. The CLOC ratio was defined as the ratio of CLOC time to allocated appointment time.

Fig. 1 Summary of time metrics. *CLOC time: time spent on the EHR outside of allocated appointment time. CLOC, Clinician Logged-in Outside Clinic; EHR, electronic health record.

Calculations were done for each week from January 1 through March 31, 2019—the 3 months (13 weeks) period immediately before the survey administration began. Face and content validity was established through end-user testing and feedback from the analytics team as recommended by Sinsky et al.[12] A study utilizing a prior iteration of the CLOC metric noted that CLOC time was correlated with self-reported time spent in the EHR after clinic hours, establishing construct validity.[23] Criterion validity of the metric was established based on prior literature suggesting associations between time spent on EHR after hours and satisfaction with amount of time spent on EHR and burnout.[22] [23] [24]

Survey Data

As previously described, data were collected from the 2019 Stanford physician wellness surveys, administered in April 2019, which incorporates the Professional Fulfillment Index (PFI) instrument to measure physician well-being including burnout and professional fulfillment.[26] This instrument was developed by national experts in physician well-being and these scoring procedures have been validated against other burnout and patient safety metrics (Appendix B).[27]

Each PFI item was scored using the associated 5-point Likert scale (range: 0–4). The work exhaustion and interpersonal disengagement domain scores were calculated by averaging the item scores of all items within each corresponding domain (range: 0–4). Burnout and professional fulfillment scale scores were calculated by averaging the item scores of all items within each corresponding scale (range: 0 to 4). These scores were later standardized to 0–10 for ease of interpretation (Appendix B).[27]

Data Linkage

The survey and the EHR use datasets were provided to an independent, institutionally approved third party administrator, SullivanLuallin Group (San Diego, California), without access to personnel or other employment records. The third-party administrator then paired the EHR data with survey data and subsequently removed all identifying information before returning an anonymous dataset set to our team for analysis. The study statistician did not have the ability to identify any specific physician or division/work unit through their analysis. All aspects of the study were reviewed by the Stanford University Institutional Review Board and deemed exempt because they involved retrospective analysis of administratively collected data using a completely anonymized dataset.

Statistical Methods

Pearson's correlation and multiple linear regression controlling for specialty were used to explore the relationship between CLOC metrics and wellness scores. Observations with missing data were removed from the analysis. All p-values were two-sided, and p <0.05 was considered significant. Statistical analyses were performed using R statistical software (version 3.6.0; R Foundation for Statistical Computing, Vienna, Austria).

Results

The wellness survey response rate for academic hospital-based ambulatory care physicians (at SHC) was 66.8% and for academic community-based practice physicians (at UHA) was 97.2%. A total of 621 academic hospital-based ambulatory care physicians (at SHC) and 235 academic community-based practice physicians (at UHA) had available survey and CLOC metrics data. Of these, 97 and 108 physicians at SHC and UHA respectively belonged to our selected specialties, had active status in EHR and logged more than 8 hours of scheduled clinic time per week, and were included in the analysis. Sample demographics are provided in [Table 1]. The majority of physicians practiced general Internal Medicine. While 38.5% physicians had high professional fulfillment (a professional fulfillment score of 7.5 or greater on a scale of 0–10), 32.2% had symptoms of burnout (score of 3.325 or greater on a scale of 0–10). On average, physicians spent an average of 12.3 (standard deviation [SD]: 8.14) CLOC hours per week with an average of 1.83 (SD: 2.73) CLOC hours per week on weekends. The average CLOC ratio was 0.71 (SD: 0.49) indicating that for every 1 hour of allocated appointment time physicians spent an additional 43 minutes logged in to the EHR.

Table 1 Summary statistics

Sample demographics
Provider specialty	All (n = 205), n (%)	SHC (n = 97), n (%)	UHA (n = 108), n (%)
Dermatology	17 (8.29%)	14 (14.43%)	3 (2.78%)
Endocrinology	10 (4.88%)	4 (4.12%)	6 (5.56%)
General Internal Medicine	138 (67.32%)	46 (47.42%)	92 (85.19%)
Hematology and Oncology	9 (4.39%)	9 (9.28%)	0 (0%)
Neurology	21 (10.24%)	20 (20.62%)	1 (0.93%)
Rheumatology	10 (4.88%)	4 (4.12%)	6 (5.56%)
Summary of wellness scores
Variables		Mean (SD)
Burnout (range 0–10)		2.79 (2.07)
Work exhaustion (range 0–10)		3.65 (2.37)
Interpersonal disengagement (range 0–10)		2.21 (2.10)
Professional fulfillment (range 0–10)		6.55 (2.05)
Summary of CLOC metrics
Variables		Mean (SD)
Total CLOC time[a] (hours per week)		12.35 (8.14)
CLOC ratio[b]		0.71 (0.49)
CLOC time weekends (hours per week)[c]		1.83 (2.73)
Summary of other metrics
Variables		Mean (SD)
Allocated appointment time[d] (hours per week)		22.59 (8.38)
Scheduled patient time (hours per week)		19.36 (7.04)
Add-on patient time (hours per week)		0.08 (0.19)
With patient time (hours per week)		19.43 (7.04)
Without patient time (hours per week)		3.15 (2.37)
No show time (hours per week)		0.89 (0.58)
Canceled time (hours per week)		0.00 (0.01)
Left without seen time (hours per week)		0.05 (0.08)

Abbreviations: CLOC, Clinician Logged-in Outside Clinic; SD, standard deviation; SHC, Stanford Health Care; UHA, University Healthcare Alliance.

^a Total CLOC time: time spent on the EHR outside of allocated appointment time.

^b CLOC ratio: ratio of CLOC time to allocated appointment time.

^c CLOC time weekends: time spent on EHR outside of allocated appointment time on weekends.

^d Allocated appointment time: time specified for patient care including scheduled and add-on appointment slots.

[Table 2] reports the relationships between CLOC metrics and burnout, the two domains of burnout, i.e., work exhaustion and interpersonal disengagement, and professional fulfillment. A larger CLOC ratio was significantly associated with higher work exhaustion (Pearson's r = 0.14; p = 0.04). We conducted a subanalysis for the largest specialty group in our study, Internal Medicine, and found no significant associations between CLOC time and CLOC ratio and wellness measures ([Table 3]). Regression analysis adjusting for specialty did not find statistically significant association between CLOC time or CLOC ratio and wellness measures.

Table 2 Correlations for CLOC metrics and wellness measures

Variables	Burnout (rho)	Work exhaustion (rho)	Interpersonal disengagement (rho)	Professional fulfillment (rho)
Total CLOC time[a]	0.01	0.07	−0.03	0.08
CLOC ratio[b]	0.10	0.14[c]	0.06	0.02

Abbreviation: CLOC, Clinician Logged-in Outside Clinic.

^a Total CLOC time: time spent on the EHR outside of allocated appointment time.

^b CLOC ratio: ratio of CLOC time to allocated appointment time.

^c p < 0.05.

Table 3 Correlations for CLOC metrics and wellness measures for the internal medicine group

Variables	Burnout (rho)	Work exhaustion (rho)	Interpersonal disengagement (rho)	Professional fulfillment (rho)
Total CLOC time[a]	0.05	0.08	0.02	0.06
CLOC ratio[b]	0.06	0.07	0.04	0.03

Abbreviation: CLOC, Clinician Logged-in Outside Clinic.

^a Total CLOC time: time spent on the EHR outside of allocated appointment time.

^b CLOC ratio: ratio of CLOC time to allocated appointment time.

We conducted exploratory analysis looking at the time of day and week (e.g., 6–9 p.m., 9 p.m. to 12 a.m.; 12 a.m. to 3 a.m.; 3 a.m. to 6 a.m., weekend time) the CLOC time was recorded; with the exception of a counterintuitive association between CLOC time on weekends and professional fulfillment that was opposite of hypothesized direction; no other associations were identified.

Discussion

Ours is the first study to examine the relationship between a nonproprietary, transparent EHR use duration metric developed with clinical insight and validation and validated measures of burnout and professional fulfillment. Although it is challenging to determine an ideal number of CLOC hours given individual variation in EHR use patterns, documentation preferences,[28] and schedules, our study establishes an association between CLOC ratio and work exhaustion. In general, this would suggest that a lower CLOC ratio, ratio of total CLOC time to allocated appointment time, is more favorable which could suggest that efforts to reduce the CLOC ratio may be worthwhile and that it could be used as an improvement metric for efforts to tackle work exhaustion. Our finding of CLOC ratio being significantly correlated with work exhaustion is consistent with association in recent literature.[20] However, the magnitude of effect of this correlation was small, which may suggest that other aspects of EHR use may be more likely to be drivers of physician burnout than time spent on the EHR alone.[29] [30] [31] [32] Moreover, the evolving body of evidence suggests that the impact of EHRs, while being an important source of frustration that needs to be addressed, does not appear to be a dominant factor driving the high rates of occupational burnout in physicians.[20] [30] [33] [34]

We observed a statistically significant correlation between CLOC ratio and the work exhaustion domain of burnout, but not the interpersonal disengagement domain of burnout or professional fulfillment. Our results indicate that the physicians who spend more time on the EHR outside of allocated appointment time are also more exhausted at work, physically and emotionally. It may very well be that physicians who are exhausted at work due to other factors spend more time on the EHR outside allocated appointment time.

Interpersonal disengagement specifically assesses empathy and connectedness with others, particularly patients and colleagues.[27] Our finding that the physicians with larger CLOC ratios are not disengaged or burned out despite being exhausted may be reflective of well-established personal resources aiding self-efficacy.[35] [36] Since Stanford is an academic medical center, these findings may also be a result of greater variation in work tasks (e.g., patient care, education, research, administrative leadership, etc.) and dedicated time for documentation available to academic physicians and the support resources available. Moreover, substantial variation exists in timing of EHR use among individual providers, more so in the academic setting, to the effect that some providers prefer documenting outside allocated appointment time during available administrative or research slots during daytime ([Table 4]). We noted that CLOC time during daytime (8 a.m. to 6 p.m.) was greater than CLOC time during after-hours (6 p.m. to 8 a.m.) across all specialties, emphasizing the importance of capturing these times as work outside work by acknowledging variation in physician schedules especially in an academic setting ([Table 4]). Majority of the work being done between the hours of 8 a.m. and 6 p.m. may have contributed to the lack of significant correlation between CLOC ratio and interpersonal disengagement and burnout. This finding may support incorporating more dedicated time for documentation into physicians' schedules. Exhaustion, however, has been shown to predict future disengagement, which is cause for concern.[37] Further studies are needed to assess the longitudinal relationship between CLOC ratio, work exhaustion, interpersonal disengagement, and overall burnout.

Table 4 Correlations for allocated appointment time and CLOC time across specialties

	Dermatology	Endocrinology	General internal medicine	Hematology and oncology	Neurology	Rheumatology
N	17	10	138	9	21	10
Allocated appointment time[a] (mean/SD) (h/wk)	16.46 (5.23)	22.85 (8.47)	24.39 (8.58)	16.25 (4.25)	18.89 (5.61)	21.32 (8.38)
Total CLOC time[b] (mean (SD)) (h/wk)	11.94 (6.49)	14.41 (13.26)	12.25 (8.41)	11.05 (6.58)	12.30 (5.55)	13.67 (7.69)
CLOC ratio[c] (mean (SD))	0.98 (0.65)	0.76 (0.60)	0.65 (0.48)	0.92 (0.51)	0.79 (0.34)	0.75 (0.38)
CLOC time 8 a.m. to 6 p.m. M-F (mean/SD) (h/wk)	7.47 (2.59)	7.33 (4.01)	7.04 (3.72)	6.92 (3.85)	9.17 (3.68)	6.55 (2.40)
CLOC time 6 p.m. to 8 a.m. M-F (mean/SD) (h/wk)	2.68 (2.98)	4.03 (4.24)	3.34 (3.85)	2.66 (2.61)	2.17 (1.94)	4.87 (4.49)
CLOC Time weekends (mean/SD) (h/wk)	1.78 (2.43)	3.04 (6.18)	1.87 (2.62)	1.47 (1.30)	0.95 (1.43)	2.52 (2.43)

Abbreviations: CLOC, Clinician Logged-in Outside Clinic; SD, standard deviation.

^a p < 0.001.

^b Total CLOC time: time spent on the EHR outside of allocated appointment time.

^c CLOC ratio: ratio of CLOC time to allocated appointment time.

Our results are in alignment with a recent study that reported that clinicians (physicians and nurse practitioners) in the top two quartiles of vendor-defined EHR time after hours on scheduled clinic days had a significantly greater odds of high exhaustion.[20] On the other hand, Adler-Milstein et al did not find any significant associations between minutes active on unscheduled days per clinical full-time equivalent and either cynicism or emotional exhaustion. In contrast to common vendor-defined use measures, the CLOC metrics fulfill many of the recommendations around use of EHR use measures toward research as recommended by Sinsky et al.[12]

A feature of the CLOC metrics is that it takes into consideration variation in physicians' schedules. For instance, a physician may choose to work certain hours of the day, e.g., 7 a.m. to 2 p.m., as it better suits their work–life balance. In this case, any work done on the EHR after 2 p.m. would be considered as work outside work (WOW) by the CLOC metric. On the other hand, most vendor-defined work outside work metrics may consider time spent on EHR after a certain time, e.g., 7 p.m., as WOW, which may underestimate work outside of work. Another instance of variability in physician schedules could be seen when a physician has long time intervals between consecutive appointment slots (e.g., schedule split between morning and afternoon sessions) which they may use to complete charting. The CLOC metrics capture time spent on the EHR during these periods between appointments as WOW, whereas vendor-defined WOW metrics may miss this work done on the EHR in between appointments which we believe also contributes to EHR burden. We noted that CLOC time during daytime (8 a.m. to 6 p.m.) was greater than CLOC time during after-hours (6 p.m. to 8 a.m.) across all specialties, emphasizing the importance of capturing these times as work outside work by acknowledging variation in physician schedules especially in an academic setting ([Table 4]). CLOC metrics also consider work done during shoulder time (the period of time, usually around 30 minutes, immediately before and after a physician's first and last allocated patient slot) as work outside work. As an example, a physician with 3 hours of allocated appointment time per day will have a relatively longer shoulder time than a physician with 8 hours of allocated appointment time. Any time spent on the EHR during shoulder time for a clinician with a smaller clinic volume would be considered significant. By including time spent on the EHR during shoulder time, CLOC metrics capture this EHR work outside work which may be missed in some vendor-defined metrics leading to underestimation of work outside of work. Moreover, CLOC metrics include inactive EHR time as it has been suggested that both active and idle times may impact the physician and the quality of their time at home.[12] Similarly, inactive time may represent review of notes/laboratory values without keyboard or mouse activity or interruptions at work and may contribute to longer time to task completion and add to frustration.

Our finding of allocated appointment time having a significant positive correlation with professional fulfillment and a significant inverse correlation with burnout, work exhaustion, and interpersonal disengagement is unexpected as independent relationship between work hours and burnout has been described in the literature ([Table 5]).[38] Our results could be interpreted as having more time dedicated to clinical work in an academic medical center being protective against work exhaustion, interpersonal disengagement, and overall burnout as has been suggested in previous studies with statements such as “seeing the patient is the joy.”[39] [40] This could also be explained by the reasoning that physicians who enjoy practicing take on higher clinic volumes.

Table 5 Correlations for other metrics of interest and wellness measures

Variables	Burnout (rho)	Work exhaustion (rho)	Interpersonal disengagement (rho)	Professional fulfillment (rho)
Allocated appointment time	−0.19[a]	−0.15[a]	−0.19[a]	0.17[a]
Scheduled patient time	−0.19[a]	−0.15[a]	−0.20[a]	0.18[a]
With patient time	−0.19[a]	−0.15[a]	−0.20[a]	0.18[a]

^a p < 0.05.

Our study has inherent limitations. Although the study used a chronologic approach that evaluated nonproprietary CLOC metrics over the 3 months immediately prior to assessment of burnout and professional fulfillment, causality for specific dimensions cannot be established. The study included physicians affiliated with a single clinical organization which creates a sample population that may be different from other institutions owing to differences in structure of administration and service, although the inclusion of academic and community-based practices may increase generalizability of our findings. Since we extracted burnout and professional fulfillment scores from surveys, nonrespondent bias is an inherent limitation of our study. CLOC metrics may overestimate WOW as they include inactive EHR time and time spent interacting with other applications simultaneously while active on the EHR. CLOC metrics include inactive time as opposed to vendor-provided metrics which may discount inactive time (the threshold of inactivity used by Epic for calculations of vendor-defined metrics at our institution is 5 seconds). We believe that inactive time may represent review of notes/laboratory values without keyboard or mouse activity or interruptions at work/home and may contribute to longer time to task completion and add to frustration as supported by the recommendations for core EHR use metrics by Sinsky et al.[12] CLOC metric also does not account for hours spent on inpatient service and other nonclinical work in the EHR (quality improvement or other administrative work, chart review for research, etc.).

Conclusion

Our study emphasizes the need to measure, report, and compare work outside work hours at scale. We introduce the CLOC metrics, some of which, like vendor-defined metrics, appear to be objective EHR activity-based markers of work exhaustion, but not professional fulfillment or burnout. This is consistent with emerging data that show that if there is an association between markers of EHR use time with burnout or professional fulfillment, it appears to be a small effect. The CLOC metrics also have the potential added benefit of transparency in calculation over proprietary metrics supporting broader implementation, interpretation, and longitudinal analysis.[41] CLOC may potentially open options for comparative research involving metrics associated with EHR use duration data and physician wellness both within and across institutions and EHRs.

Clinical Relevance Statement

The CLOC metrics are potential objective EHR activity-based markers associated with physician work exhaustion. Institutions may be able to leverage CLOC metrics for comparative and longitudinal research involving metrics associated with EHR use duration data and physician wellness. Targeting interventions at the individual level for providers with higher CLOC ratio may be beneficial in decreasing work exhaustion.

Multiple Choice Questions

1. Our study suggests that CLOC ratio, ratio of CLOC time (time spent on EHR outside of allocated appointment time) to allocated appointment time in the outpatient setting, is correlated with physician:

   • Work exhaustion

   • Overall burnout

   • Professional fulfillment

   • Interpersonal disengagement

   Correct Answer: The correct answer is option a. Work exhaustion is the correct choice. Although it is challenging to determine an ideal number of CLOC hours given individual variation in EHR use patterns, documentation preferences, and schedules, our study establishes an association between CLOC ratio and work exhaustion. Our finding is consistent with association in recent literature. In general, this would suggest a lower CLOC ratio is more favorable, which could suggest that efforts to reduce the CLOC ratio may be worthwhile and that it could be used as an improvement metric for efforts to tackle work exhaustion.

2. Clinician Logged-in Outside Clinic (CLOC) metrics take into account:

   • Inactive EHR time

   • Variability in physician schedules

   • Shoulder-time

   • All of the above

   Correct Answer: The correct answer is option d. All of the above is the correct choice. In contrast to common vendor-defined use measures, the CLOC metrics fulfill many of the recommendations around use of EHR use measures toward research as recommended by Sinsky et al.[12] It takes into consideration variations in physician's schedules, takes into account inactive time, i.e., idle time without mouse clicks and keyboard strokes, and includes work done immediately before and after a physician's first and last allocated patient slot (i.e., “shoulder time”), which prevents underestimation of work outside of work.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by Stanford University Institutional Review Board.

• References

• 1 Office of the National Coordinator for Health Information Technology. “Office-based Physician Electronic Health Record Adoption,” Health IT Quick-Stat #50. Accessed January 2019 at: dashboard.healthit.gov/quickstats/pages/physician-ehr-adoption-trends.php
• 2 Burde H. Health law the hitech act-an overview. Virtual Mentor 2011; 13 (03) 172-175
• 3 Lin SC, Jha AK, Adler-Milstein J. Electronic health records associated with lower hospital mortality after systems have time to mature. Health Aff (Millwood) 2018; 37 (07) 1128-1135
• 4 Lammers EJ, McLaughlin CG, Barna M. Physician EHR adoption and potentially preventable hospital admissions among medicare beneficiaries: panel data evidence, 2010-2013. Health Serv Res 2016; 51 (06) 2056-2075
• 5 Colicchio TK, Cimino JJ, Del Fiol G. Unintended consequences of nationwide electronic health record adoption: challenges and opportunities in the post-meaningful use era. J Med Internet Res 2019; 21 (06) e13313
• 6 Sittig DF, Wright A, Ash J, Singh H. New unintended adverse consequences of electronic health records. Yearb Med Inform 2016; 2016 (01) 7-12
• 7 Ho Y-X, Gadd CS, Kohorst KL, Rosenbloom ST. A qualitative analysis evaluating the purposes and practices of clinical documentation. Appl Clin Inform 2014; 5 (01) 153-168
• 8 Koopman RJ, Steege LMB, Moore JL. et al. Physician information needs and electronic health records (EHRs): time to reengineer the clinic note. J Am Board Fam Med 2015; 28 (03) 316-323
• 9 Ozeran L, Schreiber R. Reduce burnout by eliminating billing documentation rules to let clinicians be clinicians: a clarion call to informaticists. Appl Clin Inform 2021; 12 (01) 73-75
• 10 Downing NL, Bates DW, Longhurst CA. Physician burnout in the electronic health record era: are we ignoring the real cause?. Ann Intern Med 2018; 169 (01) 50-51
• 11 Holmgren AJ, Downing NL, Bates DW. et al. Assessment of electronic health record use between US and non-US health systems. JAMA Intern Med 2021; 181 (02) 251-259
• 12 Sinsky CA, Rule A, Cohen G. et al. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc 2020; 27 (04) 639-643
• 13 Marmor RA, Clay B, Millen M, Savides TJ, Longhurst CA. The impact of physician EHR usage on patient satisfaction. Appl Clin Inform 2018; 9 (01) 11-14
• 14 Tai-Seale M, Olson CW, Li J. et al. Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine. Health Aff (Millwood) 2017; 36 (04) 655-662
• 15 Sinsky C, Colligan L, Li L. et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016; 165 (11) 753-760
• 16 Dyrbye LN, West CP, Burriss TC, Shanafelt TD. Providing primary care in the United States: the work no one sees. Arch Intern Med 2012; 172 (18) 1420-1421
• 17 Melnick ER, Ong SY, Fong A. et al. Characterizing physician EHR use with vendor derived data: a feasibility study and cross-sectional analysis. J Am Med Inform Assoc 2021; DOI: 10.1093/jamia/ocab011.
• 18 Tait SD, Oshima SM, Ren Y. et al. Electronic health record use by sex among physicians in an academic health care system. JAMA Intern Med 2021; 181 (02) 288-290
• 19 Arndt BG, Beasley JW, Watkinson MD. et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med 2017; 15 (05) 419-426
• 20 Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc 2020; 27 (04) 531-538
• 21 Gardner RL, Cooper E, Haskell J. et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26 (02) 106-114
• 22 Robertson SL, Robinson MD, Reid A. Electronic health record effects on work-life balance and burnout within the I³ population collaborative. J Grad Med Educ 2017; 9 (04) 479-484
• 23 DiAngi YT, Stevens LA, Halpern-Felsher B, Pageler NM, Lee TC. Electronic health record (EHR) training program identifies a new tool to quantify the EHR time burden and improves providers' perceived control over their workload in the EHR. JAMIA Open 2019; 2 (02) 222-230
• 24 Tran B, Lenhart A, Ross R, Dorr DA. Burnout and EHR use among academic primary care physicians with varied clinical workloads. AMIA Jt Summits Transl Sci Proc 2019; 2019: 136-144
• 25 Melnick ER, Sinsky CA, Krumholz HM. Implementing measurement science for electronic health record use. JAMA 2021; 325 (21) 2149-2150
• 26 Shanafelt TD, Makowski MS, Wang H. et al. Association of burnout, professional fulfillment, and self-care practices of physician leaders with their independently rated leadership effectiveness. JAMA Netw Open 2020; 3 (06) e207961
• 27 Trockel M, Bohman B, Lesure E. et al. A brief instrument to assess both burnout and professional fulfillment in physicians: reliability and validity, including correlation with self-reported medical errors, in a sample of resident and practicing physicians. Acad Psychiatry 2018; 42 (01) 11-24
• 28 Cohen GR, Friedman CP, Ryan AM, Richardson CR, Adler-Milstein J. Variation in physicians' electronic health record documentation and potential patient harm from that variation. J Gen Intern Med 2019; 34 (11) 2355-2367
• 29 Kroth PJ, Morioka-Douglas N, Veres S. et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (08) e199609
• 30 Hilliard RW, Haskell J, Gardner RL. Are specific elements of electronic health record use associated with clinician burnout more than others?. J Am Med Inform Assoc 2020; 27 (09) 1401-1410
• 31 Khairat S, Burke G, Archambault H, Schwartz T, Larson J, Ratwani RM. Perceived burden of EHRs on physicians at different stages of their career. Appl Clin Inform 2018; 9 (02) 336-347
• 32 Gregory ME, Russo E, Singh H. Electronic health record alert-related workload as a predictor of burnout in primary care providers. Appl Clin Inform 2017; 8 (03) 686-697
• 33 Shanafelt TD, Dyrbye LN, Sinsky C. et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc 2016; 91 (07) 836-848
• 34 Melnick ER, Dyrbye LN, Sinsky CA. et al. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc 2020; 95 (03) 476-487
• 35 Meldrum H. Exemplary physicians' strategies for avoiding burnout. Health Care Manag (Frederick) 2010; 29 (04) 324-331
• 36 Shanafelt TD, Oreskovich MR, Dyrbye LN. et al. Avoiding burnout: the personal health habits and wellness practices of US surgeons. Ann Surg 2012; 255 (04) 625-633
• 37 Rogala A, Shoji K, Luszczynska A. et al. From exhaustion to disengagement via self-efficacy change: findings from two longitudinal studies among human services workers. Front Psychol 2016; 6: 2032
• 38 West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med 2018; 283 (06) 516-529
• 39 Shannon D. Physician well-being: a powerful way to improve the patient experience. Physician Exec 2013; 39 (04) 6-8
• 40 Spinelli WM, Fernstrom KM, Britt H, Pratt R. “Seeing the patient is the joy:” a focus group analysis of burnout in outpatient providers. Fam Med 2016; 48 (04) 273-278
• 41 Hron JD, Lourie E. Have you got the time? Challenges using vendor electronic health record metrics of provider efficiency. J Am Med Inform Assoc 2020; 27 (04) 644-646

Publication History

Received: 01 February 2021

Accepted: 25 May 2021

Article published online:
14 July 2021