Impact of Changes in EHR Use during COVID-19 on Physician Trainee Mental Health

Abstract

Objectives This article investigates the association between changes in electronic health record (EHR) use during the coronavirus disease 2019 (COVID-19) pandemic on the rate of burnout, stress, posttraumatic stress disorder (PTSD), depression, and anxiety among physician trainees (residents and fellows).

Methods A total of 222 (of 1,375, 16.2%) physician trainees from an academic medical center responded to a Web-based survey. We compared the physician trainees who reported that their EHR use increased versus those whose EHR use stayed the same or decreased on outcomes related to depression, anxiety, stress, PTSD, and burnout using univariable and multivariable models. We examined whether self-reported exposure to COVID-19 patients moderated these relationships.

Results Physician trainees who reported increased use of EHR had higher burnout (adjusted mean, 1.48 [95% confidence interval [CI] 1.24, 1.71] vs. 1.05 [95% CI 0.93, 1.17]; p = 0.001) and were more likely to exhibit symptoms of PTSD (adjusted mean = 15.09 [95% CI 9.12, 21.05] vs. 9.36 [95% CI 7.38, 11.28]; p = 0.035). Physician trainees reporting increased EHR use outside of work were more likely to experience depression (adjusted mean, 8.37 [95% CI 5.68, 11.05] vs. 5.50 [95% CI 4.28, 6.72]; p = 0.035). Among physician trainees with increased EHR use, those exposed to COVID-19 patients had significantly higher burnout (2.04, p < 0.001) and depression scores (14.13, p = 0.003).

Conclusion Increased EHR use was associated with higher burnout, depression, and PTSD outcomes among physician trainees. Although preliminary, these findings have implications for creating systemic changes to manage the wellness and well-being of trainees.

Keywords

electronic health records - COVID-19 - depression - burnout - physician trainees

Background and Significance

The coronavirus disease 2019 (COVID-19) pandemic has markedly impacted the mental health and well-being of health care workers around the world.[1] [2] In parallel, the pandemic has created considerable changes in the clinical practice environment, including that of clinical workflows (e.g., new telemedicine workflows), guidelines for care (e.g., changes in the evidence-base for care), and modes of interaction and navigation within the electronic health record (EHR) (e.g., virtual visits). This rapid transformation required physicians to adapt their work activities. Perhaps, one of the most prominent changes was the surge in telemedicine use across health care systems.[3] [4] [5] [6] Despite important benefits, including convenience and improved patient access, emerging evidence highlights the challenges these changes had on physicians, including the loss of personal connections and workflow issues associated with on-demand care.[7] [8]

Recent research has highlighted the mental health distress among physician trainees affecting burnout, stress, and anxiety.[9] [10] [11] [12] Trainees have reported concerns about their safety, safety of their patients, implications of their decisions on their family,[13] challenges of child care, and lost educational opportunities[10] [14] as key stressors during the pandemic. This is especially important given that trainees are a unique group of physicians functioning both as learners and care providers, often with limited autonomy or independence.

Even prior to the pandemic, EHR use was a key contributor to increased clinical workload, leading to stress, burnout, and adverse mental health outcomes.[15] [16] [17] [18] [19] [20] With the pandemic-induced changes these effects are likely more pronounced. Because of the high documentation burden placed on physician trainees,[21] [22] [23] [24] they spend a considerable amount of time on the EHR,[25] [26] [27] increasing their workload and burden. As such, the twin, potentially deleterious, effects of patient care during the pandemic and changes in EHR-based workflow can have a considerable impact on physician trainees and have not been previously studied.

The purpose of this study is twofold. First, we sought to investigate the relationship between changes in EHR use during the pandemic and the mental health and well-being of physician trainees, assessed as burnout, depression, anxiety, stress, and posttraumatic stress disorder (PTSD) outcomes. Second, we examined the potential interaction of exposure to COVID-19 patients on the relationship between EHR use and mental health outcomes.

Method

Participants and Survey

Physician trainees (n = 1,375), both residents and fellows, at Washington University School of Medicine received a Web-based survey on June 19, 2020, with a follow-up reminder a week later. A $50 gift card raffle was offered to survey respondents for participation. The survey was approved by the institutional review board of Washington University (IRB #202004021). Survey invitation emails described the purpose of the survey as “to understand the impact of the COVID-19 pandemic on the well-being of physician trainees” and that the survey was part of a longitudinal study assessing the wellness of physician trainees during the COVID-19 pandemic.[10]

The survey included demographic questions about race, sex, marital status, training program, and clinical role. Participants were also asked whether they were caring for patients who tested positive for COVID-19 (Y/N). Survey questions related to the five outcomes included: burnout, as measured with the Stanford Professional Fulfillment Index (PFI)[28]; PTSD, as measured by the Impact of Event Scale–Revised (IES-R)[29]; and depression, anxiety, and stress, as measured by the Depression Anxiety Stress Scale (DASS-21).[30]

The 16-item PFI was used to measure the level of burnout (workload exhaustion and interpersonal disengagement) and professional fulfillment during the past 2 weeks. The PFI burnout scale has previously been validated with the Maslach Burnout Inventory, and the fulfillment measure is correlated with quality of life scores.[28] [31] [32]

The IES-R scale was used to measure the amount of distress respondents felt with respect to the current COVID-19 pandemic. Responses were on a 5-item Likert scale ranging from “Not at all” to “Extremely.” The scores range from 0 to 88 with a score of 24 or higher indicating clinical concern for PTSD, including at least partial PTSD or some of its symptoms.[33]

The DASS-21 is a 21-item scale that has demonstrated strong validity and reliability in clinical and nonclinical samples[34] [35] and has been used with physician trainees.[36] This scale was used to ascertain level of depression, anxiety, and stress.[30]

Exposure

We considered two exposures related to EHR use: changes in overall EHR use and EHR use outside of work (with an example of “home” as a setting outside of work). Respondents were asked about changes in their EHR usage since the start of the pandemic (March 2020), indicating whether their “overall EHR use” and “EHR use outside of work hours” have been the same, increased, or decreased.

Statistical Analysis

Physician trainee responses to their self-reported changes in overall EHR use and EHR use outside of work questions were categorized into two groups: increasing and same/decreasing. Race was categorized as Caucasian, Asian, black, Hispanic or Latino, and Other/Prefer not to answer; sex was categorized as female or not female; and marital status was categorized as married or not married.

Descriptive statistics for changes in EHR use (both overall and outside of work) among increasing and same/decreasing groups, outcomes from the DASS-21 were categorized as normal or nonnormal as follows: depression (0–9 normal, 10 or greater not normal), anxiety (0–7 normal, 8 or greater not normal), and stress (0–14 normal, 15 or greater not normal). These cut-points were based on previously published literature on the DASS-21 scale.[37] PTSD was categorized as a binary variable based on the diagnostic cutoff recommended by the IES-R (0–24 no PTSD, 24 or greater indicates clinically meaningful PTSD symptomatology).[38] [39]

Burnout was determined from the average item score for the workload and depersonalization scales (score range 0–4), using a cut-point of 1.33 as described in Trockel et al,[28] where scores greater than or equal to 1.33 were considered as “burned out.”

For univariable and multivariable analyses, burnout was analyzed as a continuous outcome variable, and outcomes from the DASS-21 (depression, anxiety, and stress) and IES-R (PTSD) were analyzed as count variables. Associations between the EHR use and burnout were analyzed using linear least-squares regression analyses; depression, anxiety, and stress outcomes were analyzed using negative binomial regression analyses. Exposure effects were adjusted in multivariable models by including covariates with p-values < 0.10 in the univariable analyses. The final multivariable models investigated possible interactions between trainees exposed to COVID-19 positive patients and increasing or same/decreasing overall EHR use and EHR use outside of work. p-Values < 0.05 were considered significant. All analyses were conducted using Stata/SE version 16.0.[40]

Results

There were 250 responses to the survey. After removing 28 duplicate or incomplete entries, there were a total of 222 completed surveys (16.2% response).

General Characteristics

Demographic characteristics are presented in [Table 1]. There were equal numbers of males and females in the sample. Participants were predominantly residents (70.3%), within the first 3 years of their training (78.3%), Caucasian (64.0%), married (58.1%), and with no children (77.5%). Twenty-seven percent of the participants reported that they were exposed to COVID-19 patients. Since the start of the pandemic, 21% of trainees reported increased EHR usage, and 33% reported increased EHR use outside of work.

Table 1 Summary descriptive table of the considered variables in the survey for all trainees

Variable	Group	All trainees (n = 222)	Overall EHR use increasing (n = 46)	Overall EHR use same/decreasing (n = 175)	EHR use outside of work increasing (n = 73)	EHR use outside of work same/decreasing (n = 148)
Clinical role	Fellow	66 (29.7%)	17 (37.0%)	48 (27.4%)	21 (28.8%)	44 (29.7%)
	Resident	156 (70.3%)	29 (63.0%)	127 (72.6%)	52 (71.2%)	104 (70.4%)
Female	Yes	111 (50.0%)	25 (54.4%)	86 (49.1%)	40 (54.8%)	70 (47.3%)
Race	Caucasian	142 (64.0%)	30 (65.2%)	111 (63.8%)	48 (65.8%)	93 (63.3%)
	Asian	54 (24.3%)	8 (17.4%)	46 (26.4%)	16 (21.9%)	38 (25.9%)
	Black	5 (2.3%)	2 (4.4%)	3 (1.7%)	1 (1.4%)	4 (2.7%)
	Hispanic or Latino	6 (2.7%)	2 (4.4%)	4 (2.3%)	2 (2.7%)	4 (2.7%)
	Other/Unknown	14 (6.3%)	4 (8.7%)	10 (5.7%)	6 (8.2%)	8 (5.4%)
Married	Yes	129 (58.1%)	25 (54.4%)	103 (58.9%)	41 (56.2%)	87 (58.8%)
Children at home	Yes	50 (22.5%)	9 (19.6%)	40 (22.9%)	13 (17.8%)	37 (25.0%)
≥ 4 years of program	Yes	48 (21.7%)	5 (10.9%)	43 (24.7%)	15 (20.6%)	33 (22.5%)
Exposed to COVID-19 positive patients	Yes	81 (26.5%)	17 (37.0%)	63 (36.0%)	21 (28.8%)	59 (39.9%)
Depression	Mild to extremely severe	61 (27.5%)	16 (34.8%)	45 (25.7%)	27 (37.0%)	33 (22.3%)
Anxiety	Mild to extremely severe	42 (18.9%)	10 (21.7%)	32 (18.3%)	18 (24.7%)	24 (16.2%)
Stress	Mild to extremely severe	50 (22.5%)	15 (32.6%)	35 (20.0%)	21 (28.8%)	28 (18.9%)
Posttraumatic stress disorder (IES-R ≥ 24)	Yes	20 (9.0%)	14 (30.4%)	22 (12.6%)	17 (23.3%)	19 (12.8%)
Professional fulfillment (PFI ≥ 3)	Yes	60 (27.0%)	11 (23.9%)	48 (27.4%)	22 (30.1%)	38 (25.7%)
Burnout (mean workload, depersonalization PFI ≥ 1.33)	Yes	84 (37.8%)	24 (52.2%)	59 (33.7%)	35 (48.0%)	48 (32.4%)

Abbreviations: COVID-19, coronavirus disease 2019; EHR, electronic health record; IES-R, Impact of Event Scale–Revised; PFI, Professional Fulfillment Index.

The distribution of participants on each of the considered predictors and outcomes of both the EHR use and EHR use outside of work variables are shown in [Table 1].

Multivariable Analysis

Findings from the multivariable model indicated that physician trainees with increased overall EHR use were more likely to be burned out (1.48 [95% confidence interval [CI], 1.24–1.71] vs. 1.05 [95% CI, 0.93–1.17]; p = 0.001) (see [Table 2]; for covariates see [Appendix A]).

Table 2 Univariable and multivariable model results

Variable	Group	Unadjusted mean burnout score (95% CI)	Univariable p-value	Adjusted mean burnout score (95% CI)	Multivariable p-value
Overall EHR use	Same/Decreasing	1.05 (0.93, 1.18)	0.002	1.05 (0.93, 1.17)	0.001
	Increasing	1.46 (1.22, 1.69)		1.48 (1.24, 1.71)
EHR use outside of work	Same/Decreasing	1.06 (0.93, 1.20)	0.051	1.05 (0.92, 1.19)	0.026
	Increasing	1.30 (1.11, 1.48)		1.32 (1.13, 1.50)
Variable	Group	Unadjusted mean depression score (95% CI)	Univariable p -value	Adjusted mean depression score (95% CI)	Multivariable p -value
Overall EHR use	Same/Decreasing	5.81 (4.61, 7.01)	0.063	5.83 (4.62, 7.05)	0.079
	Increasing	8.89 (5.36, 12.42)		8.77 (5.25, 12.29)
EHR use outside of work	Same/Decreasing	5.51 (4.29, 6.74)	0.039	5.50 (4.28, 6.72)	0.035
	Increasing	8.32 (5.65, 10.98)		8.37 (5.68, 11.05)
Variable	Group	Unadjusted mean PTSD score (95% CI)	Univariable p -value	Adjusted mean PTSD score (95% CI)	Multivariable p -value
Overall EHR use	Same/Decreasing	9.26 (7.35, 11.18)	0.025	9.33 (7.38, 11.28)	0.035
	Increasing	15.49 (9.32, 21.65)		15.09 (9.12, 21.05)
EHR use outside of work	Same/Decreasing	9.14 (7.06, 11.23)	0.068	9.29 (7.15, 11.44)	0.114
	Increasing	13.23 (8.93, 17.52)		12.81 (8.68, 16.94)

Abbreviations: CI, confidence interval; EHR, electronic health record; PTSD, posttraumatic stress disorder.

Note: Burnout was analyzed using linear least-squares regression and PTSD and depression-related outcomes were analyzed using negative binomial regression. Unadjusted means correspond to means unadjusted for covariates. Adjusted means correspond to means from multivariable models adjusted for variables with p < 0.10 in the univariable analyses (see [Appendix A] and [B]). Bold indicates p value <.10.

Physician trainees reporting increased EHR use had increased symptoms of PTSD (15.09 [95% CI 9.12, 21.05] vs. 9.33 [95% CI 7.38, 11.28]; p = 0.035) (see [Table 2]; for covariates see [Appendix B]). Those reporting increased EHR use outside of work had higher depression scores (8.37 [95% CI, 5.68, 11.05] vs. 5.50 [95% CI, 4.28, 6.72]; p = 0.035) (see [Table 2]; for covariates see [Appendix C]). There were no significant differences between the EHR usage groups for stress or anxiety (see [Appendix D] and [E]).

There was a significant interaction between self-reported caring for COVID-19 patients and changes in overall EHR use and burnout (p = 0.003) and depression (p = 0.032). Among the participants reporting increased EHR use, those caring for COVID-19 patients had a significantly higher mean burnout score (2.04, p < 0.001) than those whose EHR use increased but were not caring for COVID-19 patients. Burnout score for the group whose EHR use stayed the same or decreased was not significantly different regardless of their exposure to COVID-19 patients. Similarly, among physician trainees whose EHR use increased, those that were exposed to COVID-19 patients had a significantly higher mean depression score (14.13, p = 0.003) than those who were not caring for COVID-19 patients. The depression score for the group whose reported EHR use remained the same or decreased was not significantly different for those who were exposed compared with those who were not exposed to COVID-19 patients (see [Fig. 1]).

Fig. 1 Exposure to COVID-19 positive patients moderates relationship between overall EHR use and levels of burnout (left) and depression (right).

Discussion

Based on a cross-sectional study of physician trainees, we found that self-reported increased EHR use, including EHR use outside of work settings, was associated with a higher likelihood of burnout, depression, and PTSD. An estimated 38% of physician trainees met the criteria for burnout, similar to results of a recent review of burnout among health care workers in general during COVID-19 (34.4%)[41] and slightly less than a study focusing specifically on surgery trainees during COVID-19 (39–45%).[9] Notably, the range of burnout among physician trainees prior to COVID-19 has varied widely with several studies reporting greater than 50%.[42] [43] [44] [45] Results for depression (27.5%) were similar to those for health care workers in general (26.3%), however, less than the percentage of surgical trainees (30.0%). Anxiety was less common among trainees in the present study (18.9%) than health care workers (29.0%) and surgical residents (54%), though both of these studies utilized different scales. Among physician trainees reporting increased EHR use, the average levels of burnout and depression were significantly higher for those caring for COVID-19 patients, highlighting the additive impact on EHR-based changes and clinical care challenges. Although the high prevalence of burnout among trainees is well-known,[10] [46] [47] [48] the prevalence of PTSD and depression associated with increased EHR use is potentially concerning. Of these, the contributory role of EHR use outside of work settings on depression highlights the effects of the clinical workload on mental health outcomes. As such, strategies for mitigating the effects of chronic burnout and related mental health issues are urgent among physician trainees to limit the loss of physician trainees from the workforce and to prevent a parallel pandemic, that leads to distress, loneliness, substance abuse, and other chronic illnesses.[49] [50] [51]

Although we are not able to determine why some trainees reported increased EHR use, we speculate that it may be, in part, related to the increased need for virtual visits or additional tasks associated with the changes occurring during the COVID-19 pandemic. For example, telemedicine interactions have changed the mechanism of physician–patient interactions and are associated with increased administrative burden.[52] [53] [54] [55] Telemedicine interactions are “touchless,” where physicians can no longer perform a physical exam or collect vitals. Instead, they rely on the patient's record for collecting information utilizing past medical history, notes, and video-based communication for their reasoning and clinical decision making.[56]

Challenges of the video-based medium also increase the difficulties of communication, where patients can be less forthcoming, and the nuances that are available in a face-to-face conversation may be lost.[57] [58] [59] [60] Such changes can also result in the need for additional documentation and for potentially additional clinical tests, all of which add to the clinical activities that need to be performed on the EHR. There are likely contributing factors to increased EHR use and burnout; although the effects of increased telehealth use may have been a contributor to burnout, we did not specifically investigate it in this study. As reported elsewhere, there are considerable advantages of telemedicine to both clinicians and patients.[61] [62] Recent reports have highlighted the increased number of messages (phone, inbox) that physicians receive.[63] High volume of calls/messages may also have contributed to the increased EHR use, a factor that has been strongly associated with clinician burnout.[64]

Increased EHR workload can also lead to worse mental health outcomes due to cognitive dissonance[65]—a mismatch between the trainees' need to perform mundane documentation tasks and their altruistic goals of patient care—where the twin challenges of additional cognitive work on the EHR and the external challenges and stress associated with the pandemic contributes to moral distress and burnout. Identifying and addressing the sources of such challenges can be an important first step in streamlining clinical care activities using the EHR. In addition, the additive effects of caring for patients during a pandemic with increased EHR-based work activities highlight the complex system-level factors that affect physician burnout and wellness.[51] [66] Addressing these system-level factors would require reduction of nonphysician tasks, offloading physician tasks to nonphysician members of the team, improving clinical workflows, and streamlining interactions with EHRs.[67] [68] Additional considerations of incorporating sociotechnical factors into the design of EHRs, and associated clinical work activities can also help in improving clinical work practices.[69]

Limitations

This study has several limitations. First, this was a cross-sectional, single-center study. As such, the associations between the outcomes and exposure factors may not be causal and the findings should be interpreted with caution. It is possible that there was a response bias among the participants; participants who were distressed may not have completed the survey or may have been more likely to participate because of the relevance of the topic. The response rate for the survey was low, and is only marginally comparable to response rates in other surveys of clinical trainees[47] that were conducted prior to the pandemic. Survey response rates among physician trainees during the pandemic have been similar or lower.[9]

Measurements related to EHR use and EHR use outside of work were based on self-report and may have been affected by recall bias. It is possible that “overall EHR use” was interpreted as a proxy for workload or the total number of hours worked. Physician trainees who worked more, may have potentially perceived themselves as using the EHR more. Further, these results represent a single time point during the pandemic, and although the questions were framed around EHR use and distress during the pandemic, it is difficult to ascertain how much of the change can be attributed to the pandemic rather than nonpandemic-related changes. Participants' prepandemic EHR practices and baseline mental health scores were unknown, and hence, it is difficult to ascertain the directionality of the relationship between these variables. It is possible that mental health issues are causing individuals to feel that their work is increasing.

Conclusion and Future Research

This study characterizes the unique relationship between clinical work activities that compound the stressors experienced by trainees. It was revealed that trainees have high rates of mental distress and those reporting more EHR use, particularly at home, had markedly higher rates of distress. Exposure to COVID-19 patients appears to have exacerbated these already high rates. These experiences are likely to be similar or more pronounced in other settings where the infection rates are higher.

Research relying on audit logs[70] [71] [72] [73] or similar passive measurements to complement results of self-report measures can potentially be used to ascertain the true changes in EHR use during the pandemic. Future studies should also look at longitudinal relationships between EHR use, COVID-19 patient exposure, and mental health distress.

Clinical Relevance Statement

The outcomes of this study indicate that increased EHR use is associated with higher burnout, depression, and PTSD among physician trainees. Those trainees who are exposed to COVID-19 patients have significantly higher burnout and depression scores than trainees not caring for COVID-19 patients. The combination of these stressors and their concerning effects on mental health suggest the need for systemic changes to improve the well-being of trainees, particularly during this period of vulnerability. More effective methods to combat these issues include a reduction of nonphysician tasks with the help of nonphysician team members, improving clinical workflows, and streamlining EHR practices.

Multiple Choice Questions

1. Physician trainees report which of the following concerns as key stressors during the COVID-19 pandemic?

   • Their own safety

   • Safety of their patients

   • Implications of their decisions on their family

   • Challenges of childcare

   • Lost education opportunities

   • All of the above

   Correct Answer: The correct answer is f. In research conducted during the COVID-19 pandemic, trainees report that their safety, safety of their patients, safety of their family, challenges of childcare, and lost education opportunities are key stressors.

2. What is an objective alternative to subjective self-reports for characterizing physician trainees' time spent on the EHR?

   • EHR audit logs which capture and timestamp user activity

   • Computer logins

   • Rotation schedules

   • Review of patient charts

   Correct Answer: The correct answer is option a. As described in the Discussion, it is helpful to use objective techniques, when possible, to complement self-report measures of EHR use to accurately characterize time spent on EHR activities for future research studies. One of the most effective and speedy methods of characterizing trainee time spent on the EHR is an audit log that captures and timestamps user activity.

Appendix A Univariate linear least-squares regression models for burnout and covariates

Variable	Group	Unadjusted mean (95% CI)	Univariable p-value
Clinical role	Fellow	0.99 (0.80, 1.19)	0.079
	Resident	1.21 (1.08, 1.34)
Caucasian	No	1.14 (0.95, 1.32)	0.918
	Yes	1.15 (1.01, 1.28)
Female	No	1.11 (0.95, 1.26)	0.526
	Yes	1.18 (1.02, 1.33)
Children at home	No	1.19 (1.06, 1.31)	0.128
	Yes	0.99 (0.75, 1.22)
Married	No	1.21 (1.04, 1.38)	0.285
	Yes	1.09 (0.95, 1.24)
Year in program	−	0.01 (−0.08, 0.10)	0.767
Exposed to COVID-19 positive patients	No	1.05 (0.91, 1.18)	0.021
	Yes	1.31 (1.13, 1.49)

Abbreviations: CI, confidence interval; COVID-19, coronavirus disease 2019.

Appendix B Univariate negative binomial regression models for PTSD and covariates

Variable	Group	Unadjusted mean (95% CI)	Univariable p-value
Clinical role	Fellow	1.00 (7.26, 14.74)	0.756
	Resident	10.31 (8.01, 12.62)
Caucasian	No	11.39 (7.88, 14.90)	0.514
	Yes	10.01 (7.66, 12.36)
Female	No	8.65 (6.40, 10.91)	0.054
	Yes	12.45 (9.17, 15.73)
Children at home	No	10.62 (8.36, 12.87)	0.855
	Yes	10.18 (6.19, 14.18)
Married	No	10.76 (7.68, 13.84)	0.836
	Yes	10.34 (6.69, 12.89)
Year in program	−	0.15 (−0.01, 0.30)	0.064
Exposed to COVID-19 positive patients	No	9.94 (7.60, 12.28)	0.460
	Yes	11.50 (7.98, 15.02)

Abbreviations: CI, confidence interval; COVID-19, coronavirus disease 2019; PTSD, posttraumatic stress disorder.

Appendix C Univariate negative binomial regression models for depression and covariates

Variable	Group	Unadjusted mean (95% CI)	Univariable p-value
Clinical role	Fellow	6.49 (4.31, 8.68)	0.963
	Resident	6.43 (5.02, 7.84)
Caucasian	No	6.60 (4.60, 8.60)	0.850
	Yes	6.36 (4.89, 7.83)
Female	No	5.78 (4.28, 7.28)	0.265
	Yes	7.12 (5.28, 8.96)
Children at home	No	6.92 (5.49, 8.35)	0.107
	Yes	4.82 (2.94, 6.70)
Married	No	6.86 (4.91, 8.80)	0.571
	Yes	6.16 (4.67, 7.64)
Year in program		0.02 (−0.13, 0.17)	0.805
Exposed to COVID-19 positive patients	No	6.10 (4.69, 7.51)	0.456
	Yes	7.05 (4.92, 9.18)

Abbreviations: CI, confidence interval; COVID-19, coronavirus disease 2019.

Appendix D Univariate negative binomial regression models for stress and covariates

Variable	Group	Unadjusted mean (95% CI)	Univariable p-value
Overall EHR use	Same/Decreasing	8.16 (6.71, 9.61)	0.134
	Increasing	10.98 (7.19, 14.76)
My EHR use outside of work	Same/Decreasing	8.11 (6.55, 9.67)	0.237
	Increasing	9.94 (7.18, 12.70)
Clinical role	Fellow	9.97 (7.10, 12.84)	0.276
	Resident	8.23 (6.68, 9.78)
Caucasian	No	8.75 (6.46, 11.04)	0.999
	Yes	8.75 (7.01, 10.48)
Female	No	7.83 (6.08, 9.59)	0.194
	Yes	9.65 (7.52, 11.79)
Children at home	No	8.62 (7.08, 10.17)	0.745
	Yes	9.18 (6.12, 12.25)
Married	No	8.22 (6.21, 10.22)	0.517
	Yes	9.13 (7.24, 11.02)
Year in program		0.10 (−0.03, 0.26)	0.137
Exposed to COVID-19 positive patients	No	8.30 (6.65, 9.95)	0.411
	Yes	9.52 (7.05, 12.00)

Abbreviations: CI, confidence interval; COVID-19, coronavirus disease 2019; EHR, electronic health record.

Appendix E Univariate negative binomial regression models for anxiety and covariates

Variable	Group	Unadjusted mean (95% CI)	Univariable p-value
Overall EHR use	Same/Decreasing	3.19 (2.25, 4.13)	0.198
	Increasing	4.84 (2.11, 7.58)
My EHR use outside of work	Same/Decreasing	3.03 (2.05, 4.01)	0.169
	Increasing	4.48 (2.44, 6.51)
Clinical role	Fellow	3.00 (1.57, 4.43)	0.446
	Resident	3.75 (2.57, 4.92)
Caucasian	No	4.10 (2.31, 5.90)	0.369
	Yes	3.19 (2.15, 4.24)
Female	No	2.92 (1.84, 4.00)	0.193
	Yes	4.13 (2.60, 5.66)
Children at home	No	3.78 (2.66, 4.91)	0.250
	Yes	2.61 (1.16, 4.07)
Married	No	4.07 (2.43, 5.70)	0.328
	Yes	3.12 (2.04, 4.20)
Year in program		0.09 (−0.14, 0.32)	0.438
Exposed to COVID-19 positive patients	No	3.39 (2.27, 4.51)	0.723
	Yes	3.74 (2.11, 5.38)

Abbreviations: CI, confidence interval; COVID-19, coronavirus disease 2019; EHR, electronic health record.

Conflict of Interest

T.K. reports grants from NIMH, grants from NCATS, grants from NINR, personal fees from Pfizer, outside the submitted work. B.A.E. reports grants from National Institute of Occupational Safety and Health, from null, during the conduct of the study; grants from National Institute of Occupational Safety and Health, grants from National Institute of Health, from null, outside the submitted work.

Acknowledgments

The authors wish to acknowledge the help of the Information and Core Services Team at Institute of Informatics at Washington University whose personnel assisted with the development of the REDCap-based survey.

Protection of Human and Animal Subjects

This study was reviewed by the Washington University Institutional Review Board.

• References

• 1 Huang JZ, Han M, Luo T, Ren A, Zhou X. Mental health survey of 230 medical staff in a tertiary infectious disease hospital for COVID-19. Chinese J Industrial Hygiene Occupat Dis 2020; 38: 192-195
• 2 Lai J, Ma S, Wang Y. et al. Factors associated with mental health outcomes among health care workers exposed to coronavirus disease 2019. JAMA Netw Open 2020; 3 (03) e203976-e203976
• 3 Alexander GC, Tajanlangit M, Heyward J, Mansour O, Qato DM, Stafford RS. Use and content of primary care office-based vs telemedicine care visits during the COVID-19 pandemic in the US. JAMA Netw Open 2020; 3 (10) e2021476-e2021476
• 4 Hron JD, Parsons CR, Williams LA, Harper MB, Bourgeois FC. Rapid implementation of an inpatient telehealth program during the COVID-19 pandemic. Appl Clin Inform 2020; 11 (03) 452-459
• 5 Knighton AJ, Ranade-Kharkar P, Brunisholz KD. et al. Rapid implementation of a complex, multimodal technology response to COVID-19 at an integrated community-based health care system. Appl Clin Inform 2020; 11 (05) 825-838
• 6 Sullivan C, Wong I, Adams E. et al. Moving faster than the COVID-19 pandemic: the rapid, digital transformation of a public health system. Appl Clin Inform 2021; 12 (02) 229-236
• 7 Gomez T, Anaya YB, Shih KJ, Tarn DM. A qualitative study of primary care physicians' experiences with telemedicine during COVID-19. J Am Board Fam Med 2021; 34 (Suppl): S61-S70
• 8 Barney A, Buckelew S, Mesheriakova V, Raymond-Flesch M. The COVID-19 pandemic and rapid implementation of adolescent and young adult telemedicine: challenges and opportunities for innovation. J Adolesc Health 2020; 67 (02) 164-171
• 9 Coleman JR, Abdelsattar JM, Glocker RJ. et al. COVID-19 pandemic and the lived experience of surgical residents, fellows, and early-career surgeons in the American College of Surgeons. J Am Coll Surg 2021; 232 (02) 119-135.e20
• 10 Kannampallil TG, Goss CW, Evanoff BA, Strickland JR, McAlister RP, Duncan J. Exposure to COVID-19 patients increases physician trainee stress and burnout. PLoS One 2020; 15 (08) e0237301
• 11 Gallagher TH, Schleyer AM. “We Signed Up for This!”—student and trainee responses to the COVID-19 pandemic. N Engl J Med 2020; 382 (25) e96
• 12 Li W, Frank E, Zhao Z. et al. Mental health of young physicians in China during the novel coronavirus disease 2019 outbreak. JAMA Netw Open 2020; 3 (06) e2010705-e2010705
• 13 De Brier N, Stroobants S, Vandekerckhove P, De Buck E. Factors affecting mental health of health care workers during coronavirus disease outbreaks: a rapid systematic review. PLoS One 2020; 15 (12) e0244052
• 14 Lou SS, Goss CW, Evanoff BA, Duncan JG, Kannampallil T. Risk factors associated with physician trainee concern over missed educational opportunities during the COVID-19 pandemic. BMC Med Educ 2021; 21 (01) 216
• 15 Arndt BG, Beasley JW, Watkinson MD. et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med 2017; 15 (05) 419-426
• 16 Gupta K, Murray SG, Sarkar U, Mourad M, Adler-Milstein J. Differences in ambulatory EHR use patterns for male vs. female physicians. NEJM Catal 2019; DOI: 10.1056/CAT.19.0690.
• 17 Gardner RL, Cooper E, Haskell J. et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26 (02) 106-114
• 18 DiAngi YT, Stevens LA, Halpern-Felsher B, Pageler NM, Lee TC. Electronic health record (EHR) training program identifies a new tool to quantify the EHR time burden and improves providers' perceived control over their workload in the EHR. JAMIA Open 2019; 2 (02) 222-230
• 19 Shanafelt TD, Dyrbye LN, Sinsky C. et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Paper presented at: Mayo Clinic Proceedings 2016
• 20 Tai-Seale M, Dillon EC, Yang Y. et al. Physicians' well-being linked to in-basket messages generated by algorithms in electronic health records. Health Aff (Millwood) 2019; 38 (07) 1073-1078
• 21 Cox ML, Farjat AE, Risoli TJ. et al. Documenting or operating: where is time spent in general surgery residency?. J Surg Educ 2018; 75 (06) e97-e106
• 22 Bello RJ, Sarmiento S, Meyer ML. et al. Understanding surgical resident and fellow perspectives on their operative performance feedback needs: a qualitative study. J Surg Educ 2018; 75 (06) 1498-1503
• 23 Block L, Habicht R, Wu AW. et al. In the wake of the 2003 and 2011 duty hours regulations, how do internal medicine interns spend their time?. J Gen Intern Med 2013; 28 (08) 1042-1047
• 24 Arora A, Garg A, Arora V, Rizvi M, Desai N. National survey of pediatric care providers: assessing time and impact of coding and documentation in physician practice. Clin Pediatr (Phila) 2018; 57 (11) 1300-1303
• 25 Mamykina L, Vawdrey DK, Hripcsak G. How do residents spend their shift time? A time and motion study with a particular focus on the use of computers. Acad Med 2016; 91 (06) 827-832
• 26 Gagliardi JP, Turner DA. The electronic health record and education: rethinking optimization. J Grad Med Educ 2016; 8 (03) 325-327
• 27 Wang JK, Ouyang D, Hom J, Chi J, Chen JH. Characterizing electronic health record usage patterns of inpatient medicine residents using event log data. PLoS One 2019; 14 (02) e0205379
• 28 Trockel M, Bohman B, Lesure E. et al. A brief instrument to assess both burnout and professional fulfillment in physicians: reliability and validity, including correlation with self-reported medical errors, in a sample of resident and practicing physicians. Acad Psychiatry 2018; 42 (01) 11-24
• 29 Weiss DS. The Impact of Event Scale: Revised. In: Cross-Cultural Assessment of Psychological Trauma and PTSD. New York, NY: Springer; 2007: 219-238
• 30 Henry JD, Crawford JR. The short-form version of the Depression Anxiety Stress Scales (DASS-21): construct validity and normative data in a large non-clinical sample. Br J Clin Psychol 2005; 44 (Pt 2): 227-239
• 31 Maslach C, Jackson SE, Leiter MP, Schaufeli WB, Schwab RL. Maslach Burnout Inventory. Vol. 21. Palo Alto, CA: Consulting Psychologists Press; 1986
• 32 Brady KJS, Ni P, Carlasare L. et al. Establishing Crosswalks Between Common Measures of Burnout in US Physicians. J Gen Intern Med 2021; DOI: 10.1007/s11606-021-06661-4.
• 33 Asukai N, Kato H, Kawamura N. et al. Reliability and validity of the Japanese-language version of the Impact of Event Scale-Revised (IES-R-J): four studies of different traumatic events. J Nerv Ment Dis 2002; 190 (03) 175-182
• 34 Clara IP, Cox BJ, Enns MW. Confirmatory factor analysis of the Depression–Anxiety–Stress Scales in depressed and anxious patients. J Psychopathol Behav Assess 2001; 23 (01) 61-67
• 35 Crawford JR, Henry JD. The Depression Anxiety Stress Scales (DASS): normative data and latent structure in a large non-clinical sample. Br J Clin Psychol 2003; 42 (Pt 2): 111-131
• 36 Lam TP, Wong JG, Ip MS, Lam KF, Pang SL. Psychological well-being of interns in Hong Kong: what causes them stress and what helps them. Med Teach 2010; 32 (03) e120-e126
• 37 Lovibond SH, Lovibond PF. Manual for the Depression Anxiety Stress Scales. Psychology Foundation of Australia; 1996
• 38 Creamer M, Bell R, Failla S. Psychometric properties of the Impact of Event Scale—Revised. Behav Res Ther 2003; 41 (12) 1489-1496
• 39 Rash CJ, Coffey SF, Baschnagel JS, Drobes DJ, Saladin ME. Psychometric properties of the IES-R in traumatized substance dependent individuals with and without PTSD. Addict Behav 2008; 33 (08) 1039-1047
• 40 StataCorp L.. Stata Statistical Software: Release 16. College Station, TX: 2019
• 41 Salazar de Pablo G, Vaquerizo-Serrano J, Catalan A. et al. Impact of coronavirus syndromes on physical and mental health of health care workers: systematic review and meta-analysis. J Affect Disord 2020; 275: 48-57
• 42 Axisa C, Nash L, Kelly P, Willcock S. Burnout and distress in Australian physician trainees: Evaluation of a wellbeing workshop. Australas Psychiatry 2019; 27 (03) 255-261
• 43 Marchalik D, Brems J, Rodriguez A. et al. The impact of institutional factors on physician burnout: a national study of urology trainees. Urology 2019; 131: 27-35
• 44 Mahan JD. Burnout in pediatric residents and physicians: a call to action. Pediatrics 2017; 139 (03) e20164233
• 45 Levin KH, Shanafelt TD, Keran CM. et al. Burnout, career satisfaction, and well-being among US neurology residents and fellows in 2016. Neurology 2017; 89 (05) 492-501
• 46 Card AJ. Physician burnout: resilience training is only part of the solution. Ann Fam Med 2018; 16 (03) 267-270
• 47 Dyrbye LN, West CP, Satele D. et al. Burnout among U.S. medical students, residents, and early career physicians relative to the general U.S. population. Acad Med 2014; 89 (03) 443-451
• 48 Zhou AY, Panagioti M, Esmail A, Agius R, Van Tongeren M, Bower P. Factors associated with burnout and stress in trainee physicians: a systematic review and meta-analysis. JAMA Netw Open 2020; 3 (08) e2013761-e2013761
• 49 Dzau VJ, Kirch D, Nasca T. Preventing a parallel pandemic—a national strategy to protect clinicians' well-being. N Engl J Med 2020; 383 (06) 513-515
• 50 Galea S, Merchant RM, Lurie N. The mental health consequences of COVID-19 and physical distancing: the need for prevention and early intervention. JAMA Intern Med 2020; 180 (06) 817-818
• 51 National Academies of Sciences E, Medicine. Taking Action against Clinician Burnout: A Systems Approach to Professional Well-Being. Washington, D.C.: National Academies Press; 2019
• 52 Perry K, Gold S, Shearer EM. Identifying and addressing mental health providers' perceived barriers to clinical video telehealth utilization. J Clin Psychol 2020; 76 (06) 1125-1134
• 53 Kane CK, Gillis K. The use of telemedicine by physicians: still the exception rather than the rule. Health Aff (Millwood) 2018; 37 (12) 1923-1930
• 54 Maroongroge S, Smith B, Bloom ES. et al. Telemedicine for radiation oncology in a post-COVID world. Int J Radiat Oncol Biol Phys 2020; 108 (02) 407-410
• 55 CoverMyMeds. 2021 Medication Access Report. Published 2021. Accessed March 1, 2021 at: https://assets.ctfassets.net/70w6ftfzv4je/5slQIbN5JquiQpuBvRA7yA/f8b16f6a0ff1d57591f50cc63d007923/CMM_77721_MedicationAccessReport21_FINAL__1_.pdf
• 56 Kannampallil T, Ma J. Digital translucence: adapting telemedicine delivery post-COVID-19. Telemed J E Health 2020; 26 (09) 1120-1122
• 57 Romanick-Schmiedl S, Raghu G. Telemedicine—maintaining quality during times of transition. Nat Rev Dis Primers 2020; 6 (01) 1-2
• 58 Albert NM, Prasun MA. Telemedicine in heart failure during COVID-19: like it, love it or lose it?. Heart Lung 2020; 49 (06) A11-A12
• 59 Kahn JM. Virtual visits--confronting the challenges of telemedicine. N Engl J Med 2015; 372 (18) 1684-1685
• 60 Uscher-Pines L, Sousa J, Raja P, Mehrotra A, Barnett ML, Huskamp HA. Suddenly becoming a “virtual doctor”: Experiences of psychiatrists transitioning to telemedicine during the COVID-19 pandemic. Psychiatr Serv 2020; 71 (11) 1143-1150
• 61 Dorsey E, Okun MS, Bloem BR. Care, convenience, comfort, confidentiality, and contagion: the 5 C's that will shape the future of telemedicine. J Parkinsons Dis 2020; 10 (03) 893-897
• 62 Boxer RJ, Ellimoottil C. Advantages and utilization of telemedicine. mHealth 2019; 5: 12
• 63 Judson TJ, Odisho AY, Neinstein AB. et al. Rapid design and implementation of an integrated patient self-triage and self-scheduling tool for COVID-19. J Am Med Inform Assoc 2020; 27 (06) 860-866
• 64 Hilliard RW, Haskell J, Gardner RL. Are specific elements of electronic health record use associated with clinician burnout more than others?. J Am Med Inform Assoc 2020; 27 (09) 1401-1410
• 65 Williams MS. Misdiagnosis: burnout, moral injury, and implications for the electronic health record. J Am Med Inform Assoc 2021; 28 (05) 1047-1050
• 66 Carayon P, Cassel C, Dzau VJ. Improving the system to support clinician well-being and provide better patient care. JAMA 2019; 322 (22) 2165-2166
• 67 Lourie EM, Utidjian LH, Ricci MF, Webster L, Young C, Grenfell SM. Reducing electronic health record-related burnout in providers through a personalized efficiency improvement program. J Am Med Inform Assoc 2021; 28 (05) 931-937
• 68 Toll ET. The other office. J Am Med Inform Assoc 2019; 26 (01) 71-75
• 69 Aarts J, Callen J, Coiera E, Westbrook J. Information technology in health care: socio-technical approaches. Int J Med Inform 2010; 79 (06) 389-390
• 70 Amroze A, Field TS, Fouayzi H. et al. Use of electronic health record access and audit logs to identify physician actions following noninterruptive alert opening: descriptive study. JMIR Med Inform 2019; 7 (01) e12650
• 71 Rule A, Chiang MF, Hribar MR. Using electronic health record audit logs to study clinical activity: a systematic review of aims, measures, and methods. J Am Med Inform Assoc 2020; 27 (03) 480-490
• 72 Adler-Milstein J, Adelman JS, Tai-Seale M, Patel VL, Dymek C. EHR audit logs: a new goldmine for health services research?. J Biomed Inform 2020; 101: 103343
• 73 Kannampallil T, Abraham J, Lou S, Payne PRO. Conceptual considerations for using EHR-based activity logs to measure clinician burnout and its effects. J Am Med Inform Assoc 2021; 28 (05) 1032-1037

Publication History

Received: 22 January 2021

Accepted: 03 May 2021

Article published online:
02 June 2021