Subscribe to RSS
DOI: 10.1055/a-2203-3787
Interventions to Reduce Electronic Health Record-Related Burnout: A Systematic Review
- Abstract
- Background and Significance
- Objectives
- Methods
- Results
- Discussion
- Conclusion
- Clinical Relevance Statement
- Multiple-Choice Questions
- References
Abstract
Background Electronic health records are a significant contributing factor in clinician burnout, which negatively impacts patient care.
Objectives To identify and appraise published solutions that aim to reduce EHR-related burnout in clinicians.
Methods A literature search strategy was developed following the guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Six databases were searched for articles published between January 1950 and March 2023. The inclusion criteria were peer-reviewed, full-text, English language articles that described interventions targeting EHR-related burnout in any type of clinician, with reported outcomes related to burnout, wellness, EHR satisfaction, or documentation workload. Studies describing interventions without an explicit focus on reducing burnout or enhancing EHR-related satisfaction were excluded.
Results We identified 44 articles describing interventions to reduce EHR-related burnout. These interventions included the use of scribes, EHR training, and EHR modifications. These interventions were generally well received by the clinicians and patients, with subjective improvements in documentation time and EHR satisfaction, although objective data were limited.
Conclusion The findings of this review underscore the potential benefits of interventions to reduce EHR-related burnout as well as the need for further research with more robust study designs involving randomized trials, control groups, longer study durations, and validated, objective outcome measurements.
Background and Significance
Coronavirus disease 2019 (COVID-19) has led to a substantial increase in clinician burnout, characterized by emotional exhaustion, depersonalization, and a sense of reduced personal accomplishment.[1] [2] [3] [4] An oft-noted contributor to clinician burnout is electronic health records (EHRs).[5] [6]
EHR-related burnout, or the exhaustion and dissatisfaction due to interactions with EHRs, encompasses challenges such as inconsistent user interface, high volume of inbox messages, excessive data entry requirements, and lack of interoperability.[7] [8] [9] This overwhelming documentation burden, where clinicians spend excessive time on data entry and record-keeping, can result in reduced job satisfaction and increased stress[10] [11] and ultimately affect the quality of patient care.[12] [13]
Measures to address EHR-related burnout can benefit both clinicians and patients in the long run.[14] Prior reviews have focused on understanding factors contributing to clinician burnout,[15] [16] and interventions on clinician burnout in general, such as shift length changes and stress management training.[17] [18] However, there is a paucity of literature to guide interventions specifically to reduce the EHR-related burnout.
Objectives
The objective of this systematic review was to appraise the characteristics and outcomes of interventions aimed to reduce EHR-related burnout in clinicians.
Methods
In consultation with a reference librarian, a literature search strategy was developed in accordance with Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA; [Table 1]).[19] We searched the following databases for studies published between January 1950 (i.e., prior to the first published reports of EHRs) to March 2023 that met our study criteria: PubMed, MEDLINE, Embase, PsycINFO, CINAHL, and Web of Science.
Notes: The search strategy used is shown for each database searched, which accommodated for both differences in search logic syntax as well as available keyword or index terms. Date of search: March 1, 2023.
The inclusion criteria were full-text articles published in an English language peer-reviewed journal describing an intervention intended to reduce EHR-related burnout in any type of clinician with reported outcomes on burnout or related to burnout, such as stress, job satisfaction, and documentation workload. To differentiate EHR-related burnout from general burnout, we only included studies describing interventions explicitly designed to alter the way clinicians interacted with EHRs. The exclusion criteria were studies describing EHR-related interventions without explicit intent to reduce burnout, stress, and documentation workload or improve EHR-related satisfaction. We also excluded studies that we considered “wrong study design” (i.e., abstracts, conference proceedings, nonpeer-reviewed manuscripts, and non-English studies without a translation).
The retrieved eligible studies were deduplicated using EndNote (Clarivate Analytics, Philadelphia, Pennsylvania, United States) and imported the studies into the systematic review software Covidence (Melbourne, Victoria, Australia) for screening, full-text review, and data extraction. The screening and selection process is displayed in a PRISMA flowchart ([Fig. 1]). Covidence was then used to conduct title/abstract screening, full-text review, and data extraction in Covidence. The authors developed a data template in Covidence to extract relevant information, including country, study design, setting, type of clinician, number of participants, intervention type, characteristics of the intervention, duration, objective, outcome measures, and outcomes of the study.


Results
Study Characteristics
The initial search yielded 4,258 studies after deduplication; after title/abstract and full-text review, 44 studies met the eligibility criteria. The publication year of the articles ranged from 2010 to 2023. Most studies were based in the United States (93%), whereas others were based in Canada,[20] Taiwan,[21] and the United Kingdom[22] ([Table 2]). Most studies (95%) were prospective in study timing, whereas two studies (5%) were retrospective.[23] [24] The number of participants in the interventions ranged from 13 to 6,459, with a cumulative total of 14,429 participants across all studies. Most included studies were pre/poststudies (66%). Eight studies were case reports (18%),[20] [25] [26] [27] [28] [29] [30] [31] four were cohort studies (9%),[23] [32] [33] [34] two were randomized control trials (5%),[35] [36] and one was an observational study (2%).[24]
Abbreviations: APP, advanced practice provider; CNA, certified nursing assistant; MA, medical assistants; NP, nurse practitioner; NR, not reported; PA, physician assistant; RN, registered nurse.
The type of clinician in the retrieved studies varied widely and included attending physicians (84%),[20] [22] [23] [24] [25] [26] [27] [28] [29] [30] [33] [34] [35] [36] [37] [38] [39] [40] [41] [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54] [55] [56] [57] [58] [59] advanced practice providers (APPs) such as nurse practitioners and physician assistants (30%),[22] [24] [31] [34] [37] [39] [40] [43] [44] [45] [46] [50] [51] [57] [58] [60] [61] [62] resident physicians (14%),[21] [30] [32] [33] [40] [63] nurses (14%), and other clinicians such as medical assistants, certified nursing assistants, psychologists, therapists, support and recovery workers, podiatrists, and mental health practitioners (14%; [Fig. 2]).[22] [32] [39] [40] [45] [60] Eighteen studies (41%) used interventions aimed at multiple categories of clinicians.


The study setting in the retrieved articles included academic or research hospitals,[20] [21] [25] [30] [32] [33] [38] [39] [45] [46] [51] [57] [59] [62] [63] general internal medicine,[24] [26] [41] [48] [50] [53] [57] [60] oncology,[23] [27] [29] [43] [54] family medicine,[35] [52] [56] [61] urology,[36] [47] general surgery,[34] dermatology,[44] and radiology practices,[42] large health networks and systems,[28] [37] [40] [49] [55] and community hospitals.[22] [31]
Outcome Measures
Fourteen studies measured the baseline burnout or wellness level of the participants as well as the outcomes of the interventions. These included the Mini-Z survey,[30] [38] [46] [48] Maslach Burnout Inventory,[50] [57] [61] Professional Fulfillment Index,[31] [48] [51] [59] American Medical Association Steps Forward Survey,[26] [48] Copenhagen Burnout Inventory,[44] and physician work–life study study,[45] and the Stanford Physician Wellness Survey.[42] [45]
For intervention satisfaction, written surveys[20] [21] [22] [23] [24] [25] [28] [29] [32] [33] [34] [35] [36] [37] [40] [41] [43] [47] [48] [49] [52] [53] [55] [56] [60] [63] was most often used, followed by Net Promoter Score (NPS),[39] [45] [57] [62] and the Technology Acceptance Model sub-survey.[45] The NPS represents the percentage of promoters (those who are likely to recommend the service) minus the percentage of detractors (those who are unlikely to recommend a service).
To measure EHR proficiency, studies used EHR metrics such as total time spent in the EHR,[21] [29] [58] [60] [63] and the User Settings Achievement Level and proficiency score.[40]
Types of Interventions
Interventions to reduce EHR-related clinician burnout were in three broad groups: (1) employment of scribes (48%),[23] [26] [27] [28] [31] [34] [35] [36] [41] [44] [46] [47] [48] [49] [50] [51] [52] [53] [56] [59] [60] (2) EHR training (36%),[20] [25] [29] [30] [33] [37] [38] [39] [40] [42] [43] [45] [55] [57] [58] [61] and (3) EHR modifications (25%)[20] [21] [22] [24] [32] [39] [54] [57] [58] [62] [63] ([Table 3]). Four studies combined both EHR training and modifications into one intervention (9%; [Fig. 3]).[20] [39] [57] [58]


Abbreviations: EHR, electronic health record; EMR, electronic medical record; NR, not reported; MBI, Maslach Burnout Inventory.
Scribes
The most common intervention to reduce EHR-related clinician burnout was the use of scribes. Most used in-person scribes,[23] [26] [27] [28] [31] [34] [35] [36] [41] [44] [46] [47] [49] [50] [51] [52] [53] [56] [60] whereas two studies described the use of virtual scribes.[48] [59] All studies described the use of scribes by physicians, although four studies also included APPs.[44] [46] [50] [51]
In-person scribe programs involved a scribe accompanying the clinician into the patient room and documenting the patient encounter in real time.[56] [60] For virtual scribe programs, a scribe working in a remote location would listen to the patient interaction and enter clinical information into the EHR real time or asynchronously.[48] [59] The clinicians used desktop, mobile phones, smart watch, or tablets to communicate with the scribe. The adoption of virtual scribes was driven by their suitability for practices in remote geographical areas where the cost of hiring physical scribes may be prohibitive.[59]
Studies used varying methods to train and hire the scribes. Some institutions partnered with a scribe company that provided scribes who were already trained and certified.[35] [44] [47] [48] [49] [50] [51] [53] [56] [59] [60] Other studies trained the scribes in-house,[46] utilizing premedical students,[28] [36] [52] externs,[31] or certified medical assistant or licensed practical nurses.[27]
Five studies emphasized the importance of developing a strong working relationship between the clinicians and scribes. In two studies, clinicians implemented a transition period of 1 to 2 months for quality control by providing feedback and specialty-specific training.[44] [59] In other cases, a scribe was consistently paired with the same clinician to ensure a deeper understanding of the clinician's documentation preferences.[27] [48] [50]
Clinicians generally expressed positive sentiments regarding the use of scribes, although the impact of scribes has shown mixed results. Postintervention surveys frequently revealed a downward trend in burnout metrics and an upward trend in wellness metrics.[48] [50] [59] Clinicians reported feeling less mentally burdened,[34] [56] experiencing increased joy in their practice,[28] [35] and having more time to focus on communication with their patients.[46] [49] [53] Contrastingly, studies conducted in an academic general internal medicine and dermatology clinic found that physician burnout was already low at baseline and remained unchanged after implementing scribes.[44] [53]
Additionally, two studies reported a decrease in documentation time with the implementation of scribes.[48] [51] Time logs have revealed significant reductions in time spent on the EHR, such as a 50% decrease in overall documentation time,[41] a reduction of 3 minutes and 28 seconds per patient,[60] and a reduction of 53.4 minutes in postclinic documentation time.[53]
Studies also noted an unexpected increase in clinic productivity after hiring scribes. A prospective study on use of in-person scribes in an outpatient dermatology clinic reported a 29% increase in patients seen, translating into 2.5 patients per half-day session.[44] Similarly, a study based in a general internal medicine practice found that clinicians with scribes completed more visits per hour and generated more work Relative Value Units per hour.[26]
In general, the scribe program was well-received by patients.[26] [36] [44] [47] [52] In fact, some patients felt that physicians were more attentive during visits with the scribe present.[52] [53] Studies conducted in an academic setting noted that patients are accustomed to having additional individuals, such as medical students and residents, present during their appointments, which may contribute to their accepting attitude toward the presence of a scribe.[36] [44]
Several studies have highlighted certain challenges associated with the use of scribes. One notable issue was the dropout of some participants from the program due to dissatisfaction or scribe turnover.[48] Additionally, some clinicians expressed concerns regarding minor inaccuracies in the notes generated by the scribes.[56]
Six studies emphasized that cost served as a barrier to the adoption of scribes, prompting researchers to adopt various financing models.[27] [28] [46] [48] [53] [59] In two studies, the institution covered a portion of the expenses associated with hiring scribes, whereas the remaining cost was paid by either the individual departments[46] or participating physicians.[59] In the latter study, approximately half of the physicians dropped out of the study within the first year, likely due to the costs involved and the need for continuous quality assurance. In other studies, clinicians opted to increase the number of patients they saw at the clinic to offset the costs.[27] [48] [53] In one study, a tuition-based scribe fellowship program supplied scribes to the hospital, which alleviated the financial burden on the hospital and the physicians.[28]
Electronic Health Record Training
EHR training emerged as a prevalent intervention to mitigate EHR-related burnout.[20] [25] [29] [30] [33] [37] [38] [39] [40] [42] [43] [45] [55] [57] [58] [61] In contrast to scribe programs (which are typically geared toward physicians), EHR training was often used for clinicians for various levels of training across specialties such as physicians, nurses, and APPs.
The training sessions covered a range of topics aimed at enhancing EHR satisfaction and efficiency. These topics encompassed instructions on utilizing standardized templates effectively,[25] [29] leveraging voice recognition tools,[29] managing inbox communication,[29] accessing personalized guides,[29] [40] [45] [55] and utilizing specialty-specific lists of smart tools.[55]
In the majority of EHR trainings, a combination of brief didactic sessions[40] [61] and longer individualized sessions[20] [29] [43] [61] were used. This blended approach allowed for both general knowledge transfer and personalized instruction. Alternatively, two studies opted for a more intensive training program that spanned 2 to 3 days.[40] [55]
The EHR training team varied in size from 4 to 20 people. Teams were often interdisciplinary, including roles such as chief medical information officers,[20] project manager,[20] [39] clinical informaticians,[20] [39] [45] [58] EHR analysts and trainers,[33] [39] [45] [58] and a training coordinator.[30] [38]
Seven studies describing EHR training emphasized the importance of individualization in their approaches.[30] [37] [42] [45] [55] [58] [61] To tailor the learning experience for clinicians, individualized learning plans were developed based on three key inputs. These plans typically involved a need assessment survey, vendor-generated EHR report, and an observation session, in which an informatician shadowed the provider during clinical care.[38] [43]
Researchers employed various strategies to promote participation in their studies. Two studies used protected time and leveraged existing timeslots during divisional meetings.[20] [29] Other studies provided nominal financial incentive for all departments with a high participation rate.[30] [38] In two cases, participants had the opportunity to earn continuing medical education (CME) hours and quality improvement maintenance of certification credits.[33] [40]
EHR training studies reported qualitative improvements in clinicians' efficiency and satisfaction with EHR. Participants often reported a subjective increase in EHR proficiency,[37] [38] [40] [43] [45] [61] which was sustained at 6-month postintervention in one study.[40] Other studies found that participants felt less feelings of burnout after the training.[39] [45] [58] [61]
The impact of training on the time spent on documentation varied among the studies. Four studies reported a decrease in documentation time,[29] [37] [43] [45] ranging from 8.9[29] to 20 minutes saved per day.[58] Conversely, other studies found no notable change in documentation time.[38] Interestingly, Johnson and Roth noted that while subjective EHR proficiency increased, quantitative efficiency metrics worsened, although statistically nonsignificant, which could possibly be attributed to the placebo effect.[61]
Even if EHR training does not result in statistically significant changes in EHR log data, improved perceptions of the EHR are still noteworthy. Such improvements suggest that EHR training can have a positive impact on clinician burnout by streamlining the management of EHR limitations and reducing user frustration.
Electronic Health Record Modifications
Implementing EHR enhancements emerged as another strategy to mitigate EHR-related burnout.[20] [21] [22] [24] [32] [39] [54] [57] [58] [62] [63] Some interventions focused on individual modifications such as creating a data entry automation technology,[32] revising EHR forms and workflow,[21] [22] [54] [63] and decreasing low-value inbox notifications.[24] In other cases, an EHR “Sprint” or “SWAT” team were established to resolve EHR-related requests in a timely manner.[20] [39] [57] [58] Common requests included keyword search functionality, minimizing freezing, and autofaxing.
Various positive outcomes have been associated with modifications in the EHR, such as an increase in subjective EHR usability and satisfaction.[21] [32] Additionally, researchers have noted a decrease in the time spent on documentation,[32] with reductions ranging from 18.5[63] to 60%.[21] EHR modifications have also been linked to higher documentation completion rates[22] [54] and an improvement in the quality of documentation, leading to a decrease in data errors to nearly 0%.[32]
Despite these benefits, EHR modifications did not consistently result in a significant reduction in burnout. For instance, an observational study that aimed to reduce low-priority notifications within the Veterans Health Administration found that although the initiative decreased daily inbox notifications by 5.9%, it did not result in a significant change in physician burnout.[24] Similarly, a 2-week sprint program at the University of Colorado Health did not yield significant changes in the metrics of emotional thriving, emotional recovery, and emotional exhaustion.[62] These findings suggest that while EHR enhancements can improve various aspects of the health care workflow, they may not directly address the usability defects of the EHR itself contributing to clinician burnout.
Discussion
We conducted a systematic review of 44 studies describing interventions aimed at reducing EHR-related burnout. These interventions, including scribe utilization, EHR training, and EHR modifications, were implemented in diverse academic, research, and subspecialty clinics catering to clinicians from various backgrounds. The review highlights the significant burden of EHR-related burnout experienced by clinicians across different fields and the growing interest in addressing this issue. Subjective findings indicate potential benefits for participants, such as reduced documentation time and increased EHR satisfaction, while objective data remain contradictory or limited. Nonetheless, the study emphasizes the potential of these interventions to reduce burnout and emphasizes the need for further research to establish stronger evidence.
The literature suggests that the three primary intervention types—scribe employment, EHR training, and EHR modifications—may address specific aspects of EHRs and their impact on clinicians' workload and burnout. For instance, the employment of scribes may alleviate EHR-related burnout by reducing the documentation burden on clinicians by capturing patient information more efficiently, allowing clinicians to focus more on patient care.[59] [64] EHR training can help improve clinicians' proficiency in using EHRs, leading to reduced frustration and stress.[39] [65] Additionally, EHR modifications, by streamlining EHR systems to better align with the workflow, may reduce the time and effort required for data entry, mitigating sources of frustration associated with EHR usage.[66]
We identified several gaps in the literature on interventions intended to reduce EHR-related burnout in clinicians. First, there is a lack of standardized assessment to measure EHR proficiency, satisfaction, and clinician burnout, making it difficult to distinguish between perceived and objective improvements postintervention.[67] [68] Some studies used surveys that were not pretested or validated, whereas other studies modified the surveys during the intervention, making it difficult to compare results. Other limitations were related to study design such as small sample size, low participation and response rate, high attrition rate, lack of control groups, and short duration of the intervention.[69] Enrollment was also disrupted by the COVID-19 pandemic, rotating nature of residency programs, and rolling enrollment policies. As a result, it is hard to determine the scalability or generalizability of such interventions.
This review included articles identified in six literature sources. The databases included in this study reflected broad coverage of literature (Ovid MEDLINE and Embase), psychological and mental health research (PsycINFO), as well as allied health and nursing (CINAHL), as well as search tools that complemented the search interfaces for each of the literature databases (PubMed and Web of Science). To bolster the evidence base for interventions targeting EHR-related burnout, future studies should employ stronger research designs, including randomized trials, control groups, longer study durations, and validated, objective outcome measurements.[69] Further research is needed to answer several key questions such as investigating the sustained impact of interventions on EHR time and clinician wellness, assessing variations in effectiveness across clinical settings (academic, research, and subspecialty clinics), and validating subjective findings with more objective and quantifiable data. A thorough cost–benefit analysis can also help clinicians understand the economic feasibility and potential savings associated with these interventions.
It is important to note that this study did not include studies published in gray literature or in languages other than English. By not incorporating these types of studies, we may have missed some important findings from operational settings not reported in peer-reviewed literature.
Conclusion
There were three main types of interventions that hold promise in reducing EHR-related burnout among clinicians: employment of scribes, EHR training, and EHR modifications. Our findings suggests that while interventions often yield positive outcomes such as increased EHR satisfaction and reduced documentation time, addressing the burnout directly requires a more comprehensive approach. Factors contributing to burnout extend beyond EHR systems and encompass workload, organizational culture, and work–life balance. Therefore, although interventions may positively impact certain aspects of clinicians' experience, they may not directly translate into a reduction in burnout. Long-term, large-scale studies with robust study designs need to be conducted to gain a better understanding of the sustained effects of interventions and their impact on burnout.
Clinical Relevance Statement
The review's findings suggest promising interventions, such as employing scribes, providing EHR training, and implementing EHR modifications, for reducing EHR-related burnout among clinicians. Researchers, policy makers, and administrators should adopt a comprehensive approach to address the multifaceted nature of burnout, which include factors beyond EHR systems, such as workload, organizational culture, and work–life balance.
Multiple-Choice Questions
-
What intervention is mostly geared toward physicians?
-
Scribes
-
EHR training
-
EHR modifications
-
All of the above
Correct Answer: The correct answer is option a. Scribes were primarily utilized to support physicians, whereas EHR training and EHR modifications were implemented to support clinicians from diverse backgrounds and varying levels of training.
-
-
What is one of the methods researchers used to promote participation in EHR training?
-
Making participation mandatory
-
Awarding CME hours
-
Offering social activities as part of the training
-
Providing fee merchandise and food
Correct Answer: The correct answer is option b. Researchers adopted various approaches to encourage clinicians to participate in EHR training such as using protected time, offered financial incentives, awarding CME hours and quality improvement maintenance of certification credits.
-
Conflict of Interest
None declared.
Acknowledgment
We thank Andrew Creamer from Brown University for drafting and conducting the search strategy for this project.
Protection of Human and Animal Subjects
Human and/or animal subjects were not included in the project.
-
References
- 1 Shanafelt TD, Boone S, Tan L. et al. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med 2012; 172 (18) 1377-1385
- 2 Alkhamees AA, Aljohani MS, Kalani S. et al. Physician's burnout during the COVID-19 pandemic: a systematic review and meta-analysis. Int J Environ Res Public Health 2023; 20 (05) 4598
- 3 Galanis P, Vraka I, Fragkou D, Bilali A, Kaitelidou D. Nurses' burnout and associated risk factors during the COVID-19 pandemic: a systematic review and meta-analysis. J Adv Nurs 2021; 77 (08) 3286-3302
- 4 Patel RS, Bachu R, Adikey A, Malik M, Shah M. Factors related to physician burnout and its consequences: a review. Behav Sci (Basel) 2018; 8 (11) 98
- 5 Frintner MP, Kaelber DC, Kirkendall ES, Lourie EM, Somberg CA, Lehmann CU. The effect of electronic health record burden on pediatricians' work-life balance and career satisfaction. Appl Clin Inform 2021; 12 (03) 697-707
- 6 Gesner E, Dykes PC, Zhang L, Gazarian P. Documentation burden in nursing and its role in clinician burnout syndrome. Appl Clin Inform 2022; 13 (05) 983-990
- 7 Muhiyaddin R, Elfadl A, Mohamed E. et al. Electronic health records and physician burnout: a scoping review. Stud Health Technol Inform 2022; 289: 481-484
- 8 Tran B, Lenhart A, Ross R, Dorr DA. Burnout and EHR use among academic primary care physicians with varied clinical workloads. AMIA Jt Summits Transl Sci Proc 2019; 2019: 136-144
- 9 Kroth PJ, Morioka-Douglas N, Veres S. et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (08) e199609
- 10 Moy AJ, Schwartz JM, Chen R. et al. Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review. J Am Med Inform Assoc 2021; 28 (05) 998-1008
- 11 Brady KJS, Trockel MT, Khan CT. et al. What do we mean by physician wellness? A systematic review of its definition and measurement. Acad Psychiatry 2018; 42 (01) 94-108
- 12 Baxter SL, Gali HE, Chiang MF. et al. Promoting quality face-to-face communication during ophthalmology encounters in the electronic health record era. Appl Clin Inform 2020; 11 (01) 130-141
- 13 Sinsky C, Colligan L, Li L. et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016; 165 (11) 753-760
- 14 Hobensack M, Levy DR, Cato K. et al. 25 × 5 symposium to reduce documentation burden: report-out and call for action. Appl Clin Inform 2022; 13 (02) 439-446
- 15 Yan Q, Jiang Z, Harbin Z, Tolbert PH, Davies MG. Exploring the relationship between electronic health records and provider burnout: a systematic review. J Am Med Inform Assoc 2021; 28 (05) 1009-1021
- 16 West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med 2018; 283 (06) 516-529
- 17 West CP, Dyrbye LN, Erwin PJ, Shanafelt TD. Interventions to prevent and reduce physician burnout: a systematic review and meta-analysis. Lancet 2016; 388 (10057): 2272-2281
- 18 Aryankhesal A, Mohammadibakhsh R, Hamidi Y. et al. Interventions on reducing burnout in physicians and nurses: a systematic review. Med J Islam Repub Iran 2019; 33: 77
- 19
Page MJ,
McKenzie JE,
Bossuyt PM.
et al.
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
BMJ 2021; 372 (71) n71
MissingFormLabel
- 20 Sequeira L, Almilaji K, Strudwick G, Jankowicz D, Tajirian T. EHR “SWAT” teams: a physician engagement initiative to improve electronic health record (EHR) experiences and mitigate possible causes of EHR-related burnout. JAMIA Open 2021; 4 (02) ooab018
- 21 Hsieh H-Y, Henker R, Ren D. et al. Improving effectiveness and satisfaction of an electronic charting system in Taiwan. Clin Nurse Spec 2016; 30 (06) E1-E6
- 22 Buivydaite R, Reen G, Kovalevica T. et al. Improving usability of electronic health records in a UK mental health setting: a feasibility study. J Med Syst 2022; 46 (07) 50
- 23 Gao RW, Dugala A, Maxwell J. et al. Effect of medical scribes on outpatient oncology visits at a multidisciplinary cancer center. JCO Oncol Pract 2020; 16 (02) e139-e147
- 24 O'Connor AW, Helfrich CD, Nelson KM, Sears JM, Singh H, Wong ES. Changes in electronic notification volume and primary care provider burnout. Am J Manag Care 2023; 29 (01) 57-63
- 25 Day MA, Belden JL. EMR happy hour: new approach to electronic medical record continuous learning. Ann Fam Med 2019; 17 (04) 373
- 26 Heckman J, Mukamal KJ, Christensen A, Reynolds EE. Medical scribes, provider and patient experience, and patient throughput: a trial in an academic general internal medicine practice. J Gen Intern Med 2020; 35 (03) 770-774
- 27 Hindman A. Utilizing scribes to improve patient-centered care and efficiency and reduce burnout. Oncol Issues 2019; 34 (04) 46-50
- 28 Lin S, Duong A, Nguyen C, Teng V. Five years' experience with a medical scribe fellowship: shaping future health professions students while addressing provider burnout. Acad Med 2021; 96 (05) 671-679
- 29 Livingston K, Bovi J. Department-focused electronic health record thrive training. JAMIA Open 2022; 5 (02) ooac025
- 30 Stevens LA, DiAngi YT, Schremp JD. et al. Designing an individualized EHR learning plan for providers. Appl Clin Inform 2017; 8 (03) 924-935
- 31 Hartman-Hall H, Kanwal A, Jory L. et al. Scribes with PGY-1 residents on inpatient medicine teams: effect on time spent in meaningful work. J Community Hosp Intern Med Perspect 2023; 13 (01) 25-29
- 32 Bauer JC, John E, Wood CL, Plass D, Richardson D. Data entry automation improves cost, quality, performance, and job satisfaction in a hospital nursing unit. J Nurs Adm 2020; 50 (01) 34-39
- 33 Scott K, Hathaway E, Sharp K, Smailes P. The development and evaluation of an electronic health record efficiency workshop for providers. Appl Clin Inform 2020; 11 (02) 336-341
- 34 Tohmasi S, Naaseh A, Thompson S, Smith BR. Improved perceptions of education and wellness among general surgery residents and faculty after the implementation of outpatient scribes. Am Surg 2021; 87 (10) 1616-1620
- 35 Gidwani R, Nguyen C, Kofoed A. et al. Impact of scribes on physician satisfaction, patient satisfaction, and charting efficiency: a randomized controlled trial. Ann Fam Med 2017; 15 (05) 427-433
- 36 Koshy S, Feustel PJ, Hong M, Kogan BA. Scribes in an ambulatory urology practice: patient and physician satisfaction. J Urol 2010; 184 (01) 258-262
- 37 Dastagir MT, Chin HL, McNamara M, Poteraj K, Battaglini S, Alstot L. Advanced proficiency EHR training: effect on physicians' EHR efficiency, EHR satisfaction and job satisfaction. AMIA Annu Symp Proc 2012; 2012: 136-143
- 38 DiAngi YT, Stevens LA, Halpern-Felsher B, Pageler NM, Lee TC. Electronic health record (EHR) training program identifies a new tool to quantify the EHR time burden and improves providers' perceived control over their workload in the EHR. JAMIA Open 2019; 2 (02) 222-230
- 39 English EF, Holmstrom H, Kwan BW. et al. Virtual sprint outpatient electronic health record training and optimization effect on provider burnout. Appl Clin Inform 2022; 13 (01) 10-18
- 40 Gordon JE, Belford SM, Aranguren DL. et al. Outcomes of Mayo Clinic reBoot camps for postimplementation training in the electronic health record. J Am Med Inform Assoc 2022; 29 (09) 1518-1524
- 41 Imdieke BH, Martel ML. Integration of medical scribes in the primary care setting: improving satisfaction. J Ambul Care Manage 2017; 40 (01) 17-25
- 42 Ip IK, Giess CS, Gupte A. et al. A prospective intervention to reduce burnout among academic radiologists. Acad Radiol 2023; 30 (06) 1024-1030
- 43 Kadish SS, Mayer EL, Jackman DM. et al. Implementation to optimization: a tailored, data-driven approach to improve provider efficiency and confidence in use of the electronic medical record. J Oncol Pract 2018; 14 (07) e421-e428
- 44 Lam C, Shumaker K, Butt M, Leiphart P, Miller JJ, Anderson BE. Impact of medical scribes on physician and patient satisfaction in dermatology. Arch Dermatol Res 2022; 314 (01) 71-76
- 45 Lourie EM, Utidjian LH, Ricci MF, Webster L, Young C, Grenfell SM. Reducing electronic health record-related burnout in providers through a personalized efficiency improvement program. J Am Med Inform Assoc 2021; 28 (05) 931-937
- 46 Martel ML, Imdieke BH, Holm KM. et al. Developing a medical scribe program at an academic hospital: the Hennepin County Medical Center experience. Jt Comm J Qual Patient Saf 2018; 44 (05) 238-249
- 47 McCormick BJ, Deal A, Borawski KM. et al. Implementation of medical scribes in an academic urology practice: an analysis of productivity, revenue, and satisfaction. World J Urol 2018; 36 (10) 1691-1697
- 48 Micek MA, Arndt B, Baltus JJ. et al. The effect of remote scribes on primary care physicians' wellness, EHR satisfaction, and EHR use. Healthc (Amst) 2022; 10 (04) 100663
- 49 Mishra P, Kiang JC, Grant RW. Association of medical scribes in primary care with physician workflow and patient experience. JAMA Intern Med 2018; 178 (11) 1467-1472
- 50 Morawski K, Childs-Roshak J, Weitberg A. Scribes: re-writing the story on patient and provider experience. Healthc (Amst) 2017; 5 (03) 95-97
- 51 Pfoh ER, Hong S, Baranek L. et al. Reduced cognitive burden and increased focus: a mixed-methods study exploring how implementing scribes impacted physicians. Med Care 2022; 60 (04) 316-320
- 52 Platt J, Altman W. Can medical scribes improve quality measure documentation?. J Fam Pract 2019; 68 (05) E1-E7
- 53 Pozdnyakova A, Laiteerapong N, Volerman A. et al. Impact of medical scribes on physician and patient satisfaction in primary care. J Gen Intern Med 2018; 33 (07) 1109-1115
- 54 Raney L, McManaman J, Elsaid M. et al. Multisite quality improvement initiative to repair incomplete electronic medical record documentation as one of many causes of provider burnout. JCO Oncol Pract 2020; 16 (11) e1412-e1416
- 55 Robinson KE, Kersey JA. Novel electronic health record (EHR) education intervention in large healthcare organization improves quality, efficiency, time, and impact on burnout. Medicine (Baltimore) 2018; 97 (38) e12319
- 56 Sattler A, Rydel T, Nguyen C, Lin S. One year of family physicians' observations on working with medical scribes. J Am Board Fam Med 2018; 31 (01) 49-56
- 57 Sieja A, Markley K, Pell J. et al. Optimization sprints: improving clinician satisfaction and teamwork by rapidly reducing electronic health record burden. Mayo Clin Proc 2019; 94 (05) 793-802
- 58 Sieja A, Whittington MD, Patterson VP. et al. The influence of a Sprint optimization and training intervention on time spent in the electronic health record (EHR). JAMIA Open 2021; 4 (03) ooab073
- 59 Stephens J, Kieber-Emmons AM, Johnson M, Greenberg GM. Implementation of a virtual asynchronous scribe program to reduce physician burnout. J Healthc Manag 2022; 67 (06) 425-435
- 60 Jhaveri P, Abdulahad D, Fogel B. et al. Impact of scribe intervention on documentation in an outpatient pediatric primary care practice. Acad Pediatr 2022; 22 (02) 289-295
- 61 Johnson E, Roth E. Improving physician wellness through electronic health record education. Int J Psychiatry Med 2021; 56 (05) 327-333
- 62 Simpson JR, Lin CT, Sieja A, Sillau SH, Pell J. Optimizing the electronic health record: an inpatient sprint addresses provider burnout and improves electronic health record satisfaction. J Am Med Inform Assoc 2021; 28 (03) 628-631
- 63 Lindsay MR, Lytle K. Implementing best practices to redesign workflow and optimize nursing documentation in the electronic health record. Appl Clin Inform 2022; 13 (03) 711-719
- 64 Yan C, Rose S, Rothberg MB, Mercer MB, Goodman K, Misra-Hebert AD. Physician, scribe, and patient perspectives on clinical scribes in primary care. J Gen Intern Med 2016; 31 (09) 990-995
- 65 Kruse CS, Mileski M, Dray G, Johnson Z, Shaw C, Shirodkar H. Physician burnout and the electronic health record leading up to and during the first year of COVID-19: systematic review. J Med Internet Res 2022; 24 (03) e36200
- 66 Lääveri T, Viitanen J. Physicians' perspectives on EHR usability: results from four large cross-sectional surveys from 2010 to 2021. Stud Health Technol Inform 2023; 304: 16-20
- 67 Ahmed I, Ishtiaq S. Reliability and validity: importance in medical research. J Pak Med Assoc 2021; 71 (10) 2401-2406
- 68 Knox M, Willard-Grace R, Huang B, Grumbach K. Maslach Burnout Inventory and a self-defined, single-item burnout measure produce different clinician and staff burnout estimates. J Gen Intern Med 2018; 33 (08) 1344-1351
- 69 Aggarwal R, Ranganathan P. Study designs: part 4 - interventional studies. Perspect Clin Res 2019; 10 (03) 137-139
Address for correspondence
Publication History
Received: 10 August 2023
Accepted: 02 November 2023
Accepted Manuscript online:
03 November 2023
Article published online:
03 January 2024
© 2024. Thieme. All rights reserved.
Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany
-
References
- 1 Shanafelt TD, Boone S, Tan L. et al. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med 2012; 172 (18) 1377-1385
- 2 Alkhamees AA, Aljohani MS, Kalani S. et al. Physician's burnout during the COVID-19 pandemic: a systematic review and meta-analysis. Int J Environ Res Public Health 2023; 20 (05) 4598
- 3 Galanis P, Vraka I, Fragkou D, Bilali A, Kaitelidou D. Nurses' burnout and associated risk factors during the COVID-19 pandemic: a systematic review and meta-analysis. J Adv Nurs 2021; 77 (08) 3286-3302
- 4 Patel RS, Bachu R, Adikey A, Malik M, Shah M. Factors related to physician burnout and its consequences: a review. Behav Sci (Basel) 2018; 8 (11) 98
- 5 Frintner MP, Kaelber DC, Kirkendall ES, Lourie EM, Somberg CA, Lehmann CU. The effect of electronic health record burden on pediatricians' work-life balance and career satisfaction. Appl Clin Inform 2021; 12 (03) 697-707
- 6 Gesner E, Dykes PC, Zhang L, Gazarian P. Documentation burden in nursing and its role in clinician burnout syndrome. Appl Clin Inform 2022; 13 (05) 983-990
- 7 Muhiyaddin R, Elfadl A, Mohamed E. et al. Electronic health records and physician burnout: a scoping review. Stud Health Technol Inform 2022; 289: 481-484
- 8 Tran B, Lenhart A, Ross R, Dorr DA. Burnout and EHR use among academic primary care physicians with varied clinical workloads. AMIA Jt Summits Transl Sci Proc 2019; 2019: 136-144
- 9 Kroth PJ, Morioka-Douglas N, Veres S. et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (08) e199609
- 10 Moy AJ, Schwartz JM, Chen R. et al. Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review. J Am Med Inform Assoc 2021; 28 (05) 998-1008
- 11 Brady KJS, Trockel MT, Khan CT. et al. What do we mean by physician wellness? A systematic review of its definition and measurement. Acad Psychiatry 2018; 42 (01) 94-108
- 12 Baxter SL, Gali HE, Chiang MF. et al. Promoting quality face-to-face communication during ophthalmology encounters in the electronic health record era. Appl Clin Inform 2020; 11 (01) 130-141
- 13 Sinsky C, Colligan L, Li L. et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016; 165 (11) 753-760
- 14 Hobensack M, Levy DR, Cato K. et al. 25 × 5 symposium to reduce documentation burden: report-out and call for action. Appl Clin Inform 2022; 13 (02) 439-446
- 15 Yan Q, Jiang Z, Harbin Z, Tolbert PH, Davies MG. Exploring the relationship between electronic health records and provider burnout: a systematic review. J Am Med Inform Assoc 2021; 28 (05) 1009-1021
- 16 West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med 2018; 283 (06) 516-529
- 17 West CP, Dyrbye LN, Erwin PJ, Shanafelt TD. Interventions to prevent and reduce physician burnout: a systematic review and meta-analysis. Lancet 2016; 388 (10057): 2272-2281
- 18 Aryankhesal A, Mohammadibakhsh R, Hamidi Y. et al. Interventions on reducing burnout in physicians and nurses: a systematic review. Med J Islam Repub Iran 2019; 33: 77
- 19
Page MJ,
McKenzie JE,
Bossuyt PM.
et al.
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
BMJ 2021; 372 (71) n71
MissingFormLabel
- 20 Sequeira L, Almilaji K, Strudwick G, Jankowicz D, Tajirian T. EHR “SWAT” teams: a physician engagement initiative to improve electronic health record (EHR) experiences and mitigate possible causes of EHR-related burnout. JAMIA Open 2021; 4 (02) ooab018
- 21 Hsieh H-Y, Henker R, Ren D. et al. Improving effectiveness and satisfaction of an electronic charting system in Taiwan. Clin Nurse Spec 2016; 30 (06) E1-E6
- 22 Buivydaite R, Reen G, Kovalevica T. et al. Improving usability of electronic health records in a UK mental health setting: a feasibility study. J Med Syst 2022; 46 (07) 50
- 23 Gao RW, Dugala A, Maxwell J. et al. Effect of medical scribes on outpatient oncology visits at a multidisciplinary cancer center. JCO Oncol Pract 2020; 16 (02) e139-e147
- 24 O'Connor AW, Helfrich CD, Nelson KM, Sears JM, Singh H, Wong ES. Changes in electronic notification volume and primary care provider burnout. Am J Manag Care 2023; 29 (01) 57-63
- 25 Day MA, Belden JL. EMR happy hour: new approach to electronic medical record continuous learning. Ann Fam Med 2019; 17 (04) 373
- 26 Heckman J, Mukamal KJ, Christensen A, Reynolds EE. Medical scribes, provider and patient experience, and patient throughput: a trial in an academic general internal medicine practice. J Gen Intern Med 2020; 35 (03) 770-774
- 27 Hindman A. Utilizing scribes to improve patient-centered care and efficiency and reduce burnout. Oncol Issues 2019; 34 (04) 46-50
- 28 Lin S, Duong A, Nguyen C, Teng V. Five years' experience with a medical scribe fellowship: shaping future health professions students while addressing provider burnout. Acad Med 2021; 96 (05) 671-679
- 29 Livingston K, Bovi J. Department-focused electronic health record thrive training. JAMIA Open 2022; 5 (02) ooac025
- 30 Stevens LA, DiAngi YT, Schremp JD. et al. Designing an individualized EHR learning plan for providers. Appl Clin Inform 2017; 8 (03) 924-935
- 31 Hartman-Hall H, Kanwal A, Jory L. et al. Scribes with PGY-1 residents on inpatient medicine teams: effect on time spent in meaningful work. J Community Hosp Intern Med Perspect 2023; 13 (01) 25-29
- 32 Bauer JC, John E, Wood CL, Plass D, Richardson D. Data entry automation improves cost, quality, performance, and job satisfaction in a hospital nursing unit. J Nurs Adm 2020; 50 (01) 34-39
- 33 Scott K, Hathaway E, Sharp K, Smailes P. The development and evaluation of an electronic health record efficiency workshop for providers. Appl Clin Inform 2020; 11 (02) 336-341
- 34 Tohmasi S, Naaseh A, Thompson S, Smith BR. Improved perceptions of education and wellness among general surgery residents and faculty after the implementation of outpatient scribes. Am Surg 2021; 87 (10) 1616-1620
- 35 Gidwani R, Nguyen C, Kofoed A. et al. Impact of scribes on physician satisfaction, patient satisfaction, and charting efficiency: a randomized controlled trial. Ann Fam Med 2017; 15 (05) 427-433
- 36 Koshy S, Feustel PJ, Hong M, Kogan BA. Scribes in an ambulatory urology practice: patient and physician satisfaction. J Urol 2010; 184 (01) 258-262
- 37 Dastagir MT, Chin HL, McNamara M, Poteraj K, Battaglini S, Alstot L. Advanced proficiency EHR training: effect on physicians' EHR efficiency, EHR satisfaction and job satisfaction. AMIA Annu Symp Proc 2012; 2012: 136-143
- 38 DiAngi YT, Stevens LA, Halpern-Felsher B, Pageler NM, Lee TC. Electronic health record (EHR) training program identifies a new tool to quantify the EHR time burden and improves providers' perceived control over their workload in the EHR. JAMIA Open 2019; 2 (02) 222-230
- 39 English EF, Holmstrom H, Kwan BW. et al. Virtual sprint outpatient electronic health record training and optimization effect on provider burnout. Appl Clin Inform 2022; 13 (01) 10-18
- 40 Gordon JE, Belford SM, Aranguren DL. et al. Outcomes of Mayo Clinic reBoot camps for postimplementation training in the electronic health record. J Am Med Inform Assoc 2022; 29 (09) 1518-1524
- 41 Imdieke BH, Martel ML. Integration of medical scribes in the primary care setting: improving satisfaction. J Ambul Care Manage 2017; 40 (01) 17-25
- 42 Ip IK, Giess CS, Gupte A. et al. A prospective intervention to reduce burnout among academic radiologists. Acad Radiol 2023; 30 (06) 1024-1030
- 43 Kadish SS, Mayer EL, Jackman DM. et al. Implementation to optimization: a tailored, data-driven approach to improve provider efficiency and confidence in use of the electronic medical record. J Oncol Pract 2018; 14 (07) e421-e428
- 44 Lam C, Shumaker K, Butt M, Leiphart P, Miller JJ, Anderson BE. Impact of medical scribes on physician and patient satisfaction in dermatology. Arch Dermatol Res 2022; 314 (01) 71-76
- 45 Lourie EM, Utidjian LH, Ricci MF, Webster L, Young C, Grenfell SM. Reducing electronic health record-related burnout in providers through a personalized efficiency improvement program. J Am Med Inform Assoc 2021; 28 (05) 931-937
- 46 Martel ML, Imdieke BH, Holm KM. et al. Developing a medical scribe program at an academic hospital: the Hennepin County Medical Center experience. Jt Comm J Qual Patient Saf 2018; 44 (05) 238-249
- 47 McCormick BJ, Deal A, Borawski KM. et al. Implementation of medical scribes in an academic urology practice: an analysis of productivity, revenue, and satisfaction. World J Urol 2018; 36 (10) 1691-1697
- 48 Micek MA, Arndt B, Baltus JJ. et al. The effect of remote scribes on primary care physicians' wellness, EHR satisfaction, and EHR use. Healthc (Amst) 2022; 10 (04) 100663
- 49 Mishra P, Kiang JC, Grant RW. Association of medical scribes in primary care with physician workflow and patient experience. JAMA Intern Med 2018; 178 (11) 1467-1472
- 50 Morawski K, Childs-Roshak J, Weitberg A. Scribes: re-writing the story on patient and provider experience. Healthc (Amst) 2017; 5 (03) 95-97
- 51 Pfoh ER, Hong S, Baranek L. et al. Reduced cognitive burden and increased focus: a mixed-methods study exploring how implementing scribes impacted physicians. Med Care 2022; 60 (04) 316-320
- 52 Platt J, Altman W. Can medical scribes improve quality measure documentation?. J Fam Pract 2019; 68 (05) E1-E7
- 53 Pozdnyakova A, Laiteerapong N, Volerman A. et al. Impact of medical scribes on physician and patient satisfaction in primary care. J Gen Intern Med 2018; 33 (07) 1109-1115
- 54 Raney L, McManaman J, Elsaid M. et al. Multisite quality improvement initiative to repair incomplete electronic medical record documentation as one of many causes of provider burnout. JCO Oncol Pract 2020; 16 (11) e1412-e1416
- 55 Robinson KE, Kersey JA. Novel electronic health record (EHR) education intervention in large healthcare organization improves quality, efficiency, time, and impact on burnout. Medicine (Baltimore) 2018; 97 (38) e12319
- 56 Sattler A, Rydel T, Nguyen C, Lin S. One year of family physicians' observations on working with medical scribes. J Am Board Fam Med 2018; 31 (01) 49-56
- 57 Sieja A, Markley K, Pell J. et al. Optimization sprints: improving clinician satisfaction and teamwork by rapidly reducing electronic health record burden. Mayo Clin Proc 2019; 94 (05) 793-802
- 58 Sieja A, Whittington MD, Patterson VP. et al. The influence of a Sprint optimization and training intervention on time spent in the electronic health record (EHR). JAMIA Open 2021; 4 (03) ooab073
- 59 Stephens J, Kieber-Emmons AM, Johnson M, Greenberg GM. Implementation of a virtual asynchronous scribe program to reduce physician burnout. J Healthc Manag 2022; 67 (06) 425-435
- 60 Jhaveri P, Abdulahad D, Fogel B. et al. Impact of scribe intervention on documentation in an outpatient pediatric primary care practice. Acad Pediatr 2022; 22 (02) 289-295
- 61 Johnson E, Roth E. Improving physician wellness through electronic health record education. Int J Psychiatry Med 2021; 56 (05) 327-333
- 62 Simpson JR, Lin CT, Sieja A, Sillau SH, Pell J. Optimizing the electronic health record: an inpatient sprint addresses provider burnout and improves electronic health record satisfaction. J Am Med Inform Assoc 2021; 28 (03) 628-631
- 63 Lindsay MR, Lytle K. Implementing best practices to redesign workflow and optimize nursing documentation in the electronic health record. Appl Clin Inform 2022; 13 (03) 711-719
- 64 Yan C, Rose S, Rothberg MB, Mercer MB, Goodman K, Misra-Hebert AD. Physician, scribe, and patient perspectives on clinical scribes in primary care. J Gen Intern Med 2016; 31 (09) 990-995
- 65 Kruse CS, Mileski M, Dray G, Johnson Z, Shaw C, Shirodkar H. Physician burnout and the electronic health record leading up to and during the first year of COVID-19: systematic review. J Med Internet Res 2022; 24 (03) e36200
- 66 Lääveri T, Viitanen J. Physicians' perspectives on EHR usability: results from four large cross-sectional surveys from 2010 to 2021. Stud Health Technol Inform 2023; 304: 16-20
- 67 Ahmed I, Ishtiaq S. Reliability and validity: importance in medical research. J Pak Med Assoc 2021; 71 (10) 2401-2406
- 68 Knox M, Willard-Grace R, Huang B, Grumbach K. Maslach Burnout Inventory and a self-defined, single-item burnout measure produce different clinician and staff burnout estimates. J Gen Intern Med 2018; 33 (08) 1344-1351
- 69 Aggarwal R, Ranganathan P. Study designs: part 4 - interventional studies. Perspect Clin Res 2019; 10 (03) 137-139





