RSS-Feed abonnieren

DOI: 10.1055/a-2595-0415
Qualitative Verification of Machine Learning-Based Burnout Predictors in Primary Care Physicians: An Exploratory Study
Funding This research was supported by research grants from the Agency for Healthcare Research and Quality (K08 HS027837, PI: D.T.), the Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01 HD084679, PI: J.P.), and the American Medical Association's Practice Transformation Initiative (PI: T.S.).

Abstract
Background
Electronic health record (EHR) usage measures may quantify physician activity at scale and predict practice settings with a high risk for physician burnout, but their relation to experiences is poorly understood.
Objectives
This study aimed to explore the EHR-related experiences and well-being of primary care physicians in comparison to EHR usage measures identified as important for predicting burnout from a machine learning model.
Methods
Exploratory qualitative study with semi-structured interviews of primary care physicians and clinic managers from a large academic health system and its community physician partners. We included primary care clinics with high burnout scores, low burnout scores, or large changes in burnout scores between 2020 and 2022, relative to all primary care clinics in the health system. We conducted inductive and deductive coding of interview responses using a priori themes related to the machine learning model categories of patient load, documentation burden, messaging burden, orders, and physician distress and fulfillment.
Results
Interviews with 16 physicians and 4 clinic managers identified burdens related to three dominant themes: (1) messaging and documentation burdens are high and require more time than most physicians have available during standard working hours. (2) While EHR-related burdens are high they also provide patient-care benefits. (3) Turnover and insufficient staffing exacerbate time demands associated with patient load. Dimensions that are difficult to quantify, such as a perceived imbalance between job demands and individual resources, also contribute to burnout and were consistent across all themes.
Conclusion
EHR-related work burden, largely quantifiable through EHR usage measures, are major source of distress among primary care physicians. Organizational recognition of this work as well as staffing and support to predict associated work burden may increase professional fulfillment and reduce burnout among primary care physicians.
Keywords
physician burnout - machine learning - experiences - electronic health record usage measuresProtection of Human and Animal Subjects
This study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was approved by the Institutional Review Board of Stanford University (protocol #49374).
Publikationsverlauf
Eingereicht: 11. Dezember 2024
Angenommen: 21. April 2025
Accepted Manuscript online:
28. April 2025
Artikel online veröffentlicht:
05. September 2025
© 2025. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)
Georg Thieme Verlag KG
Oswald-Hesse-Straße 50, 70469 Stuttgart, Germany
-
References
- 1 Shanafelt TD, West CP, Dyrbye LN. et al. Changes in burnout and satisfaction with work-life integration in physicians during the first 2 years of the COVID-19 pandemic. Mayo Clin Proc 2022; 97 (12) 2248-2258
- 2 Sexton JB, Adair KC, Proulx J. et al. Emotional exhaustion among US health care workers before and during the COVID-19 pandemic, 2019-2021. JAMA Netw Open 2022; 5 (09) e2232748
- 3 Tawfik DS, Profit J, Morgenthaler TI. et al. Physician burnout, well-being, and work unit safety grades in relationship to reported medical errors. Mayo Clin Proc 2018; 93 (11) 1571-1580
- 4 Tawfik DS, Scheid A, Profit J. et al. Evidence relating health care provider burnout and quality of care: a systematic review and meta-analysis. Ann Intern Med 2019; 171 (08) 555-567
- 5 Shanafelt T, Goh J, Sinsky C. The business case for investing in physician well-being. JAMA Intern Med 2017; 177 (12) 1826-1832
- 6 Brown SD, Goske MJ, Johnson CM. Beyond substance abuse: stress, burnout, and depression as causes of physician impairment and disruptive behavior. J Am Coll Radiol 2009; 6 (07) 479-485
- 7 Fahrenkopf AM, Sectish TC, Barger LK. et al. Rates of medication errors among depressed and burnt out residents: prospective cohort study. BMJ 2008; 336 (7642) 488-491
- 8 Hall LH, Johnson J, Watt I, Tsipa A, O'Connor DB. Healthcare staff wellbeing, burnout, and patient safety: a systematic review. PLoS One 2016; 11 (07) e0159015
- 9 Linzer M, Manwell LB, Williams ES. et al; MEMO (Minimizing Error, Maximizing Outcome) Investigators. Working conditions in primary care: physician reactions and care quality. Ann Intern Med 2009; 151 (01) 28-36 , W6-9
- 10 Lu DW, Dresden S, McCloskey C, Branzetti J, Gisondi MA. Impact of burnout on self-reported patient care among emergency physicians. West J Emerg Med 2015; 16 (07) 996-1001
- 11 Shanafelt TD, Bradley KA, Wipf JE, Back AL. Burnout and self-reported patient care in an internal medicine residency program. Ann Intern Med 2002; 136 (05) 358-367
- 12 Tawfik DS, Sexton JB, Kan P. et al. Burnout in the neonatal intensive care unit and its relation to healthcare-associated infections. J Perinatol 2017; 37 (03) 315-320
- 13 Weigl M, Schneider A, Hoffmann F, Angerer P. Work stress, burnout, and perceived quality of care: a cross-sectional study among hospital pediatricians. Eur J Pediatr 2015; 174 (09) 1237-1246
- 14 Wen J, Cheng Y, Hu X, Yuan P, Hao T, Shi Y. Workload, burnout, and medical mistakes among physicians in China: a cross-sectional study. Biosci Trends 2016; 10 (01) 27-33
- 15 West CP, Tan AD, Habermann TM, Sloan JA, Shanafelt TD. Association of resident fatigue and distress with perceived medical errors. JAMA 2009; 302 (12) 1294-1300
- 16 Tawfik D, Bayati M, Liu J. et al. Predicting primary care physician burnout from electronic health record use measures. Mayo Clin Proc 2024; 99 (09) 1411-1421
- 17 Liu H, Lou SS, Warner BC, Harford DR, Kannampallil T, Lu C. HiPAL: A Deep Framework for Physician Burnout Prediction Using Activity Logs in Electronic Health Records. Presented at: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining; 2022 ; Washington DC, USA. https://doi.org/10.1145/3534678.3539056
- 18 Lou SS, Liu H, Warner BC, Harford D, Lu C, Kannampallil T. Predicting physician burnout using clinical activity logs: model performance and lessons learned. J Biomed Inform 2022; 127: 104015
- 19 Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc 2020; 27 (04) 531-538
- 20 Babbott S, Manwell LB, Brown R. et al. Electronic medical records and physician stress in primary care: results from the MEMO Study. J Am Med Inform Assoc 2014; 21 (e1): e100-e106
- 21 Dyrbye LN, Gordon J, O'Horo J. et al. Relationships between EHR-based audit log data and physician burnout and clinical practice process measures. Mayo Clin Proc 2023; 98 (03) 398-409
- 22 Dyrbye LN, Sotile W, Boone S. et al. A survey of U.S. physicians and their partners regarding the impact of work-home conflict. J Gen Intern Med 2014; 29 (01) 155-161
- 23 Eschenroeder HC, Manzione LC, Adler-Milstein J. et al. Associations of physician burnout with organizational electronic health record support and after-hours charting. J Am Med Inform Assoc 2021; 28 (05) 960-966
- 24 Gardner RL, Cooper E, Haskell J. et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26 (02) 106-114
- 25 Gregory ME, Russo E, Singh H. Electronic health record alert-related workload as a predictor of burnout in primary care providers. Appl Clin Inform 2017; 8 (03) 686-697
- 26 Hilliard RW, Haskell J, Gardner RL. Are specific elements of electronic health record use associated with clinician burnout more than others?. J Am Med Inform Assoc 2020; 27 (09) 1401-1410
- 27 Kroth PJ, Morioka-Douglas N, Veres S. et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (08) e199609
- 28 Linzer M, Poplau S, Grossman E. et al. A cluster randomized trial of interventions to improve work conditions and clinician burnout in primary care: results from the healthy work place (HWP) study. J Gen Intern Med 2015; 30 (08) 1105-1111
- 29 McPeek-Hinz E, Boazak M, Sexton JB. et al. Clinician burnout associated with sex, clinician type, work culture, and use of electronic health records. JAMA Netw Open 2021; 4 (04) e215686
- 30 Melnick ER, Dyrbye LN, Sinsky CA. et al. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc 2020; 95 (03) 476-487
- 31 Peccoralo LA, Kaplan CA, Pietrzak RH, Charney DS, Ripp JA. The impact of time spent on the electronic health record after work and of clerical work on burnout among clinical faculty. J Am Med Inform Assoc 2021; 28 (05) 938-947
- 32 Rassolian M, Peterson LE, Fang B. et al. Workplace factors associated with burnout of family physicians. JAMA Intern Med 2017; 177 (07) 1036-1038
- 33 Robertson SL, Robinson MD, Reid A. Electronic health record effects on work-life balance and burnout within the I3 population collaborative. J Grad Med Educ 2017; 9 (04) 479-484
- 34 Shah M, De Arrigunaga S, Forman LS, West M, Rowe SG, Mishuris RG. Cumulated time to chart closure: a novel electronic health record-derived metric associated with clinician burnout. JAMIA Open 2024; 7 (01) ooae009
- 35 Tawfik DS, Phibbs CS, Sexton JB. et al. Factors associated with provider burnout in the NICU. Pediatrics 2017; 139 (05) e20164134
- 36 Tawfik DS, Shanafelt TD, Dyrbye LN. et al. Personal and professional factors associated with work-life integration among US physicians. JAMA Netw Open 2021; 4 (05) e2111575
- 37 Tawfik DS, Sinha A, Bayati M. et al. Frustration with technology and its relation to emotional exhaustion among health care workers: cross-sectional observational study. J Med Internet Res 2021; 23 (07) e26817
- 38 Tran B, Lenhart A, Ross R, Dorr DA. Burnout and EHR use among academic primary care physicians with varied clinical workloads. AMIA Jt Summits Transl Sci Proc 2019; 2019: 136-144
- 39 Kang C, Sarkar IN. Interventions to reduce electronic health record-related burnout: a systematic review. Appl Clin Inform 2024; 15 (01) 10-25
- 40 Livaudais M, Deng D, Frederick T, Grey-Theriot F, Kroth PJ. Perceived value of the electronic health record and its association with physician burnout. Appl Clin Inform 2022; 13 (04) 778-784
- 41 Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007; 19 (06) 349-357
- 42 Trockel M, Bohman B, Lesure E. et al. A brief instrument to assess both burnout and professional fulfillment in physicians: reliability and validity, including correlation with self-reported medical errors, in a sample of resident and practicing physicians. Acad Psychiatry 2018; 42 (01) 11-24
- 43 Creswell JW, Creswell JD. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. SAGE Publications; 2017
- 44 Campbell JL, Quincy C, Osserman J, Pedersen OK. Coding in-depth semistructured interviews: problems of unitization and intercoder reliability and agreement. Sociol Methods Res 2013; 42 (03) 294-320
- 45 Creswell JW. Qualitative Inquiry and Research Design: Choosing Among Five Approaches. SAGE Publications; 2013
- 46 Arndt BG, Beasley JW, Watkinson MD. et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med 2017; 15 (05) 419-426
- 47 Tai-Seale M, Dillon EC, Yang Y. et al. Physicians' well-being linked to in-basket messages generated by algorithms in electronic health records. Health Aff (Millwood) 2019; 38 (07) 1073-1078
- 48 Smith L, Kirk W, Bennett MM, Youens K, Ramm J. From headache to handled: advanced in-basket management system in primary care clinics reduces provider workload burden and self-reported burnout. Appl Clin Inform 2024; 15 (05) 869-876
- 49 Koranne R, Williams ES, Poplau S. et al. Reducing burnout and enhancing work engagement among clinicians: the Minnesota experience. Health Care Manage Rev 2022; 47 (01) 49-57
- 50 Johnson S, Yañez C, Panzer J, Stiehl E. Integration of primary care teams within the examination room: a qualitative study of advanced team-based care. J Ambul Care Manage 2022; 45 (01) 63-72
- 51 Rodríguez-Fernández JM, Loeb JA, Hier DB. It's time to change our documentation philosophy: writing better neurology notes without the burnout. Front Digit Health 2022; 4: 1063141
- 52 Shaw JG, Winget M, Brown-Johnson C. et al. Primary care 2.0: a prospective evaluation of a novel model of advanced team care with expanded medical assistant support. Ann Fam Med 2021; 19 (05) 411-418
- 53 Sinsky CA, Bodenheimer T. Powering-up primary care teams: advanced team care with in-room support. Ann Fam Med 2019; 17 (04) 367-371
- 54 Hung DY, Harrison MI, Truong Q, Du X. Experiences of primary care physicians and staff following lean workflow redesign. BMC Health Serv Res 2018; 18 (01) 274
- 55 Smith CD, Balatbat C, Corbridge S. et al. Implementing optimal team-based care to reduce clinician burnout. NAM Perspectives. National Academy of Medicine; 2018 https://nam.edu/implementing-optimal-team-based-care-to-reduce-clinician-burnout
- 56 Brown-Johnson CG, Chan GK, Winget M. et al. Primary care 2.0: design of a transformational team-based practice model to meet the quadruple aim. Am J Med Qual 2019; 34 (04) 339-347
- 57 Misurac J, Knake LA, Blum JM. The effect of ambient artificial intelligence notes on provider burnout. Appl Clin Inform 2025; 16 (02) 252-258
- 58 Linzer M, Poplau S, Prasad K. et al; Healthy Work Place Investigators. Characteristics of health care organizations associated with clinician trust: results from the healthy work place study. JAMA Netw Open 2019; 2 (06) e196201-e196201
- 59 Edwards ST, Marino M, Solberg LI. et al. Cultural and structural features of zero-burnout primary care practices. Health Aff (Millwood) 2021; 40 (06) 928-936
- 60 Linzer M, Sinsky CA, Poplau S, Brown R, Williams E. Healthy Work Place Investigators. Joy in medical practice: clinician satisfaction in the healthy work place trial. Health Aff (Millwood) 2017; 36 (10) 1808-1814