Explaining Variability in Electronic Health Record Effort in Primary Care Ambulatory Encounters

Abstract

Background Electronic health record (EHR) user interface event logs are fast providing another perspective on the value and efficiency EHR technology brings to health care. Analysis of these detailed usage data has demonstrated their potential to identify EHR and clinical process design factors related to user efficiency, satisfaction, and burnout.

Objective This study aimed to analyze the event log data across 26 different health systems to determine the variability of use of a single vendor's EHR based on four event log metrics, at the individual, practice group, and health system levels.

Methods We obtained de-identified event log data recorded from June 1, 2018, to May 31, 2019, from 26 health systems' primary care physicians. We estimated the variability in total Active EHR Time, Documentation Time, Chart Review Time, and Ordering Time across health systems, practice groups, and individual physicians.

Results In total, 5,444 physicians (Family Medicine: 3,042 and Internal Medicine: 2,422) provided care in a total of 2,285 different practices nested in 26 health systems. Health systems explain 1.29, 3.55, 3.45, and 3.30% of the total variability in Active Time, Documentation Time, Chart Review Time, and Ordering Time, respectively. Practice-level variability was estimated to be 7.96, 13.52, 8.39, and 5.57%, respectively, and individual physicians explained the largest proportion of the variability for those same outcomes 17.09, 27.49, 17.51, and 19.75%, respectively.

Conclusion The most variable physician EHR usage patterns occurs at the individual physician level and decreases as you move up to the practice and health system levels. This suggests that interventions to improve individual users' EHR usage efficiency may have the most potential impact compared with those directed at health system or practice levels.

Keywords

documentation burden - workflows and human interactions - human–computer interaction - interfaces and usability - workarounds and unanticipated consequences - user acceptance and resistance - organizational change management

Protection of Human Subjects

This study was determined to be exempt research by the Western Michigan University Homer Stryker M.D. School of Medicine Institutional Review Board.

Publication History

Received: 02 May 2023

Accepted: 30 January 2024

Article published online:
20 March 2024

• References

• 1 Adler-Milstein J, Adelman JS, Tai-Seale M, Patel VL, Dymek C. EHR audit logs: a new goldmine for healths services research?. J Biomed Inform 2020; 101: 103343
• 2 Cohen GR, Boi J, Johnson C, Brown L, Patel V. Measuring time clinicians spend using EHRs in the inpatient setting: a national, mixed-methods study. J Am Med Inform Assoc 2021; 28 (08) 1676-1682
• 3 Rule A, Melnick ER, Apathy NC. Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures. J Am Med Inform Assoc 2022; 30 (01) 144-154
• 4 Baxter SL, Apathy NC, Cross DA, Sinsky C, Hribar MR. Measures of electronic health record use in outpatient settings across vendors. J Am Med Inform Assoc 2021; 28 (05) 955-959
• 5 Sinsky CA, Rule A, Cohen G. et al. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc 2020; 27 (04) 639-643
• 6 Munstermann B, Andreas Eckhardt A, Weitzel T. The performance impact of business process standardization. Bus Process Manag 2010; 16 (01) 29-56
• 7 Goel K, Bandara W, Gable G. A typology of business proceess standardization strategies. Bus Inf Syst Eng 2021; 63 (06) 621-635
• 8 Lavelle J, Schast A, Keren R. Standardizing care processes and improving quality using pathways and continuous quality improvement. Curr Treat Options Pediatr 2015; 1: 347-358
• 9 Kurtin P, Stucky E. Standardize to excellence: improving the quality and safety of care with clinical pathways. Pediatr Clin North Am 2009; 56 (04) 893-904
• 10 Téoule P, Birgin E, Mertens C. et al. Clinical pathways for oncological gastrectomy: Are they a suitable instrument for process standardization to improve process and outcome quality for patients undergoing gastrectomy? A retrospective cohort study. Cancers (Basel) 2020; 12 (02) 434
• 11 de Hoop T, Neumuth T. Evaluating electronic health record limitations and time expenditure in a German medical center. Appl Clin Inform 2021; 12 (05) 1082-1090
• 12 Moy AJ, Aaron L, Cato KD. et al. Characterizing multitasking and workflow fragmentation in electronic health records among emergency department clinicians: Using time-motion data to understand documentation burden. Appl Clin Inform 2021; 12 (05) 1002-1013
• 13 Tai-Seale M, McGuire TG, Zhang W. Time allocation in primary care office visits. Health Serv Res 2007; 42 (05) 1871-1894
• 14 Tai-Seale M, McGuire T, Colenda C, Rosen D, Cook MA. Two-minute mental health care for elderly patients: inside primary care visits. J Am Geriatr Soc 2007; 55 (12) 1903-1911
• 15 Zheng K, Guo MH, Hanauer DA. Using the time and motion method to study clinical work processes and workflow: methodological inconsistencies and a call for standardized research. J Am Med Inform Assoc 2011; 18 (05) 704-710
• 16 Heinen Y, Wolff G, Klein K. et al. Process standardization in high-risk coronary interventions is associated with quality of care measures. J Invasive Cardiol 2022; 34 (10) E743-E749
• 17 Jaulin F, Lopes T, Martin F. Standardised handover process with checklist improves quality and safety of care in the postanaesthesia care unit: the Postanaesthesia Team Handover trial. Br J Anaesth 2021; 127 (06) 962-970
• 18 Philips K, Zhou R, Lee DS. et al. Implementation of a standardized approach to improve the pediatric discharge medication process. Pediatrics 2021; 147 (02) e20192711
• 19 Fontánez-Nieves TD, Frost M, Anday E, Davis D, Cooperberg D, Carey AJ. Prevention of unplanned extubations in neonates through process standardization. J Perinatol 2016; 36 (06) 469-473
• 20 Rozich JD, Howard RJ, Justeson JM, Macken PD, Lindsay ME, Resar RK. Standardization as a mechanism to improve safety in health care. Jt Comm J Qual Saf 2004; 30 (01) 5-14
• 21 Arndt BG, Beasley JW, Watkinson MD. et al. Tethered to the EHR: Primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med 2017; 15 (05) 419-426
• 22 Overhage JM, McCallie Jr D. Physician time spent using the electronic health record during outpatient encounters: a descriptive study. Ann Intern Med 2020; 172 (03) 169-174
• 23 Overhage JM, Johnson KB. Pediatrician electronic health record time use for outpatient encounters. Pediatrics 2020; 146 (06) e20194017
• 24 Melnick ER, Ong SY, Fong A. et al. Characterizing physician EHR use with vendor derived data: a feasibility study and cross-sectional analysis. J Am Med Inform Assoc 2021; 28 (07) 1383-1392
• 25 Rotenstein LS, Holmgren AJ, Downing NL, Longhurst CA, Bates DW. Differences in clinician electronic health record use across adult and pediatric primary care specialties. JAMA Netw Open 2021; 4 (07) e2116375
• 26 Tai-Seale M, Olson CW, Li J. et al. Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine. Health Aff (Millwood) 2017; 36 (04) 655-662
• 27 Hron JD, Lourie E. Have you got the time? Challenges using vendor electronic health record metrics of provider efficiency. J Am Med Inform Assoc 2020; 27 (04) 644-646
• 28 Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med 2014; 12 (06) 573-576
• 29 Melnyk B. National academy of medicine's action collaborative on clinician well-being and resilience: a solution-focused strategy is designed to curtail the burnout epidemic. Am Nurse Today 2019; 14 (04) 61-64
• 30 Tai-Seale M, Dillon EC, Yang Y. et al. Physicians' well-being linked to in-basket messages generated by algorithms in electronic health records. Health Aff (Millwood) 2019; 38 (07) 1073-1078
• 31 Tutty MA, Carlasare LE, Lloyd S, Sinsky CA. The complex case of EHRs: examining the factors impacting the EHR user experience. J Am Med Inform Assoc 2019; 26 (07) 673-677
• 32 Longhurst CA, Davis T, Maneker A. et al; Arch Collaborative. Local investment in training drives electronic health record user satisfaction. Appl Clin Inform 2019; 10 (02) 331-335
• 33 Cross DA, Holmgren AJ, Apathy NC. The role of organizations in shaping physician use of electronic health records. Health Serv Res 2024; 59 (01) e14203
• 34 Obinna M, Ogoke U, Nduka E. Automation of balanced nested design; NeDPy. Int J Stat Appl 2020; 10 (01) 17-23
• 35 Cohen GR, Friedman CP, Ryan AM, Richardson CR, Adler-Milstein J. Variation in physicians' electronic health record documentation and potential patient harm from that variation. J Gen Intern Med 2019; 34 (11) 2355-2367
• 36 Levy DR, Sloss EA, Chartash D. et al. Reflections on the documentation burden reduction amia plenary session through the lens of 25 × 5. Appl Clin Inform 2023; 14 (01) 11-15
• 37 Hobensack M, Levy DR, Cato K. et al. 25 × 5 symposium to reduce documentation burden: report-out and call for action. Appl Clin Inform 2022; 13 (02) 439-446
• 38 DeShazo JP, Hoffman MA. A comparison of a multistate inpatient EHR database to the HCUP Nationwide Inpatient Sample. BMC Health Serv Res 2015; 15: 384