Download PDF
Appl Clin Inform 2021; 12(04): 737-744
DOI: 10.1055/s-0041-1733851
Research Article

Measuring Electronic Health Record Use in the Pediatric ICU Using Audit-Logs and Screen Recordings

Amrita Sinha
1   Division of Pediatric Critical Care Medicine, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States
,
Lindsay A. Stevens
2   Division of General Pediatrics, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States
,
Felice Su
1   Division of Pediatric Critical Care Medicine, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States
,
Natalie M. Pageler
1   Division of Pediatric Critical Care Medicine, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States
3   Division of Biomedical Informatics Research, Department of Medicine, Stanford University School of Medicine, Stanford, California, United States
,
Daniel S. Tawfik
1   Division of Pediatric Critical Care Medicine, Department of Pediatrics, Stanford University School of Medicine, Stanford, California, United States
› Author Affiliations Funding None.
› Further Information
   Permissions and Reprints

Abstract

Background Time spent in the electronic health record (EHR) has been identified as an important unit of measure for health care provider clinical activity. The lack of validation of audit-log based inpatient EHR time may have resulted in underuse of this data in studies focusing on inpatient patient outcomes, provider efficiency, provider satisfaction, etc. This has also led to a dearth of clinically relevant EHR usage metrics consistent with inpatient provider clinical activity.

Objective The aim of our study was to validate audit-log based EHR times using observed EHR-times extracted from screen recordings of EHR usage in the inpatient setting.

Methods This study was conducted in a 36-bed pediatric intensive care unit (PICU) at Lucile Packard Children's Hospital Stanford between June 11 and July 14, 2020. Attending physicians, fellow physicians, hospitalists, and advanced practice providers with ≥0.5 full-time equivalent (FTE) for the prior four consecutive weeks and at least one EHR session recording were included in the study. Citrix session recording player was used to retrospectively review EHR session recordings that were captured as the provider interacted with the EHR.

Results EHR use patterns varied by provider type. Audit-log based total EHR time correlated strongly with both observed total EHR time (r = 0.98, p < 0.001) and observed active EHR time (r = 0.95, p < 0.001). Each minute of audit-log based total EHR time corresponded to 0.95 (0.87–1.02) minutes of observed total EHR time and 0.75 (0.67–0.83) minutes of observed active EHR time. Results were similar when stratified by provider role.

Conclusion Our study found inpatient audit-log based EHR time to correlate strongly with observed EHR time among pediatric critical care providers. These findings support the use of audit-log based EHR-time as a surrogate measure for inpatient provider EHR use, providing an opportunity for researchers and other stakeholders to leverage EHR audit-log data in measuring clinical activity and tracking outcomes of workflow improvement efforts longitudinally and across provider groups.

Keywords

EHRs and systems - EHR time - pediatrics - audit-log - databases - data validation and verification - intensive and critical care - workflow - physician

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by Stanford Institutional Review Board.




Publication History

Received: 29 January 2021

Accepted: 28 June 2021

Article published online:
11 August 2021

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Adler-Milstein J, Jha AK. HITECH act drove large gains in hospital electronic health record adoption. Health Aff (Millwood) 2017; 36 (08) 1416-1422
    CrossrefPubMedGoogle Scholar
  • 2 Washington V, DeSalvo K, Mostashari F, Blumenthal D. The HITECH era and the path forward. N Engl J Med 2017; 377 (10) 904-906
    CrossrefPubMedGoogle Scholar
  • 3 Office of the National Coordinator for Health Information Technology. “Office-based physician electronic health record adoption,” health IT quick-stat #50. Accessed January 2019 at: dashboard.healthit.gov/quickstats/pages/physician-ehr-adoption-trends.php
    PubMedGoogle Scholar
  • 4 Office of the National Coordinator for Health Information Technology. “Percent of hospitals, by type, that possess certified health IT,” health IT quick-stat #52. dashboard.healthit.gov/quickstats/pages/certified-electronic-health-record-technology-in-hospitals.php . Accessed September 2018
    PubMedGoogle Scholar
  • 5 Standards for Health Information Technology to Protect Electronic Health Information Created, Maintained, and Exchange =d, 45 C.F.R. § 170.210. Accessed 2021 at: https://www.law.cornell.edu/cfr/text/45/170.2102015
    PubMedGoogle Scholar
  • 6 ASTM E2147-Standard Specification for Audit and Disclosure Logs for Use in Health Information Systems. Available at: https://www.astm.org/Standards/ E2147.htm
    PubMedGoogle Scholar
  • 7 Sinsky CA, Rule A, Cohen G. et al. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc 2020; 27 (04) 639-643
    CrossrefPubMedGoogle Scholar
  • 8 Rule A, Chiang MF, Hribar MR. Using electronic health record audit logs to study clinical activity: a systematic review of aims, measures, and methods. J Am Med Inform Assoc 2020; 27 (03) 480-490
    CrossrefPubMedGoogle Scholar
  • 9 Adler-Milstein J, Adelman JS, Tai-Seale M, Patel VL, Dymek C. EHR audit logs: A new goldmine for health services research?. J Biomed Inform 2020; 101: 103343
    CrossrefPubMedGoogle Scholar
  • 10 Gardner RL, Cooper E, Haskell J. et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019; 26 (02) 106-114
    CrossrefPubMedGoogle Scholar
  • 11 DiAngi YT, Stevens LA, Halpern-Felsher B, Pageler NM, Lee TC. Electronic health record (EHR) training program identifies a new tool to quantify the EHR time burden and improves providers' perceived control over their workload in the EHR. JAMIA Open 2019; 2 (02) 222-230
    CrossrefPubMedGoogle Scholar
  • 12 Arndt BG, Beasley JW, Watkinson MD. et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med 2017; 15 (05) 419-426
    CrossrefPubMedGoogle Scholar
  • 13 Hribar MR, Read-Brown S, Goldstein IH. et al. Secondary use of electronic health record data for clinical workflow analysis. J Am Med Inform Assoc 2018; 25 (01) 40-46
    CrossrefPubMedGoogle Scholar
  • 14 Read-Brown S, Hribar MR, Reznick LG. et al. Time requirements for electronic health record use in an academic ophthalmology center. JAMA Ophthalmol 2017; 135 (11) 1250-1257
    CrossrefPubMedGoogle Scholar
  • 15 Hribar MR, Read-Brown S, Reznick L. et al. Secondary use of EHR timestamp data: validation and application for workflow optimization. AMIA Annu Symp Proc 2015; 2015: 1909-1917
    PubMedGoogle Scholar
  • 16 Tai-Seale M, Olson CW, Li J. et al. Electronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine. Health Aff (Millwood) 2017; 36 (04) 655-662
    CrossrefPubMedGoogle Scholar
  • 17 Hribar MR, Read-Brown S, Reznick L. et al. Secondary use of EHR timestamp data: validation and application for workflow optimization. AMIA Annu Symp Proc.. Accessed 2015 at: https://pubmed.ncbi.nlm.nih.gov/26958290/
    PubMedGoogle Scholar
  • 18 Hribar MR, Huang AE, Goldstein IH. et al. Data-driven scheduling for improving patient efficiency in ophthalmology clinics. Ophthalmology 2019; 126 (03) 347-354
    CrossrefPubMedGoogle Scholar
  • 19 Wang JK, Ouyang D, Hom J, Chi J, Chen JH. Characterizing electronic health record usage patterns of inpatient medicine residents using event log data. PLoS ONE. 2019; 14 (02) e0205379
    PubMedGoogle Scholar
  • 20 Baxter SL, Apathy NC, Cross DA, Sinsky C, Hribar MR. Measures of electronic health record use in outpatient settings across vendors. J Am Med Inform Assoc 2020
    PubMedGoogle Scholar
  • 21 Unertl KM, Novak LL, Johnson KB, Lorenzi NM. Traversing the many paths of workflow research: developing a conceptual framework of workflow terminology through a systematic literature review. J Am Med Inform Assoc 2010; 17 (03) 265-273
    CrossrefPubMedGoogle Scholar
  • 22 Zheng K, Guo MH, Hanauer DA. Using the time and motion method to study clinical work processes and workflow: methodological inconsistencies and a call for standardized research. J Am Med Inform Assoc 2011; 18 (05) 704-710
    CrossrefPubMedGoogle Scholar
  • 23 Lopetegui M, Yen P-Y, Lai A, Jeffries J, Embi P, Payne P. Time motion studies in healthcare: what are we talking about?. J Biomed Inform 2014; 49: 292-299
    CrossrefPubMedGoogle Scholar
  • 24 Patel VL, Kannampallil TG, Kaufman DR. Cognitive Informatics for Biomedicine: Human Computer Interaction in Healthcare. Springe =r International Publishing; 2015. DOI: 10.1007/978-3-319-17272-9
    CrossrefGoogle Scholar
  • 25 National Research Network | Department of Medicine. Accessed May 28, 2020 at: https://medicine.ucsf.edu/center-clinical-informatics-and-improvement-research/national-research-network
    PubMedGoogle Scholar