Impact of Patient Census and Admission Mortality on Pediatric Intensive Care Unit Attending Electronic Health Record Activity: A Preliminary Study

Abstract

Background Physicians may spend a significant amount of time using the electronic health record (EHR), but this is understudied in the pediatric intensive care unit (PICU). The objective of this study is to quantify PICU attending physician EHR usage and determine its association with patient census and mortality scores.

Methods During the year 2016, total EHR, chart review, and documentation times of 7 PICU physicians were collected retrospectively utilizing an EHR-embedded time tracking software package. We examined associations between documentation times and patient census and maximum admission mortality scores. Odds ratios (ORs) are reported per 1-unit increase in patient census and mortality scores.

Results Overall, total daily EHR usage time (median time [hh:mm] [25th, 75th percentile]) was 2:10 (1:31, 3:08). For all hours (8 a.m.–8 a.m.), no strong association was noted between total EHR time, chart review, and documentation times and patient census, Pediatric Index of Mortality 2 (PIM2), or Pediatric Risk of Mortality 3 (PRISM3) scores. For regular hours (8 a.m.–7 p.m.), no strong association was noted between total EHR, chart review, and documentation times and patient census, PIM2, or PRISM3 scores. When patient census was higher, the odds of EHR after-hour usage (7 p.m.–8 a.m.) was higher (OR 1.262 [1.135, 1.403], p < 0.0001), but there were no increased odds with PIM2 (OR 1.090 [0.956, 1.242], p = 0.20) and PRISM3 (OR 1.010 [0.984, 1.036], p = 0.47) scores. A subset of physicians spent less time performing EHR-related tasks when patient census and admission mortality scores were elevated.

Conclusion We performed a novel evaluation of physician EHR workflow in our PICU. Our pediatric critical care physicians spend approximately 2 hours (out of an expected 10-hour shift) each service day using the EHR, but there was no strong or consistent association between EHR usage and patient census or mortality scores. Future larger scale studies are needed to ensure validity of these results.

Keywords

intensive and critical care - electronic health records and systems - inpatient care - workflow - pediatrics - attending rounds - residency - patient care

Background and Significance

In the pediatric intensive care unit (PICU), clinicians are tasked to manage fragile, complex, and critically ill patients.[1] Therefore, conventional wisdom dictates clinicians should be at the bedside to avoid risking a delayed or missed diagnosis.[2] Given such conventional thinking, the tools available to monitor and care for patients need to be easily accessible and not create barriers to care.

One tool utilized in the management of this unique patient population is the electronic health record (EHR). The EHR contains up-to-date medical data crucial for patient care. However, to use the EHR effectively, the provider must know what is pertinent, where to locate it, and be able to efficiently document this data for ongoing communication and continuity of patient care.[3] While there are no studies directly evaluating whether bedside physical reevaluation of a critically ill patient increases the quality of patient care, nursing literature suggests that when tasks such as shift change communication reports are performed at the bedside, patient safety outcomes are improved and fewer adverse effects occur.[4]

The growing body of literature on EHR usage is complex. On the one hand, existing data suggests that adult critical care physicians are spending the majority of their time using the EHR instead of being at the patient's bedside.[5] [6] On the other, thorough clinical review and processing of patient data retrieved from the EHR is important, as studies have shown that data gathering for daily intensive care unit (ICU) rounds is prone to omissions and inaccuracies.[7] In addition, previous research has also shown that inadequate analysis leads to diagnostic error and patient harm.[8] In the pediatric critical care literature, studies on EHR data collection and usage are sparse.[9] Thus, we do not know whether PICU physicians are spending more time at the bedside managing patients prone to rapid clinical decompensation or in front of a computer screen. Furthermore, we do not know if the acuity level or the patient census in the PICU is in any way correlated to or compromised by EHR usage.

Objectives

The study objectives were to quantify PICU attending EHR usage (using a built-in time tracking program embedded within our EHR) and to determine the association between usage and either patient census or newly admitted critically ill patients. We hypothesized that when patient census is increased or when a patient who is newly admitted has a high mortality risk, EHR usage will increase.

Methods

Study Design

This was a retrospective observational study focusing on PICU attending EHR usage approved as nonhuman research by our institutional review board.

Study Setting

The PICU at Penn State Hershey Children's Hospital consists of 18 beds and is a university-affiliated tertiary care facility in which medical, general surgical, and cardiothoracic patients are treated.

Pediatric critical care attending physicians spend 1 week at a time on service, and are responsible for writing all admitted patients that require critical care services. They write all pediatric cardiothoracic surgery patient notes and attest written resident notes. On weekends, resident notes may be written on both pediatric cardiothoracic patient and medical/surgical patients, thus the attending is responsible for ensuring all notes are written and/or attested to. Clinical data are presented during rounds by the residents and pediatric cardiothoracic surgery physician assistants on weekdays and only residents on weekends. Pediatric critical care attending physicians do not perform consultations at our institution (instead opting to transfer/admit the patient to the PICU if contacted by another service).

Attending physicians arrive in the PICU at 8:00 a.m. and end their shifts when work is completed (with an expected shift time of ∼10 hours). Attending EHR usage typically occurs before and after rounds as well as during rounds using EHR-equipped mobile computers. Handoff occurs at 4:00 p.m. on weekdays and 5:00 p.m. on weekends and holidays.

Data Collection

Our medical center utilizes Cerner Powerchart (Cerner Corp, Kansas City, Missouri, United States) as our primary EHR. It is exclusively used for clinical documentation and can retrieve laboratory, radiological, hemodynamic, and patient flowsheet data. This EHR has a built-in time tracking program called Cerner Advance (Cerner Corp). Its timers are designed to activate when the user performs any one of the following actions within the EHR: 3 mouse clicks, 15 keystrokes, or 1,700 “mouse miles” per minute (distance a cursor travels across the monitor screen). Any EHR activity is defined as active time with the software capable of recording and distinguishing different user EHR actions (i.e., documentation, chart review, etc.).

PICU attending EHR usage activity was obtained from the Cerner Advance User Experience Management Service. The database was queried from January 1, 2016 until December 31, 2016, focusing on attending EHR usage during service days. We obtained the following data: total active time (any EHR activity including documentation, chart review, patient flowsheet review, messaging, orders, etc.), chart review time (time spent reviewing clinical documents, patient flowsheet, medication administration record), documentation time (time spent performing clinical documentation), and order entry (time spent performing orders). The data included the times of 8:00 a.m. until 8:00 a.m. the next day, but was unable to distinguish if the EHR was used within the PICU or via remote access (i.e., at home). Also, the purpose of EHR usage was unknown (i.e., chart review before shift start, continuing documentation from previous day, starting documentation for current day). Because of these factors and the potential for remote access skewing our results (i.e., EHR usage may be more efficient at home without potential interruptions or slower as the clinician may have more time to carefully analyze the chart or document), we elected to separate and analyze the data by certain hours that reflect a higher likelihood the provider was in the PICU. Three sets of hours were defined: (1) regular hours (8:00 a.m. until 7:00 p.m.); (2) all hours (8:00 a.m. until 8:00 a.m. the next day); and (3) after-hours (7:00 p.m. until 8:00 a.m.). Regular hours most likely reflect exclusive EHR usage within the PICU. All hours reflect not only EHR usage in the PICU, but most likely external (remote access) use as well. The time-tracking software only evaluated EHR usage, thus other uses of the computer were not tracked or recorded (i.e., Web site access, email communications, etc.).

In our institution's PICU, there is no designated shift-end time to mark the start of after-hours time. Because our objective was to determine how the clinician utilized the EHR within the environment of the PICU (and ensuring that the data we are analyzing fully incorporates the distractions that a clinician may experience during work), we selected 7:00 p.m. as the start of after-hours until 8:00 a.m. This time period was chosen as most of our clinicians continue working after handoff whether it is following up on patients, reviewing the EHR for final results, or finishing documentation. Prerounding (preshift chart review and documentation initiation) is commonly employed by pediatric critical care physicians to prepare for a shift, but this was not considered in our analysis. This determination was made because it was (1) difficult to evaluate if documentation was being initiated for the new service day or completed for the previous service day; and (2) our lack of ability in determining the location of the chart review (within the PICU environment or remotely).

Current methods that provide patient census within our institution counted all patients (patients who were admitted under pediatric critical care and those who were not), thus potentially resulting inaccurate numbers. Therefore, patient census was obtained by querying the Cerner Advance database to provide the number of clinical notes written or attested to on each day the attending was on service. To ensure this method had an adequate reflection of the patient census at the time our clinicians were providing patient care, we included only clinical notes (i.e., history and physical exam, daily progress notes), we removed other note types (i.e., procedure notes, and duplicate progress notes), and made certain that the patients for whom notes were written for were provided pediatric critical care services by reviewing the Virtual PICU Systems database (VPS, LLC, Los Angeles, California, United States). The VPS database is maintained by our research nurse who records all patients admitted to our PICU who required pediatric critical care services, collects clinical data, and documents the start and end dates of the medical care provided. Therefore, if a clinical note was written within the period recorded where the patient was provided pediatric critical care services by the VPS database, it was counted toward the patient census.

To evaluate the acuity of newly admitted critically ill patients, we collected Pediatric Risk of Mortality (PRISM) 3 and Pediatric Index of Mortality (PIM) 2 scores (with increasing scores indicating increasing risk of mortality, increasing severity of illness, and possibly higher intensity of care provided) from the VPS database. Of these patients, we only included the maximum PRISM 3 and PIM 2 scores (not the average or median) seen each service day by the PICU service attending to assess the impact a newly admitted patient with the highest risk of mortality has on EHR usage.

Because EHR activity potentially included other patients who were not in the PICU, we obtained a patient identification number list that was accessed by the PICU attending physician. Patients were included in the study if they were determined to be in the PICU at chart access time and confirmed to have been seen by the attending physician defined by the presence of a clinical note on the day of chart access. Patient charts accessed due to informal communications (where a pediatric critical care physician was made aware of a patient who potentially could develop critical illness and accessed their record) were not included in the analysis. This was due to the potential for skewed results as a full chart review and documentation may not have been necessary as the patient was not admitted to the PICU. Encounters reported on the first night on service were excluded, as the chart review and documentation time varied based on the amount of admissions and provider preference to become familiar with the patients in the unit.

Time Spent per Patient Calculations

After exclusions, a sum of total EHR usage time, chart review time, and clinical documentation time was obtained for each PICU attending day of service for all hours and regular hours. These sums were divided by the number of clinical notes signed per day to determine total EHR, chart review, and documentation time spent per patient. We did not make any calculations for the after-hours timeframe, but it was noted if EHR usage occurred.

Statistical Analysis

Per day, PICU attending EHR activity (total EHR time, chart review time, and documentation time per patient) was linked to the patient census and maximum mortality scores for newly admitted patients. Pearson's correlations were used to assess the association between the daily PICU attending EHR activity and patient census and mortality scores. Logistic regression was used to assess the relationship between an increase in patient census and mortality scores and the odds of working on the EHR after-hours. Odds ratios (ORs) with 95% confidence intervals are reported per 1-unit increase in patient census and mortality scores.

Results

PICU Service Time Characteristics

Seven attending PICU physicians were included in this study. The median service time (25th, 75th percentile) that a PICU physician performed during the study time period was 7 weeks (5, 8) ([Table 1]).

Patient Encounters

A total number of 21,269 patient encounters occurred during the study period ([Fig. 1]). Of these, the final analysis to determine the impact of patient census and mortality on EHR usage included 11,014 patient encounters. The following exclusions were made: 8,973 due to encounter accessed on a “nonservice day”; 1,150 due to encounter accessed on the first night of service; 82 if encounter accessed on a particular day was not present in the PICU or hospital during the same timeframe; 40 if patient encounter accessed was for a patient no longer on the ICU service; and 10 if no patient identifier was available for chart review for the given patient encounter. Five patient encounters were seen by a PICU attending provider, but no mortality scores were available due to the patient not being documented as intensive care status or not being reported in the VPS database.

Fig. 1 Inclusion and exclusion criteria applied to the dataset to determine the impact of patient census and mortality on attending pediatric intensive care unit (PICU) physician electronic health record (EHR) usage.

PICU Attending EHR Activity

The amount of time (median [hh:mm] [25th, 75th percentile]) spent using the EHR each day was 2:10 (1:31, 3:08). Time spent performing chart review was 35 minutes (00:14, 1:10) and documentation time was 1 hour (00:45, 1:22). The median time spent per patient using the EHR was 8 minutes (00:06, 00:12), time spent performing chart review, 2 minutes (00.01, 00:05), and documentation time, 3 minutes (00:03, 00:05).

Patient Characteristics and Patient Census

During the study time period, the median maximum admission PRISM 3 score (median [25th, 75th percentile]) during the service weeks was 7.0 (4, 12.25). The median maximum admission PIM 2 score was –3.52 (–4.55, –3.06). The median patient census per day was 16 (14, 18).

Impact of Patient Census and Newly Admitted Patient Mortality Scores on PICU Attending EHR Usage Activity

For all day hours, there was no strong association noted between patient census and total time spent utilizing the EHR (r = –0.047, p = 0.40), chart review time (r = –0.037, p = 0.50), and documentation time (r = –0.029, p = 0.60); maximum admission PIM2 score and total time spent utilizing EHR (r = 0.027, p = 0.64), chart review time (r = 0.030, p = 0.61), and documentation (r = –0.038, p = 0.52); and maximum admission PRISM 3 score and total time spent utilizing EHR (r = –0.056, p = 0.34), chart review time (r = –0.072, p = 0.22), and documentation time (r = –0.078, p = 0.18).

When only regular hours were included, there was no strong correlation between patient census, maximum admission PIM2, and PRISM3 scores and total time spent in the EHR (r = –0.184, p = 0.0007; r = –0.043, p = 0.46; r = –0.102, p = 0.08), chart review time (r = –0.092, p = 0.10; r = –0.014, p = 0.81; r = –0.107, p = 0.07), and documentation time (r = –0.192, p = 0.0004; r = –0.095, p = 0.10; r = –0.112, p = 0.06).

Patient census was associated with after-hour EHR usage (OR 1.262 [1.135, 1.403], p < 0.0001), but not admission PIM2 (OR 1.090 [0.956, 1.242], p = 0.20) and PRISM3 (OR 1.010 [0.984, 1.036], p = 0.47) scores.

Individual Provider and Association with Patient Census, PIM2, and PRISM3 Scores

While there were no associations noted overall, there were some individual associations. For all day hours, attending providers 4 and 5 had a moderate negative correlation between patient census and total EHR usage time (r = –0.418, p = 0.01; r = –0.304, p = 0.05) ([Table 2]); attending providers 2, 4, 5, and 7 had a moderate negative correlation between patient census and time spent performing chart review (r = –0.309, p = 0.02; r = –0.392, p = 0.02; r = –0.330, p = 0.03; r = –0.480, p = 0.01) ([Table 2]); one attending provider (4) had a moderate negative correlation between patient census and documentation time (r = –0.367, p = 0.03) ([Table 2]).

Table 1 2016 attending service time at Penn State Children's Hospital PICU

Attending	Number of weeks of service
1	9
2	8
3	5
4	5
5	6
6	8
7	7

Abbreviation: PICU, pediatric intensive care unit.

Table 2 The individual Pearson's correlation coefficients between patient census and types of EHR usage

		All day (8:00 a.m. until 11:59 p.m.)			Regular hours (8:00 a.m. until 6:00 p.m.)
Attending	Number of days	Total time	Chart review	Documentation	Total time	Chart review	Documentation
1	62	–0.101 (0.43)	0.124 (0.34)	–0.294 (0.02)	–0.536 (< 0.001)	–0.071 (0.59)	–0.488 (< 0.001)
2	56	–0.269 (0.04)	–0.30 (0.02)	–0.046 (0.74)	–0.267 (0.05)	–0.304 (0.02)	–0.01 (0.94)
3	35	–0.034 (0.84)	0.176 (0.31)	–0.015 (0.93)	–0.126 (0.47)	0.362 (0.03)	0.014 (0.93)
4	35	–0.418 (0.01)	–0.392 (0.02)	–0.367 (0.03)	–0.426 (0.01)	–0.349 (0.04)	–0.357 (0.04)
5	42	–0.304 (0.05)	–0.330 (0.03)	–0.145 (0.36)	–0.310 (0.05)	–0.293 (0.06)	–0.142 (0.37)
6	56	–0.176 (0.19)	–0.280 (0.03)	–0.053 (0.70)	–0.257 (0.06)	–0.308 (0.02)	–0.171 (0.21)
7	46	–0.250 (0.09)	–0.480 (0.01)	0.211 (0.16)	–0.398 (0.01)	–0.496 (0.01)	0.07 (0.66)

Abbreviation: EHR, electronic health record.

Note: Numbers in parentheses are p-values.

For regular hours, one attending provider (1) had a strong association between patient census and total EHR usage time (r = –0.536, p < 0.001) ([Table 2]); three attending providers (4, 5, and 7) had a moderate negative correlation between patient census and total EHR usage time (r = –0.426, p = 0.01; r = –0.310, p = 0.05; r = –0.398, p = 0.01) ([Table 2]); two attending providers (1 and 2) had a moderate negative correlation between admission PIM2 (r = –0.390, p = 0.003; r = –0.385, p = 0.006) and PRISM3 (r = –0.396, p = 0.003; r = –0.426, p = 0.002) scores and total time spent utilizing the EHR ([Tables 3] and [4]); four attending providers (2, 4, 6, and 7) had a moderate negative correlation between patient census and time spent performing chart review (r = –0.304, p = 0.02; r = –0.349, p = 0.04; r = –0.308, p = 0.02; r = –0.496, p = 0.01); one individual provider (3) had a moderate positive correlation between patient census and time spent performing chart review (r = 0.362, p = 0.03) ([Table 2]); one attending provider (2) was noted to have a moderate negative correlation (r = –0.331, p = 0.02; r = –0.369, p = 0.009) ([Tables 3] and [4]); two individual providers (1 and 4) had a moderate negative correlation between patient census and documentation time (r = –0.488, p < 0.001; r = –0.357, p = 0.04) ([Table 2]); one individual provider (1) had a moderate negative correlation between admission PIM2 scores and documentation time (r = –0.308, p = 0.02) ([Table 3]); and one individual provider (2) was noted to have a moderate negative correlation between admission PRISM3 scores and documentation (r = –0.320, p = 0.02).

Table 3 The individual Pearson's correlation coefficients between maximum PIM2 score and types of EHR usage

		All day (8:00 a.m. until 11:59 p.m.)			Regular hours (8:00 a.m. until 6:00 p.m.)
Attending	Number of days	Total time	Chart review	Documentation	Total time	Chart review	Documentation
1	55	0.112 (0.42)	0.240 (0.08)	–0.102 (0.46)	–0.390 (0.003)	0.009 (0.95)	–0.308 (0.02)
2	49	–0.140 (0.34)	–0.134 (0.36)	–0.089 (0.55)	–0.385 (0.006)	–0.331 (0.02)	–0.298 (0.04)
3	32	0.008 (0.96)	–0.013 (0.94)	–0.011 (0.95)	0.098 (0.59)	–0.145 (0.422)	0.064 (0.73)
4	30	0.207 (0.27)	0.099 (0.60)	0.193 (0.31)	0.040 (0.84)	–0.046 (0.81)	0.041 (0.83)
5	39	0.136 (0.41)	0.124 (0.45)	–0.005 (0.98)	0.144 (0.38)	0.080 (0.63)	–0.115 (0.48)
6	48	–0.006 (0.97)	–0.177 (0.23)	0.058 (0.70)	0.045 (0.76)	–0.074 (0.62)	0.123 (0.41)
7	43	–0.112 (0.48)	–0.020 (0.90)	–0.231 (0.14)	–0.181 (0.25)	–0.061 (0.70)	–0.195 (0.21)

Abbreviations: EHR, electronic health record; PIM2, Pediatric Index of Mortality 2.

Note: Numbers in parentheses are p-values.

Table 4 The individual Pearson's correlation coefficients between maximum PRISM3 score and types of EHR usage

		All day (8:00 a.m. until 11:59 p.m.)			Regular hours (8:00 a.m. until 6:00 p.m.)
Attending	Number of days	Total time	Chart review	Documentation	Total time	Chart review	Documentation
1	55	0.021 (0.88)	0.141 (0.30)	–0.159 (0.25)	–0.396 (0.003)	–0.020 (0.89)	–0.299 (0.03)
2	49	–0.197 (0.18)	–0.208 (0.15)	–0.086 (0.56)	–0.426 (0.002)	–0.369 (0.009)	–0.320 (0.03)
3	32	0.045 (0.81)	–0.033 (0.86)	0.031 (0.87)	0.172 (0.35)	–0.030 (0.87)	0.120 (0.51)
4	30	0.179 (0.34)	0.067 (0.73)	0.158 (0.41)	0.108 (0.57)	–0.051 (0.79)	0.091 (0.63)
5	39	0.114 (0.49)	0.019 (0.91)	0.130 (0.43)	0.126 (0.45)	–0.012 (0.94)	–0.011 (0.95)
6	48	–0.058 (0.70)	–0.191 (0.19)	–0.045 (0.76)	–0.089 (0.55)	–0.206 (0.16)	–0.099 (0.50)
7	43	–0.072 (0.65)	–0.039 (0.80)	–0.142 (0.36)	–0.022 (0.89)	–0.058 (0.71)	0.019 (0.91)

Abbreviations: EHR, electronic health record; PRISM3, Pediatric Risk of Mortality 3.

Discussion

The aims of this study were to quantify PICU attending EHR usage using a built-in time tracking program, and to determine the association between patient census and newly admitted patients' mortality risk on EHR usage. We hypothesized that PICU attending EHR usage would have a negative correlation with increasing patient census and admission mortality scores. While approximately 2 hours (out of an expected 10-hour shift) of a pediatric critical care physician's time was spent utilizing the EHR, we observed no strong correlations. These preliminary findings are the first steps that need to be taken to understand how PICU physician EHR workflow is impacted. While we observed no association between EHR usage and patient census or newly admitted patients' mortality risk, our results demonstrate that while novel tools such as the Cerner Advance User Experience Management Service exist to allow time and workflow analysis of physicians, they may only illuminate part of the story.

EHRs are needed to operate efficiently in the PICU. They store and organize a large amount of clinical data (i.e., vital signs, laboratory data), provide clinical decision support systems to assist clinicians in making medical decisions, prevent medication errors, and allow order entry anywhere there is a computer.[10] [11] [12] [13] Furthermore, some studies have shown that EHRs are associated with lower mortality rates and complications.[10] [12] [14] [15] [16] But, while EHRs are necessary to deliver medical care, they do not replace it.

Time motion studies are currently the gold standard for analyzing work patterns, and have been previously utilized in the adult ICU setting.[6] Carayon et al described that, upon implementation of EHR technology, increased time was spent performing clinical review and documentation[5] however, the amount of physical care provided to the patient was unchanged. Hefter et al performed the largest time motion study to date, and demonstrated that as the average illness severity in the unit increased, more time was spent on direct patient care and less time on computer tasks.[6] This study, however, may underestimate EHR usage, as directly observed time motion studies usually only sample short periods of time. Therefore, it is unknown if the EHR-related tasks are being completed later in the shift or remotely at home. Our study is unique, in that there are few studies utilizing EHR-embedded tracking software as a surrogate measure for time motion evaluation and none evaluating the impact of patient census or mortality. Data in this study were not from small samples of time, but rather all EHR usage.[17] Therefore, even when the provider was using the EHR outside the unit (including remote home usage), EHR usage continued to be recorded. This information can provide insight on not only how tasks are completed during a shift, but the type of tasks that are completed at home. Finally, to our knowledge, this was the first workflow study of PICU attending physicians managing complex, fragile, critically ill pediatric patients.

Critical care physicians are a limited resource whose expertise lies in clinical decision making, managing a multidisciplinary team, performing procedures, and supervising trainees, thus their time should be utilized efficiently to provide clinical care. Achieving time efficiency, however, continues to be a challenge. There is an increased volume of critically ill patients, an increasing amount of clinical data, and technological limitations (i.e., poor EHR design) that hamper efficient information retrieval.[18] [19] [20] [21] An inefficient EHR can consume the amount of time a clinician spends resulting in cognitive overload, decreased performance levels, clinician stress, and burnout placing patients at risk.[20] [22] [23] [24] [25] Thus, workflow studies that include an analysis of EHR usage are necessary to identify issues and make critical changes to ensure the well-being of patients and clinicians.[26]

Our examination of 24/7 EHR usage gave rise to results different from previous time motion studies. There was no strong association between EHR usage and patient census or admission mortality scores, even when strictly focusing on PICU attending shifts. But, when the patient census increased, the odds of after-hour EHR usage were higher. The data, while limited in being able to perform an adequate interpretation of these findings, demonstrate just how complex examining EHR usage can be in any clinical setting. We expected that severely critically ill patients and a high census should cause an increase in EHR usage, surmising that physicians would want to not only review the care of patients constantly, but also to document clinical care for communication purposes. Applying our experiential knowledge of PICU physician workflow we speculate there are several reasons why we did not find this association. First, physicians may have consistently managed patients equally and effectively because they had the extra capacity to not be impacted by the absolute numbers of patient census and severity of illness that we observed. PICU physicians often pursue this type of career to ensure the well-being of critically ill children. This population is underserved and it may not be surprising to some PICU physicians that the computer could be the least utilized piece of medical equipment when faced with a child on the verge of death. Second, PICU physicians are typically trained to be expert communicators and team leaders, and therefore, it may not have been necessary to utilize the EHR for data that was already being reported verbally by the team or bedside nursing. Thus, human interaction may be more efficient than logging into the computer, finding the information, and interpreting it. Third, the days with high relative census may be something that our PICU physicians had already adapted to. Workarounds, such as copying and pasting or prewritten templates, may have already been developed by these physicians in an effort to reduce the computer workload. Finally, the way a physician documents in response to ICU environmental factors may have been lessened by the fact that the EHR could be accessed remotely at home, but resulting in after-hours EHR usage. Even though these adaptations were performed to enhance efficiency, it may not ensure clinician well-being, and may be possibly contributing to clinician stress and burnout. Further study on how a clinician adapts in times of stress in the clinical environment may be necessary to understand why there was no increase in EHR usage and how burnout occurs in these situations.

While this software and any study performed is limited to institutions who utilize Cerner, the results highlight important aspects to EHR usage studies. In this study, we were unable to find a correlation between times spent performing various EHR tasks and patient census or admission PICU mortality severity scores. In the EHR literature, evaluating clinician efficiency is often the objective, utilizing the time spent or number of mouse clicks to complete a task to demonstrate the impact of an intervention.[27] [28] Using time as a measure is understandable as decreasing the time spent performing EHR tasks can help improve the quality and safety of patient care,[29] improve clinician satisfaction, and allow more patients to be seen.[30] It is also an objective measure that demonstrates a targeted outcome. Time tracking tools such as the one used in this study, however, may have been designed assuming that the less time spent is what makes clinicians efficient and is marketed that way. Is this a fair assumption? The EHR is utilized in various aspects of clinical care, and in the PICU, by different clinicians working together as a team. Thus, there may have been no correlation between the environmental factors studied and the attending physician's EHR usage, because in this clinical setting, there were other factors like the presence of a well-functioning team. Possibly, there were days where the team was communicating well precluding the need for the attending physician to access and spend time on the EHR. On other days, this may have been less so requiring more EHR access. Patient complexity also may have led to more EHR usage to enhance an attending physician's understanding, whereas other patients that have a clear clinical course, communication with the team may have been relied upon more requiring less time using the EHR. Until clinician efficiency is well defined and accounts for the various factors involved in clinical care, any study evaluating the time spent on important patient-related EHR tasks should be interpreted with caution.

Our study has several limitations. First, it was completed with a small number of subjects in a single institution. Larger scale studies may be necessary to ensure that the associations we uncovered are valid and if other providers who utilize the EHR (i.e., residents, physician assistants, nurse practitioners) are impacted in a similar manner. Second, our data only evaluated EHR usage and not additional physician behaviors, thus we do not know if patient care or interprofessional communication was affected during this study. The validity of the data collected is unknown. Therefore, a validation study is underway in our institution that will seek to compare real-time EHR usage sessions and the software times. The time reported for our EHR may not be similar to other systems. The times chosen, particularly “after-hours” were chosen based on our knowledge of PICU workflow and our inability to determine the location where EHR usage occurred (within the PICU environment or remotely). While it is rare for our clinicians to leave before 5 p.m. and/or not access the chart before the start of a shift, it is possible that they may have at various times. Thus, a portion of regular hours may reflect external (remote) usage. Finally, this study was limited by confounding factors (i.e., patient complexity, patient comorbidities, multiorgan system involvement, and age) that may have affected our primary outcomes.

Conclusion

We performed a novel evaluation of physician EHR workflow in our PICU. Our pediatric critical care physicians spend approximately 2 hours (out of an expected 10-hour shift) each service day using the EHR, but there was no strong or consistent association between EHR usage and patient census or mortality scores. Future larger scale studies are needed to determine the validity of these results, whether EHR usage may be a benefit or barrier to the provision of clinical care, and the impact of EHR usage time on patient outcomes and physician well-being.

Clinical Relevance Statement

Pediatric critical care physicians are a limited resource who should be utilized efficiently to provide clinical care. One barrier to this could be EHR usage. This study provides a preliminary evaluation of pediatric critical care physician EHR usage using embedded EHR time-tracking technology and the impact of external clinical environmental factors. The knowledge gained could serve as a guide to designing future EHR and physician efficiency studies.

Multiple Choice Questions

1. Which of the following best describes clinician practices that allow effective EHR usage?

   • Understanding which data are pertinent.

   • Knowing where the data are located for efficient retrieval.

   • Efficient documentation.

   • All of the above.

   Correct Answer: The correct answer is option d, all of the above. To be able to use the EHR effectively, clinicians must be clear about the clinical data they are seeking and know the location of the data for retrieval. If the clinician wants to review a chest radiograph for their own interpretation, they must access the imaging records. Alternatively, if only the report is required, they must know this is the record they are accessing. Clinicians spend a majority of their time documenting patient care information, thus after review of the chart, they must understand the pertinent the data to document so that the patient's overall condition is understood and facilitate communication to other clinicians. If these EHR best practices are not followed, the clinicians are at risk for spending more time on the computer than at the bedside.

2. When utilizing an embedded time-tracking software within the EHR to analyze PICU attending physician workflow, which of the following should be considered?

   • PICU attending physicians may not rely on the EHR for up-to-date clinical data.

   • There is no risk for PICU attending physicians to utilize EHR workarounds.

   • PICU attending physicians do not complete their EHR work at home.

   • PICU attending physicians will use the EHR over providing direct patient care.

   Correct Answer: The correct answer is option a. PICU attending physicians may not rely on the EHR for up-to-date clinical data, instead possibly preferring verbal means of communication with nursing and other team members. EHR workarounds are a possibility in all clinical situations and should be considered. If there is a remote access option, it is conceivable that PICU attending physicians may complete their EHR work at home (as was the case in our study). Critically ill children may require constant bedside reassessment, thus the EHR may not be utilized during the course of a PICU shift.

Conflict of Interest

None declared.

Acknowledgment

We would like to thank Cristin S. Marker from Cerner for providing us the data analyzed in this study.

Protection of Human and Animal Subjects

This study was reviewed by Penn State Health's institutional review board and was determined to be nonhuman research.

• References

• 1 Rehder KJ, Cheifetz IM, Markovitz BP, Turner DA. ; Pediatric Acute Lung Injury and Sepsis Investigators Network. Survey of in-house coverage by pediatric intensivists: characterization of 24/7 in-hospital pediatric critical care faculty coverage*. Pediatr Crit Care Med 2014; 15 (02) 97-104
• 2 Verghese A, Brady E, Kapur CC, Horwitz RI. The bedside evaluation: ritual and reason. Ann Intern Med 2011; 155 (08) 550-553
• 3 Golob Jr JF, Como JJ, Claridge JA. The painful truth: the documentation burden of a trauma surgeon. J Trauma Acute Care Surg 2016; 80 (05) 742-745 , discussion 745–747
• 4 Groves PS, Manges KA, Scott-Cawiezell J. Handing off safety at the bedside. Clin Nurs Res 2016; 25 (05) 473-493
• 5 Carayon P, Wetterneck TB, Alyousef B. , et al. Impact of electronic health record technology on the work and workflow of physicians in the intensive care unit. Int J Med Inform 2015; 84 (08) 578-594
• 6 Hefter Y, Madahar P, Eisen LA, Gong MN. A time-motion study of ICU workflow and the impact of strain. Crit Care Med 2016; 44 (08) 1482-1489
• 7 Artis KA, Dyer E, Mohan V, Gold JA. Accuracy of laboratory data communication on ICU daily rounds using an electronic health record. Crit Care Med 2017; 45 (02) 179-186
• 8 Zwaan L, Thijs A, Wagner C, Timmermans DRM. Does inappropriate selectivity in information use relate to diagnostic errors and patient harm? The diagnosis of patients with dyspnea. Soc Sci Med 2013; 91 (91) 32-38
• 9 Han YY, Carcillo JA, Dragotta MA. , et al. Early reversal of pediatric-neonatal septic shock by community physicians is associated with improved outcome. Pediatrics 2003; 112 (04) 793-799
• 10 Ernst KD. Electronic alerts improve immunization rates in two-month-old premature infants hospitalized in the neonatal intensive care unit. Appl Clin Inform 2017; 8 (01) 206-213
• 11 Beam KS, Cardoso M, Sweeney M, Binney G, Weingart SN. Examining perceptions of computerized physician order entry in a neonatal intensive care unit. Appl Clin Inform 2017; 8 (02) 337-347
• 12 Wong A, Wright A, Seger DL, Amato MG, Fiskio JM, Bates D. Comparison of overridden medication-related clinical decision support in the intensive care unit between a commercial system and a legacy system. Appl Clin Inform 2017; 8 (03) 866-879
• 13 Rehr CA, Wong A, Seger DL, Bates DW. Determining inappropriate medication alerts from “inaccurate warning” overrides in the intensive care unit. Appl Clin Inform 2018; 9 (02) 268-274
• 14 Menachemi N, Chukmaitov A, Saunders C, Brooks RG. Hospital quality of care: does information technology matter? The relationship between information technology adoption and quality of care. Health Care Manage Rev 2008; 33 (01) 51-59
• 15 Amarasingham R, Plantinga L, Diener-West M, Gaskin DJ, Powe NR. Clinical information technologies and inpatient outcomes: a multiple hospital study. Arch Intern Med 2009; 169 (02) 108-114
• 16 Flatow VH, Ibragimova N, Divino CM. , et al. Quality outcomes in the surgical intensive care unit after electronic health record implementation. Appl Clin Inform 2015; 6 (04) 611-618
• 17 Chen L, Guo U, Illipparambil LC. , et al. Racing against the clock: internal medicine residents' time spent on electronic health records. J Grad Med Educ 2016; 8 (01) 39-44
• 18 Kannampallil TG, Jones LK, Patel VL, Buchman TG, Franklin A. Comparing the information seeking strategies of residents, nurse practitioners, and physician assistants in critical care settings. J Am Med Inform Assoc 2014; 21 (e2): e249-e256
• 19 Kannampallil TG, Franklin A, Mishra R, Almoosa KF, Cohen T, Patel VL. Understanding the nature of information seeking behavior in critical care: implications for the design of health information technology. Artif Intell Med 2013; 57 (01) 21-29
• 20 Kroth PJ, Morioka-Douglas N, Veres S. , et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (08) e199609
• 21 Grinspan ZM, Eldar YC, Gopher D. , et al. Guiding principles for a pediatric neurology ICU (neuroPICU) bedside multimodal monitor: findings from an international working group. Appl Clin Inform 2016; 7 (02) 380-398
• 22 Mazur LM, Mosaly PR, Moore C, Marks L. Association of the usability of electronic health records with cognitive workload and performance levels among physicians. JAMA Netw Open 2019; 2 (04) e191709-e191709
• 23 Shanafelt TD, Hasan O, Dyrbye LN. , et al. Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014. Mayo Clin Proc 2015; 90 (12) 1600-1613
• 24 Murphy DR, Giardina TD, Satterly T, Sittig DF, Singh H. An exploration of barriers, facilitators, and suggestions for improving electronic health record inbox-related usability: a qualitative analysis. JAMA Netw Open 2019; 2 (10) e1912638-e1912638
• 25 Nolan ME, Cartin-Ceba R, Moreno-Franco P, Pickering B, Herasevich V. A multisite survey study of EMR review habits, information needs, and display preferences among medical ICU clinicians evaluating new patients. Appl Clin Inform 2017; 8 (04) 1197-1207
• 26 Nolan ME, Siwani R, Helmi H, Pickering BW, Moreno-Franco P, Herasevich V. Health IT usability focus section: data use and navigation patterns among medical ICU clinicians during electronic chart review. Appl Clin Inform 2017; 8 (04) 1117-1126
• 27 King K, Quarles J, Ravi V. , et al. The impact of a location-sensing electronic health record on clinician efficiency and accuracy: a pilot simulation study. Appl Clin Inform 2018; 9 (04) 841-848
• 28 Gellert GA, Crouch JF, Gibson LA, Conklin GS, Webster SL, Gillean JA. Clinical impact and value of workstation single sign-on. Int J Med Inform 2017; 101: 131-136
• 29 Bates DW, Kuperman GJ, Wang S. , et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 10 (06) 523-530
• 30 Nambudiri VE, Watson AJ, Buzney EA, Kupper TS, Rubenstein MH, Yang FC. Medical scribes in an academic dermatology practice. JAMA Dermatol 2018; 154 (01) 101-103