Keywords
electronic health records - nursing - health care quality - work environment
Background and Significance
Background and Significance
The Health Information Technology for Economic and Clinical Health (HITECH) Act of
2009 spurred the rapid adoption of electronic health records (EHRs) in U.S. hospitals.[1] In 2008, 13% of hospitals in the country were using a basic EHR system; by 2015,
nearly 90% of hospitals had one in place.[2] Over the same 7-year period, the number of hospitals implementing comprehensive
EHR systems that have more advanced functionalities, such as clinical decision support,
grew from less than 2 to 40%.[2] While some evidence suggests that higher levels of EHR adoption produce safer and
higher quality care,[3]
[4]
[5]
[6]
[7] a growing body of literature has documented unintended negative consequences of
the technology,[8]
[9] including disruptions in clinical workflows and usability concerns that may negatively
affect quality of care and patient safety.[10]
[11]
[12] These system issues may disproportionally impact the work of nurses who are among
the highest EHR users in hospitals.
Nurses use EHR systems for documentation, medication administration, clinical monitoring
and decision-making, and coordination of patient care. Few studies have examined the
usability of EHR systems from the nurse perspective, including overall satisfaction,
as well as how these systems influence care activities and patient outcomes. With
the exception of one statewide study in Texas,[13] the vast majority of evidence from U.S. hospitals conducted following the HITECH
Act comes from single institutions.[14]
[15]
[16] Taken together, the findings of these studies are mixed. While nurses have identified
positive aspects of EHRs,[9]
[13]
[14]
[16] many nurses remain dissatisfied with their use, find them burdensome and time consuming,
and express doubts in the ability of the systems to improve patient care.[15]
[17] These inconclusive findings may be attributable not only to the level of EHR adoption,
but also to differences in the organizational and clinical environments into which
the systems are implemented. Some evidence suggests that hospitals with better work
environments may be more likely to adopt EHR systems of greater complexity;[18] however, it remains unknown how the work environment may affect the relationship
between EHR use and outcomes.
Sociotechnical theory suggests that EHR systems cannot be designed or implemented
successfully without thoughtful consideration of the “fit” of the technology with
the people providing care and the existing context of care delivery, including the
work environment.[19]
[20]
[21]
[22]
[23] Indeed, reviews of EHR outcome studies often cite this lack of fit as one of the
most likely reasons for undesirable results following system introduction.[24]
[25]
[26]
[27] Most of these reviews, however, draw upon qualitative reports in single institutions,
while large-scale, empirical studies of the environmental factors that facilitate
or impede EHR adoption and use remain virtually absent from the literature.
A substantial body of research has identified features of work environments that support
nurses in their practice and are conducive to high-quality patient care, including
adequate nurse staffing and resources, promotion of nurse autonomy, involvement of
nurses in administrative decision-making, strong nursing leadership, and teamwork
among staff.[28]
[29]
[30]
[31] While Hessels and coworkers[4] considered the work environment as a potential confounder of the relationship between
EHR adoption and patient satisfaction, no studies to our knowledge have considered
the work environment as a primary factor in the relationship between EHR adoption,
end-user usability assessments, and the quality and safety of patient care.
Objective
The objective of this study was to examine the independent and joint effects of comprehensive
EHR system adoption and the work environment on nurse reports of system usability,
including satisfaction and effectiveness, and nurse-reported quality of care and safety.
Methods
Study Design
This study was a secondary analysis of three linked data sources, including (1) the
2015 to 2016 RN4CAST-US nurse survey,[32] (2) the 2015 American Hospital Association (AHA) Annual Survey Database,[33] and (3) the 2015 AHA Healthcare Information Technology (IT) Database.[34] Data were linked using a common hospital identifier. The analysis included hospitals
that had at least 10 nurse survey respondents to provide reliable estimates of the
work environment[32] and were represented in the AHA Annual Survey and IT databases. The response rate
for the AHA Annual Survey exceeds 75% each year,[35] while the response rate to the 2015 AHA Healthcare IT Survey was 56%.[2]
Data Sources
The RN4CAST-US survey[32] was mailed to the home addresses of a random sample of 231,000 registered nurses
(RNs) obtained from state licensure lists across four U.S. states, including California,
Pennsylvania, New Jersey, and Florida. The RN4CAST-US survey instrument included questions
regarding nurse demographics, patient record system usability, and validated measures
of the work environment, staffing, quality of care and patient safety. Nurses who
reported working in hospitals were also asked to provide the name of their employing
hospital. This made it possible to match nurses to hospitals and allowed for the aggregation
of individual nurse responses to the hospital level. Ultimately, we obtained responses
from 12,377 staff nurses working in direct patient care in 353 hospitals in the four
states for which we also had AHA Annual Survey and IT data. On average, each hospital
included in our analysis was represented by 35 nurse respondents.
The 2015 AHA Annual Survey provided information related to structural characteristics
of hospitals, including teaching status, size, and ability to perform high-technology
procedures, core based statistical area (CBSA)—a census-based measure of population
density—and ownership. The 2015 AHA IT Database provided information on the degree
to which hospitals have implemented a set of 31 individual functionalities related
to the following six areas: (1) electronic clinical documentation, (2) results viewing,
(3) computerized provider order entry (CPOE) for laboratory and radiology tests, medication,
consultation requests, and nursing orders, (4) decision support, (5) bar coding, and
(6) other functionalities (e.g., telemedicine). Each of the functionalities are coded
on a 6-point scale of: (1) fully implemented across all units; (2) fully implemented
in at least one unit; (3) beginning to implement in at least one unit; (4) have resources
to implement in the next year; (5) do not have resources but considering implementing;
and (6) not in place and not considering implementation.
Measures
EHR Adoption Level
Using the AHA IT Database, we classified hospitals into two groups based on their
reported EHR adoption level: (1) basic system or less or (2) comprehensive system.
Following definitions provided by the Office of the National Coordinator for Health
Information Technology,[2] hospitals were classified as having a basic EHR if the following were either fully
implemented on at least one clinical unit or across all units: (1) electronic clinical
documentation of demographics, problem lists, medication lists, and discharge summaries,
(2) electronic laboratory, radiologic and diagnostic test results, and (3) CPOE. Given
most hospitals have some form of EHR technology,[2] we combined hospitals with a basic EHR system with those that did not meet the requirements
of a basic system. Comprehensive EHR systems were defined as the presence of the three
core components of a basic system (i.e., electronic clinical documentation of demographics,
problem lists, medication lists, and discharge summaries; electronic laboratory, radiologic
and diagnostic test results, and CPOE), as well as 14 additional functionalities that
have been implemented fully across all units.[2]
Work Environment
The hospital work environment was measured using the Practice Environment Scale of
the Nursing Work Index (PES-NWI), a 31-item instrument that has been endorsed by the
National Quality Forum as a patient safety performance measure.[31]
[36] The PES-NWI is one of the most commonly used and reliable instruments available
to conduct comparisons of organizational work factors among hospital nurses.[37]
[38] Nurses were asked to indicate the degree to which various organizational features
are present in their practice setting. A hospital-level measure of the work environment
was created by first aggregating individual nurse responses for each of the five established
PES-NWI subscales. The subscales include nurse participation in hospital affairs (9
items), nursing foundations for quality care (10 items), nurse manager ability, leadership,
and support of nurses (5 items), staffing and resource adequacy (4 items), and collegial
nurse–physician relations (3 items).[31] A PES composite score for each hospital was created by calculating the mean of the
five hospital-level subscales. For this analysis, we used the PES composite score
for each hospital to categorize work environments as follows: better (≥75th percentile),
mixed (26–74th percentile), and poor (≤25th percentile).[29]
[39]
EHR Usability
Nurse respondents rated their satisfaction with their current patient record system
on a 4-point Likert scale ranging from “very satisfied” to “very dissatisfied.” Nurses
were also asked to indicate their level of agreement with a set of seven items adapted
from previous studies[40]
[41] that measured their evaluations of the system's effectiveness in their daily work.
On a 4-point Likert scale ranging from “strongly agree” to “strongly disagree,” nurses
reported on: ability to quickly access patient information, interference of the system
in the provision of care, ease of use, trust in the accuracy of patient assessment
and medication information, the system's ability to support work efficiency, and the
ability of the system to easily share information between team members. Responses
were dichotomized into two categories (i.e., satisfied/dissatisfied; agree/disagree)
to facilitate interpretation of the results. Respondents were also asked to respond
to an item inquiring whether nurses were involved in choosing or modifying the system.
Nurse-Reported Quality of Care and Patient Safety
Respondents were also asked to rate the quality of care provided in their work setting
on a 4-point Likert scale of excellent, good, fair, or poor. Responses of fair/poor,
and excellent/good were combined for the analysis. Using an item adapted from the
Agency for Healthcare Research and Quality's Patient Safety Culture Survey,[42] nurses assigned an overall grade for their unit on patient safety as: A: excellent;
B: very good; C: acceptable; D: poor; or F: failing. Grades of C, D, or F were defined
as a poor safety grade.
Covariates
Hospital structural characteristics were derived from AHA Annual Survey data. These
characteristics have been shown to be related to quality of care as well as EHR adoption.[2] Teaching status was classified into three categories based on the ratio of medical
trainees to beds: none, minor (<1:4), and major (>1:4). Size was classified as small
(≤100 beds), medium (101–250 beds), and large (>250 beds). Hospitals with high technology
capability were identified as facilities that provide services for open-heart surgery,
organ transplantation, or both. CBSA type was classified as metropolitan, micropolitan,
or rural. Ownership was categorized as government nonfederal, nonprofit, and for-profit.
State was specified as California, Pennsylvania, New Jersey, or Florida. Nurse characteristics
obtained from the RN4CAST-US survey included age, sex, years of RN experience, and
education level (bachelor's degree in nursing or higher) and were included as covariates
because of their associations with EHR acceptance.[13]
[14]
[24] Using nurse responses from RN4CAST-US survey, we also accounted for nurse staffing
by creating a hospital-level measure that was derived by dividing the average number
of patients present on each unit during the last shift by the average number of RNs
present on the last shift.
Data Analysis
Descriptive statistics were calculated to assess differences in hospital and nurse
characteristics by EHR adoption level. EHR usability measures and nurse-reported patient
outcomes were examined descriptively for nurses who worked in hospitals with and without
comprehensive EHRs and by work environment classification. Robust logistic regression
models that accounted for the clustering of nurses within hospitals were used to examine
the effect of comprehensive EHR system adoption and the work environment on outcomes,
first separately, then jointly, and finally in a fully adjusted model that accounted
for other hospital and nurse characteristics. Odds ratios (ORs) created by exponentiating
the regression coefficients are presented for ease of interpretation, along with 95%
confidence intervals (CIs). An interaction term between comprehensive EHR system adoption
and work environment was tested, but was not statistically significant. Analyses were
conducted with SAS version 9.3, (SAS Institute, Inc, Cary, North Carolina, United
States) and results were considered statistically significant at p < 0.05.
Results
Of the 353 hospitals in the sample, 157 (44%) had adopted a comprehensive EHR system,
while 196 (56%) had a basic system or less. Among hospitals with a basic EHR system
or less, 53 (15%) had less than a basic system in place. [Table 1] shows the characteristics of nurses and hospitals in the sample by level of EHR
adoption. Of note, higher proportions of nurses working in hospitals with comprehensive
EHR systems held a bachelor's degree in nursing or higher (58.9 vs. 55.2%, p < 0.001) and, on average, had more years of RN experience (18.9 vs. 17.9, p < 0.001) compared with nurses working in hospitals with basic systems or less. Among
hospital characteristics, significant variation in EHR adoption level was observed
by teaching status and ownership (both p < 0.001). There were no statistically significant differences in the adoption of
comprehensive EHRs by hospital size, technology status, state, or CBSA.
Table 1
Distribution of nurse and hospital characteristics by level of electronic health record
(EHR) adoption
|
|
EHR adoption level
|
|
Nurse characteristics
|
All
(n = 12,377 nurses)
|
Basic EHR or less
(n = 6,133)
|
Comprehensive EHR
(n = 6,244)
|
p-Value[a]
|
|
Age, mean (SD)
|
46.8 (12.3)
|
46.7 (12.2)
|
46.9 (12.3)
|
0.43
|
|
Female, n (%)
|
11,158 (90.4%)
|
5,487 (89.6%)
|
5,671 (91.1%)
|
0.004
|
|
Highest nursing degree, baccalaureate, n (%)
|
7,034 (57.0%)
|
3,369 (55.2%)
|
3,665 (58.9%)
|
<0.001
|
|
Years of RN experience, mean (SD)
|
18.4 (12.9)
|
17.9 (12.7)
|
18.9 (13.0)
|
<0.001
|
|
Hospital characteristics
|
All (
n
= 353 hospitals)
|
Basic EHR or less
(
n
= 196)
|
Comprehensive EHR (
n
= 157)
|
p
-Value
[a]
|
|
Size, n (%)
|
|
≤ 100 beds
|
16
|
9 (56.3%)
|
7 (43.8%)
|
0.99
|
|
101–250 beds
|
127
|
70 (55.1%)
|
57 (44.9%)
|
|
> 250 beds
|
210
|
117 (55.7%)
|
93 (44.3%)
|
|
Teaching status, n (%)
|
|
Nonteaching
|
148
|
66 (33.7%)
|
82 (52.2%)
|
<0.001
|
|
Minor teaching
|
165
|
114 (58.2%)
|
51 (32.5%)
|
|
Major teaching
|
40
|
16 (8.2%)
|
24 (15.3%)
|
|
Technology capability, n (%)
|
|
Low
|
136
|
78 (57.4%)
|
58 (42.7%)
|
0.58
|
|
High (performs open heart surgery and/or organ transplants)
|
217
|
118 (54.4%)
|
99 (45.6%)
|
|
State, n (%)
|
|
California
|
138
|
81 (58.7%)
|
57 (41.3%)
|
0.46
|
|
Florida
|
95
|
55 (57.9%)
|
40 (42.1%)
|
|
New Jersey
|
41
|
19 (46.3%)
|
22 (53.7%)
|
|
Pennsylvania
|
79
|
41 (51.9%)
|
38 (48.1%)
|
|
Core-based statistical area (CBSA), n (%)
|
|
Metropolitan
|
342
|
188 (55.0%
|
154 (45.0%)
|
0.32
|
|
Micropolitan
|
8
|
5 (62.5%)
|
3 (37.5)
|
|
Rural
|
3
|
3 (100.0%)
|
0 (0%)
|
|
Ownership, n (%)
|
|
Government, nonfederal
|
32
|
15 (46.9%)
|
17 (53.1%)
|
<0.001
|
|
Nonprofit
|
265
|
134 (50.6%)
|
131 (49.4%)
|
|
For-profit
|
56
|
47 (83.9%)
|
9 (16.1%)
|
Abbreviations: RN, registered nurse; SD, standard deviation.
Note: Percentages may not add to 100 due to rounding and small amounts of missing
data (<5%) on nurse characteristics. A total of 53 hospitals had adopted less than
a basic EHR.
a
p-values generated from chi-square for all variables, except for CBSA where Fisher's
exact test was used.
Nurse assessments of the system usability and quality of care by level of EHR adoption
are displayed in [Table 2]. Overall, 25.1% of nurses expressed dissatisfaction with the EHR system. The percentage
of nurses who were dissatisfied was slightly lower in hospitals with comprehensive
EHRs compared with hospitals with a basic system or less (21.6 vs. 28.7%, p < 0.001). Across all outcomes studied, the percentage of nurses reporting poor usability
outcomes with the EHR was significantly lower in hospitals with comprehensive systems.
Notably, over half (55.4%) of the surveyed nurses reported that EHRs interfered with
patient care, while nearly one-third (31.9%) reported that they were not easy to use
and did not help them to do their work in an efficient way (32.2%). Nearly half (48.5%)
of survey respondents reported that nurses were not involved in choosing or modifying
the system; however, this percentage was slightly lower in hospitals adopting comprehensive
systems (46.4 vs. 50.7%, p < 0.001). Compared with hospitals with basic EHR systems or less, fewer nurses in
hospitals with comprehensive systems reported poor quality of care (9.9 vs. 14.0%,
p < 0.001) and a poor patient safety grade (26.5 vs. 30.3%, p < 0.001).
Table 2
Nurse assessments of electronic health record (EHR) system usability and quality of
care in hospitals by level of EHR adoption
|
|
N (%) of nurses agreeing with statement who work in hospitals with:
|
|
Outcomes
|
All (n = 12,377)
|
Basic EHR or less (n = 6,133)
|
Comprehensive EHR (n = 6,244)
|
p-Value[a]
|
|
Not satisfied with the system
|
2,597 (25.1%)
|
1,462 (28.7%)
|
1,135 (21.6%)
|
<0.001
|
|
The systems do not make it easy to access a patient's clinical data quickly
|
2,034 (19.7%)
|
1,204 (23.7%)
|
830 (15.8%)
|
<0.001
|
|
The systems interfere with the provision of patient care
|
5,668 (55.4%)
|
2,848 (56.7%)
|
2,820 (54.2%)
|
0.009
|
|
The systems are not easy to use
|
3,290 (31.9%)
|
1,809 (35.7%)
|
1,481 (28.2%)
|
<0.001
|
|
I do not trust the accuracy of the patient assessment data documented in the systems
|
1,802 (17.5%)
|
955 (18.8%)
|
847 (16.2%)
|
<0.001
|
|
I do not trust the accuracy of the medication information in the systems
|
1,145 (11.1%)
|
617 (12.2%)
|
528 (10.1%)
|
<0.001
|
|
The systems do not help me to do my work in an efficient way
|
3,321 (32.2%)
|
1,778 (35.1%)
|
1,543 (29.4%)
|
<0.001
|
|
The systems do not make it easy to share information in a timely way with other members
of the health care team
|
2,107 (20.5%)
|
1,248 (24.6%)
|
859 (16.4%)
|
<0.001
|
|
Nurses were not involved in choosing (or modifying) the patient record system
|
3,183 (48.5%)
|
1,643 (50.7%)
|
1,540 (46.4%)
|
<0.001
|
|
Quality of care on unit is fair or poor
|
1,244 (11.9%)
|
721 (14.0%)
|
523 (9.9%)
|
<0.001
|
|
Poor overall unit grade on patient safety (C,D, or F)
|
2,980 (28.4%)
|
1,565 (30.3%)
|
1,415 (26.5%)
|
<0.001
|
Note: Percentages in the “All” column may not align with overall sample size due to
missing data. Sample sizes range from 6,565 to 10,495.
a
p-values generated from chi-squares.
[Table 3] demonstrates statistically significant differences (p < 0.001) in each of the studied outcomes related to EHR usability and nurse-reported
quality of care and safety across work environment types. Nearly 40% of nurses working
in hospitals with poor work environments reported dissatisfaction with the record
system compared with less than 20% of nurses working in hospitals with better environments.
Also of note, nearly half of nurses (45.6%) working in poor environments reported
that the EHR system did not help them to work efficiently, compared with one-quarter
(25.2%) of nurses in better environments. In hospitals with poor work environments,
over two-thirds (67.6%) of nurses reported that nurses were not involved in choosing
or modifying the record system compared with about one-third (34.4%) of nurses working
in better conditions.
Table 3
Nurse assessments of electronic health record (EHR) system usability and quality of
care in hospitals by work environment type
|
|
N (%) of nurses agreeing with statement who work in
hospitals with a work environment that is:
|
|
Outcomes
|
All (n = 12,377)
|
Poor (n = 2,256)
|
Mixed (n = 6,214)
|
Better (n = 3,907)
|
p-Value[a]
|
|
Not satisfied with the patient record system
|
2,597 (25.1%)
|
730 (38.6%)
|
1,282 (24.7%)
|
585 (17.9%)
|
<0.001
|
|
The systems do not make it easy to access a patient's clinical data quickly
|
2,034 (19.7%)
|
557 (29.7%)
|
1,001 (19.3%)
|
476 (14.6%)
|
<0.001
|
|
The systems interfere with the provision of patient care
|
5,668 (55.4%)
|
1,146 (61.9%)
|
2,874 (55.8%)
|
1648 (51.2%)
|
<0.001
|
|
The systems are not easy to use
|
3,290 (31.9%)
|
816 (43.5%)
|
1,657 (31.9%)
|
817 (25.1%)
|
<0.001
|
|
I do not trust the accuracy of the patient assessment data documented in the systems
|
1,802 (17.5%)
|
522 (27.8%)
|
883 (17.0%)
|
397 (12.2%)
|
<0.001
|
|
I do not trust the accuracy of the medication information in the systems
|
1,145 (11.1%)
|
313 (16.7%)
|
579 (11.2%)
|
253 (7.8%)
|
<0.001
|
|
The systems do not help me to do my work in an efficient way
|
3,321 (32.2%)
|
854 (45.6%)
|
1,649 (31.8%)
|
818 (25.2%)
|
<0.001
|
|
The systems do not make it easy to share information in a timely way with other members
of the health care team
|
2,107 (20.5%)
|
559 (29.8%)
|
1,055 (20.4%)
|
493 (15.2%)
|
<0.001
|
|
Nurses were not involved in choosing (or modifying) the patient record system
|
3,183 (48.5%)
|
819 (67.6%)
|
1,641 (50.5%)
|
723 (34.4%)
|
<0.001
|
|
Quality of care on unit is fair or poor
|
1,244 (11.9%)
|
433 (23.0%)
|
631 (12.0%)
|
180 (5.5%)
|
<0.001
|
|
Poor overall unit grade on patient safety (C,D, or F)
|
2,980 (28.4%)
|
889 (46.8%)
|
1,538 (29.1%)
|
553 (16.7%)
|
<0.001
|
Note: Percentages in the “All” column may not align with overall sample size due to
missing data. Sample sizes range from 6,565 to 10,495.
a
p-values generated from chi-squares.
[Table 4] shows the ORs estimating the effects of EHR adoption level and work environment
type on each of the study outcomes. The first column of the table presents models
where the effects of EHR adoption level and work environment on outcomes were estimated
separately. In these bivariate models, both predictors had a statistically significant
effect on outcomes with adoption of a comprehensive EHR and better work environments
being associated with lower likelihoods of unfavorable usability and nurse-reported
quality and safety outcomes except for two cases where EHR adoption level was not
significant at the p < 0.05 level (system interference with the provision of patient care and nurses were
not involved in choosing/modifying the record system). In jointly estimated models
(second column), both main effects were slightly attenuated but maintained statistical
significance in most outcomes. In three cases where EHR adoption level was significant
when it was estimated separately, the effect was no longer significant once work environment
was accounted for (i.e., trust in the accuracy of patient assessment data, trust in
the accuracy of medication information, and poor patient safety grade).
Table 4
Odds ratios (OR) indicating the effects of a comprehensive electronic health record
(EHR) and work environment on nurse assessments of EHR system usability and quality
of care (n = 12,377)
|
Estimated separately
|
Estimated jointly
|
Fully adjusted
|
|
Outcome
|
OR (95% CI)
|
OR (95% CI)
|
OR (95% CI)
|
|
Not satisfied with the system
|
|
Comprehensive EHR
|
0.68 (0.55–0.84)[a]
|
0.74 (0.61–0.90)[b]
|
0.75 (0.61–0.92)[b]
|
|
Work environment
|
0.59 (0.51–0.68)[a]
|
0.61 (0.52–0.70)[a]
|
0.67 (0.58–0.77)[a]
|
|
System does not make it easy to access a patient's clinical data quickly
|
|
Comprehensive EHR
|
0.60 (0.49–0.74)[a]
|
0.64 (0.53–0.78)[a]
|
0.65 (0.53–0.79)[a]
|
|
Work environment
|
0.64 (0.55–0.73)[a]
|
0.66 (0.58–0.76)[a]
|
0.73 (0.64–0.83)[a]
|
|
The systems interfere with the provision of patient care
|
|
Comprehensive EHR
|
0.90 (0.81–1.01)
|
0.93 (0.84–1.04)
|
0.97 (0.87–1.08)
|
|
Work environment
|
0.81 (0.74–0.88)[a]
|
0.81 (0.75–0.88)[a]
|
0.84 (0.77–0.92)[a]
|
|
Systems are not easy to use
|
|
Comprehensive EHR
|
0.71 (0.59–0.84)[a]
|
0.75 (0.64–0.89)[a]
|
0.78 (0.66–0.93)[b]
|
|
Work environment
|
0.66 (0.59–0.75)[a]
|
0.68 (0.60–0.76)[a]
|
0.76 (0.67–0.86)[a]
|
|
I do not trust the accuracy of the patient assessment data documented in the system
|
|
Comprehensive EHR
|
0.83 (0.71–0.98)[c]
|
0.90 (0.78–1.05)
|
0.94 (0.81–1.10)
|
|
Work environment
|
0.60 (0.54–0.67)[a]
|
0.61 (0.54–0.68)[a]
|
0.64 (0.57–0.72)[a]
|
|
I do not trust the accuracy of the medication information in the systems
|
|
Comprehensive EHR
|
0.81 (0.68–0.96)[c]
|
0.87 (0.73–1.03)
|
0.85 (0.70–1.02)
|
|
Work environment
|
0.65 (0.57–0.73)[a]
|
0.66 (0.58–0.74)[a]
|
0.69 (0.60–0.80)[a]
|
|
Systems do not help me to do my work in an efficient way
|
|
Comprehensive EHR
|
0.77 (0.65–0.91)[b]
|
0.82 (0.71–0.96)[b]
|
0.85 (0.73–1.00)[c]
|
|
Work environment
|
0.64 (0.57–0.72)[a]
|
0.65 (0.58–0.73)[a]
|
0.71 (0.63–0.80)[a]
|
|
Systems do not make it easy to share information in a timely way with other members
of the health care team
|
|
Comprehensive EHR
|
0.60 (0.49–0.73)[a]
|
0.64 (0.53–0.77)[a]
|
0.64 (0.54–0.77)[a]
|
|
Work environment
|
0.65 (0.57–0.75)[a]
|
0.67 (0.59–0.77)[a]
|
0.76 (0.67–0.86)[a]
|
|
Nurses were not involved in choosing/modifying the patient record system
|
|
Comprehensive EHR
|
0.84 (0.69–1.03)
|
0.94 (0.80–1.11)
|
1.01 (0.86–1.18)
|
|
Work environment
|
0.50 (0.45–0.56)[a]
|
0.51 (0.45–0.57)[a]
|
0.59 (0.53–0.67)[a]
|
|
Quality of care on unit is fair or poor
|
|
Comprehensive EHR
|
0.67 (0.56–0.82)[a]
|
0.77 (0.67–0.88)[a]
|
0.83 (0.71–0.96)[c]
|
|
Work environment
|
0.44 (0.40–0.49)[a]
|
0.45 (0.41–0.50)[a]
|
0.47 (0.42–0.52)[a]
|
|
Poor patient safety grade (C, D, or F)
|
|
Comprehensive EHR
|
0.83 (0.71–0.97)[c]
|
0.94 (0.85–1.03)
|
0.98 (0.89–1.09)
|
|
Work environment
|
0.48 (0.44–0.52)[a]
|
0.48 (0.45–0.52)[a]
|
0.50 (0.46–0.54)[a]
|
Abbreviations: CBSA, core-based statistical area; CI, confidence interval; RN, registered
nurse.
Notes: Odds ratios are from robust logistic regression models adjusted for hospital
characteristics (size, teaching status, high-technology capability, ownership, CBSA
type, state, and nurse staffing level), nurse characteristics (age, sex, highest level
of education—baccalaureate in nursing or higher, and years of RN experience), and
the clustering of nurses within hospitals. The odds ratio associated with the work
environment can be interpreted as the difference in the odds of the nurse-reported
outcome between hospitals with “better versus mixed” environments, and between hospitals
with “mixed versus poor” environments.
a
p < 0.001.
b
p < 0.01.
c
p < 0.05.
The last column of [Table 4] presents the fully adjusted models that include EHR adoption level and work environment
in addition to hospital and nurse characteristics. In models related to EHR usability
outcomes, adoption of a comprehensive EHR was associated with lower odds of nurses
reporting dissatisfaction with the system (OR: 0.75; 95% CI: 0.61–0.92), that the
system does not make it easy to access clinical data (OR: 0.65; 95% CI: 0.53–0.79),
the system is not easy to use (OR: 0.78; 95% CI: 0.66–0.93), that the system does
not help in performing work efficiently (OR: 0.85; 95% CI: 0.73–1.00), and that the
system does not make it easy to share information with other members of the health
care team (OR: 0.64; 95% CI: 0.54–0.77). The work environment was associated with
all EHR usability outcomes with nurses in hospitals with better environments being
less likely than nurses in mixed environments to report: overall dissatisfaction with
their current record system environments (OR: 0.67; 95% CI: 0.58–0.77), that the system
does not make it easy to access clinical data (OR: 0.73; 95% CI: 0.64–0.83), system
interference with the provision of patient care (OR: 0.84; 95% CI: 0.77–0.92), that
the system is not easy to use (OR: 0.76; 95% CI: 0.67–0.86), distrust in the accuracy
of patient assessment data (OR: 0.64; 95% CI: 0.57–0.72) and medication information
(OR: 0.69; 95% CI: 0.60–0.80), that the system does not help in performing work efficiently
(OR: 0.71; 95% CI: 0.63–0.80), and that the system does not make it easy to share
information with other health care team members (OR: 0.76; 95% CI: 0.67–0.86). Nurses
in better work environments were also nearly half as likely as those in mixed environments
to report that nurses were not involved in choosing/modifying the system (OR: 0.59;
95% CI: 0.53–0.67). By extension, these models also imply that nurses working in better
environments were significantly less likely than those working in poor environments
to report these negative outcomes. For example, nurses working in better environments
were 55% ([1–0.672] × 100) less likely than nurses in poor environments to report
overall dissatisfaction with the EHR. Among the hospital and nurse characteristics
included in the models as controls, we noted that a different set of these covariates
was associated with each outcome; however, hospital ownership and nurse experience
were two of the covariates that were consistently statistically significant. In fully
adjusted models of nurse-reported quality and patient safety outcomes, the odds of
nurses reporting fair/poor quality of care were significantly lower in hospitals that
had adopted a comprehensive EHR system compared with those that did not (OR: 0.83;
95% CI: 0.71–0.96) and in hospitals with better as opposed to mixed work environments
(OR: 0.47; 95% CI: 0.42–0.52). Compared with nurses in mixed environments, nurses
working in hospitals in better environments were half as likely to report a poor patient
safety grade (OR: 0.50; 95% CI: 0.46–0.54). Adoption of comprehensive EHR system did
not have a significant effect on safety grade after the work environment was taken
into account.
Discussion
Our findings suggest that adoption of a comprehensive EHR is associated with more
positive usability ratings by nurses and higher nurse-reported quality of care. We
also found that—independent of EHR adoption level—the work environment plays a significant
role in how nurses evaluate EHR usability. EHR adoption level was associated with
6 of the 11 outcomes studied, while the work environment was associated with all 11.
This result implies that adoption of advanced EHR systems and the quality of the work
environment both play significant roles in the delivery of high quality care, and
also that the work environment may play a significant role in the success of EHR system
adoption.
To our knowledge, this is one of the first studies to examine the concurrent effects
of EHR adoption level and the hospital work environment on usability and quality outcomes.
In models where EHR adoption level and work environment were considered separately,
both variables had significant associations with study outcomes. However, when both
factors were considered simultaneously in fully adjusted models, we noted that—in
most cases—the effect of EHR adoption level on outcomes was moderated and sometimes
rendered insignificant after including the work environment. This moderation effect
was especially notable for the outcome of poor/failing patient safety grade, and suggests
that the work environment may play a more important role in the delivery of safe patient
care than the type of EHR system. A similar phenomenon was also observed by Hessels
and coworkers[4] who noted that one element of the work environment—staffing and resource adequacy—moderated
the relationship between EHR adoption level and patient satisfaction in a sample of
New Jersey hospitals. Other research has documented that face-to-face communication
between nurses and physicians decreases following EHR implementation,[43] which suggests that having a strong culture of interprofessional teamwork—another
component of good work environments—becomes even more important with the introduction
of electronic record systems. Taken together, these findings provide empirical evidence
for the consideration of the hospital work environment as a critical factor in the
implementation and use of hospital EHR systems.
With a sample of over 12,000 nurses, our study represents one of the largest to examine
nurse satisfaction with EHR systems and reports of their clinical utility. Although
nurses working in hospitals with comprehensive EHR systems and in the highest-rated
work environments reported the most favorable outcomes, our study highlights a significant
need for improvement across hospitals at all levels of EHR adoption. Overall, we found
that 25% of nurses are dissatisfied with their current record systems while similarly
high percentages reported usability issues. Overall, nearly 50% of nurses in our sample
reported that they were not involved in the selection or modification of the EHR.
This is a concerning finding given that nurses are one of the primary groups of EHR
end-users and may provide an explanation for the significant proportions of nurses
reporting dissatisfaction and poor usability. Across all studied EHR usability outcomes,
nurses working in better work environments were significantly less likely to assign
negative evaluations of the system compared with nurses working in less favorable
environments, which suggests that strengthening organizational structures, such as
nurse leadership and engagement in every stage of EHR adoption from system selection
to the post-implementation period, may serve as potential points of intervention.
This set of findings is aligned with prior research demonstrating that successful
EHR implementation is more likely when EHRs are perceived by clinicians to be helpful
to them in their practice and to have positive effects on patient outcomes.[9]
[44]
[45] A concerted effort is also needed to improve hospital work environments: nearly
one-third of nurses in our sample reported working in fair or poor conditions.
As nearly all U.S. hospitals have adopted some form of EHR technology, the focus now
turns toward the adoption of advanced functionalities that relate to interoperability,
performance measurement, and patient engagement.[46] Evidence is mounting that use of these advanced functionalities—that are often included
in comprehensive EHR systems—is needed to achieve the EHR's desired effects.[3]
[4]
[7] Our study adds to this evidence base by demonstrating that nurses are less likely
to report poor quality in hospitals with comprehensive EHR systems, and that comprehensive
systems are more likely to support nurses in the complex tasks of patient care delivery.
This finding provides valuable evidence to policymakers and to hospitals enrolled
in the Medicare and Medicaid EHR Incentive Program by demonstrating that investment
in advanced systems may translate into the delivery of higher quality of care. As
participants in this program, eligible hospitals must attest to regularly scheduled
advancements in the functionality of their EHR system to avoid payment penalties.
Future work should explore associations between EHR system capabilities, usability,
and clinical patient outcomes, such as mortality and readmissions.
Limitations
Our cross-sectional analysis limits the ability to draw causal inferences about the
relationships we observed. We also acknowledge the potential bias that may be introduced
due to unmeasured variables. For example, we were unable to account for the number
of years that the EHR system was in place for each hospital. The more experience that
clinicians have with EHR systems may be associated with more favorable observations
of their effectiveness,[13]
[16] as well as better patient outcomes.[47] Few rural hospitals were included in our sample which limits conclusions that can
be made about EHR use in those hospitals. Recent research suggests a widening divide
in adoption of comprehensive EHRs between urban and rural hospitals.[48] Our sample was limited to hospitals in California, New Jersey, Pennsylvania, and
Florida, which may limit the generalizability of our findings; however, the hospitals
located in these four states represent over 20% of all acute care hospitals nationally,
and discharge over 25% of all patients in the country. Further, a previous comparison
of RN4CAST-US and 2017 Current Population Survey data found that hospital RNs in these
four states are similar to hospital RNs nationally on characteristics such as age
and education.[49] Finally, our categorization of basic EHR system or less includes hospitals that
vary widely in their adoption levels, ranging from a basic system implemented in all
units, a basic system implemented on some units, or less than a basic system. However,
the primary aim of this study was to examine whether the adoption of comprehensive
EHR systems with advanced capabilities—the desired goal for all U.S. hospitals—was
associated with more favorable reports of EHR usability and quality of care.
Conclusion
EHRs, particularly those with advanced capabilities, have been widely promoted as
a means to decrease health care costs, improve efficiency, and optimize patient safety.
Our study findings suggest that adoption of a comprehensive EHR system is associated
with greater nurse satisfaction with the system, more favorable reports of the system's
usability, and higher quality of care. The value of EHR may not be fully realized,
however, unless we understand the context in which they most effectively function.
Independent of the EHR adoption level, we found that the work environment is highly
associated with nurses' ratings of the usefulness of EHR systems, and most importantly
that the quality of the work environment may determine the extent to which comprehensive
EHR systems have their intended impact on quality and safety. The best patient outcomes
may be achieved by concurrently implementing comprehensive EHR systems and improving
hospital work environments.
Clinical Relevance Statement
Clinical Relevance Statement
One of the leading reasons for poor outcomes following EHR adoption may be the lack
of consideration for how the system will interface with end-users and the existing
organizational structure. Adoption of a comprehensive EHR system is associated with
greater satisfaction and usability ratings by nurses, as well as higher nurse-reported
quality of care. The best outcomes may be achieved by concurrently implementing comprehensive
EHR systems and improving hospital work environments.
Multiple Choice Questions
Multiple Choice Questions
-
The findings of this study suggest that the most favorable outcomes related to EHR
adoption may be achieved by implementing a comprehensive EHR system and improving
what organizational feature?
Correct Answer: The correct answer is option a, the hospital work environment. Our findings suggest
that adoption of a comprehensive EHR system is associated with greater nurse satisfaction
with the system, more favorable reports of the system's usability, and higher quality
of care. Independent of the EHR adoption level, we found that the work environment
is highly associated with nurses' ratings of the usefulness of EHR systems. The quality
of the work environment may determine the extent to which comprehensive EHR systems
have their intended impact on health care quality and safety.
-
What is an example of a work environment-focused intervention that could potentially
improve nurse perceptions of electronic health record (EHR) system usability?
-
Hire fewer nurses.
-
Provide handouts on the benefits of the system for nurses to read.
-
Strengthen nurse leadership and engagement in all phases of EHR adoption.
-
Conduct seminars on communication on every unit.
Correct Answer: The correct answer is option c, strengthen nurse leadership and engagement in all
phases of EHR adoption. Sociotechnical theory suggests that EHR systems cannot be
implemented successfully without thoughtful consideration of the “fit” of the technology
with the people providing care and the existing context of care delivery, including
the work environment. Our study found that the hospital work environment was highly
associated with nurses' ratings of the usefulness of EHR systems. Key features of
the work environment include adequate nurse staffing and resources, promotion of nurse
autonomy, involvement of nurses in administrative decision-making, strong nursing
leadership, and teamwork.
