Keywords
optimization - virtual EHR training - sprint - virtual training - virtual optimization
- strategies for health IT training - clinical informatics - professional training
- education - distance - process improvement - system improvement - efficiency improvement
- performance improvement
Background and Significance
Background and Significance
In 2020, U.S. health care systems faced the unprecedented collision of health care
worker (HCW) burnout[1] and the novel coronavirus disease 2019 (COVID-19; severe acute respiratory syndrome-coronavirus-2
[SARS-CoV-2]) pandemic. Uncertainty of viral transmission, limitations of physical
infrastructure, and workforce shortages created new challenges but also promoted flexibility,
innovation, and teamwork.[2]
[3] The rapid innovation in health care delivery and the electronic health record (EHR)
accelerated development, training, and use of tools including tablets and remote monitors,
COVID-19 treatment clinical pathways, and training for routine use of video or virtual
patient encounters.[3] Health care systems faced a conundrum in growing need for EHR training and optimization
in a health care workforce suffering from high rates of burnout.
Postimplementation, EHR training has been shown to improve EHR satisfaction[4]
[5]
[6] and burnout[7]
[8] when content is personalized and/or delivered elbow-to-elbow. To our knowledge,
a fully virtual EHR training program has not been studied. Wellness experts have called
for organizationally funded, comprehensive initiatives that role model–effective leadership,
promote values alignment, and enhance communication and teamwork.[9]
[10]
[11] At UCHealth, we utilize our Sprint EHR Training and Optimization program for this
purpose.
The Sprint Team is a multidisciplinary group of medical informaticists, project managers,
EHR trainers, and EHR analysts that partner with ambulatory clinics to explore, investigate,
simplify, or solve local health care delivery issues with workflow and technical solutions.[8] Core program components include group and one-to-one EHR training in addition to
user-centered EHR design.[12] Our full-time Sprint Team conducts 1 to 4 weeks, clinic-specific EHR training and
optimization events throughout the year. In addition, Sprint physician and nurse informaticists
actively balance the needs of specific clinics against the strategic needs of the
health system.
The UCHealth Sprint Team engages the entire clinic care team, including providers
(physician and advanced practice providers), nurses, medical assistants, and technical
and administrative staff. From 2016 to 2020, the Sprint Team conducted onsite Sprint
events in >100 clinics and in March 2020, the team began to reimagine sprint events
due to stay-at-home orders in Colorado. With a goal to continue to improve teamwork,
increase EHR satisfaction, and reduce provider and staff burnout, The Sprint Team
pivoted to a fully virtual model in May 2020 and conducted sprints in 21 ambulatory
clinics from May to December 2020. We report on this experience.
Objective
This study aimed to develop a virtual EHR training and optimization program and evaluate
the impact of the virtual model on provider and staff burnout and EHR experience.
Methods
Sprints at UCHealth
The UCHealth system funded a 20-person Sprint Team that was responsible for clinic-based
Epic EHR (version 2020, Epic Systems) training and optimization for the organization.
The Sprint Team was subdivided equally into two smaller teams which concomitantly
ran two separate Sprint events. The team for each Sprint event includes a full-time
project manager (PM), a clinical informaticist (CI), a physician informaticist (PI),
four ambulatory-certified EHR trainers, and three ambulatory-certified EHR analysts.
[Fig. 1] elucidates roles of each Sprint Team member.
Fig. 1 Sprint and clinical team roles and responsibilities.
Prior to March 2020, all members of the Sprint Team except the analysts were colocated
onsite in target clinics from 8 a.m. to 5 p.m. The number of providers in each clinic
determines Sprint duration were typically 1 to 4 weeks (e.g., 20 providers = 1-week
Sprint, 40 providers = 2-week Sprint, and so on). Small clinics sometimes combine
with other clinics to ensure full utilization of the Sprint Team resource. Sprint-Team
leaders selected clinics based on interest, engagement, and strong leadership whenever
possible. Clinics block provider schedules, so most providers and staff do not have
clinical responsibilities during training, but providers do not receive reimbursement
for loss of clinical revenue. Twenty-one clinics participated in Sprints from May
to December 2020 but only 19 clinics received the same pre- and postsurvey. Pre- and
postsurvey responses are anonymous and do not reveal the identity of the participants.
[Table 1] lists clinic participants who participated in virtual Sprints from May to December
2020.
Table 1
Participants by role in virtual sprint events from May 2020 to December 2020
|
Providers
|
Staff
|
|
Role
|
Number
|
Role
|
Number
|
|
Physician
|
177
|
|
|
|
Advanced practice provider
|
110
|
Nurse, medical assistant
|
344
|
|
Resident
|
24
|
Front office staff
|
82
|
|
Speech, physical, occupational therapist
|
126
|
Clinic manager
|
49
|
|
Other
|
56
|
Other
|
62
|
|
Total
|
493
|
Total
|
538
|
Microsoft Teams
UCHealth was in the process of adopting Microsoft Teams (Microsoft Inc., Redmond,
Washington, United States, Teams version 1.0) as the preferred virtual platform at
the start of the pandemic. This comprehensive platform provided tools for video meetings,
recording capability, individual and group messaging, discreet event channels, document
and screen sharing, and a shared wiki page. As a team, we developed and iterated the
Sprint program with various Teams tools (channels, folders, threads, posts, chat,
and video) to replicate key Sprint processes. Perhaps the most widely appreciated
was video streaming to maintain interpersonal connection within the Sprint Team and
with clinics. Other examples included the consistent use of posts, notifications to
maintain closed loop communication, and specific emojis, for example, a thumbs up
to indicate a new request would be tracked. For the target clinic, the PM sent a Teams
video and tips with training session invitations, and trainers were available before,
during, and after all meetings to assist. Video use facilitated communication during
huddles, in the virtual conference room, and in workflow sessions. Teams and e-mail
communication to virtual Sprint participants included schedules sent before the Sprint
begins and included just-in-time education on Microsoft Teams.
Fig. 2 Standard Sprint program with virtual adaptations.
Virtual Sprint Conversion
The basic structure of Sprints remained intact during the transition to a virtual
model ([Fig. 2]). Pre-Sprint meetings needed for planning, blocking schedules, and setting expectations
with clinics remained unchanged. These meetings had been and continued to be conducted
virtually due to geographic variability in work locations of the Sprint Team and the
clinical teams. [Fig. 3] details all modifications made from the on site to the virtual model.
Fig. 3 Modifications of key sprint components in a virtual model.
Standard all-clinic training sessions, kick off (EHR personalization) and wrap up
(new tool and solidified workflow presentation) changed to a virtual format through
Teams. The curriculum content and informaticist session leaders did not change. The
project managers and Sprint trainers remained available prior to and throughout group
sessions to help participants utilize Teams and to assist with end-user questions.
Session leaders made proactive suggestions that participants keep a list of items
they wanted to investigate further in their 1:1 training sessions.
To compensate for the lack of direct workflow observation in clinical areas, the Sprint
Team developed virtual workflow group sessions. These sessions, facilitated by the
CI and PI, held early in the Sprint, included integral members of the clinical care
team. The clinical team members would share their screens during this exercise, so
the Sprint team members could better understand the workflows, and clinic members
could share pain points and ideas for improvement in workflow.
To foster communication during onsite Sprints, the PM, CI, and PI colocated in a clinic
conference room. Providers or staff could drop in with questions, EHR optimization
ideas, or extra training session requests. To create this atmosphere in a virtual
model, the Sprint Team hosted a Virtual Conference Room (VCR), an all-day Teams meeting
scheduled from 8 a.m. to 5 p.m. All Sprint participants received daily meeting invitations
from the PM via Microsoft Outlook prior to the start of Sprint.
New with virtual Sprints, we introduced the concept of ad hoc group teaching sessions
taught by the PI or CI. In onsite Sprints, opportunities for trainer assistance included
1:1 training and clinic rounding. In virtual Sprints, clinic participants chat or
joined the VCR with questions. To provide ample opportunity for training on basic
topics such as in basket, documentation, clinical review, ordering, and mobile/telemedicine
tools, we added optional virtual group training sessions to supplement 1:1 training.
During these sessions, the PI or CI spent 10 to 20 minutes covering a topic, followed
by staff and/or provider discussion of how the tool applies to their clinic workflows.
Most topics were determined and scheduled during pre-Sprint meetings, but others could
be added as needs arise during Sprint. Group sessions were offered twice to accommodate
various schedules. To encourage participation in the group sessions, workflow sessions,
and the VCR, a daily e-mail from the PM provided hyperlinks to the day's events.
Data Collection
All Sprint participants received an e-mail with a survey link from the Sprint PM 1
to 2 weeks prior to Sprint and, again, a few days before the Sprint. Post-Sprint surveys
were distributed similarly on the last day of Sprint and again 1 week after Sprint.
Staff and providers received separate survey links as some sections of the survey
differed including barriers to participation and specific roles. The surveys asked
questions about EHR usability and proficiency, job satisfaction, and burnout.[13] Satisfaction with the EHR and with the Sprint itself were measured using the Net
Promoter Score (NPS), a likelihood to recommend tool.[14] The NPS asked “How likely is it you would recommend X to a friend or colleague,”
on a scale of 0 to 10. Scores 9 to 10 are considered promoters, 7 to 8 are considered
passive, and 0 to 6 are considered detractors. The percentage of detractors is subtracted
from the percentage of promoters to arrive at the NPS. A score of +50 or above is
considered excellent.[14] Informal staff and provider quotes were collected throughout the Sprint event and
were also extracted from our postsurvey comments.
Statistical Analysis
Descriptive statistics were used to summarize survey responses. The provider and staff
survey responses were dichotomized and compared pre- and post-Sprint using tests of
proportions. The distribution of promoters, passives, and detractors from the NPS
were compared using Chi-square tests. All analyses were performed using R software
(v4.04).
Results
The response rate for pre-Sprint and post-Sprint provider surveys was 52 (257/493)
and 40% (195/493), respectively. The response rate for pre-Sprint and post-Sprint
staff surveys was 29 (156/538) and 31% (169/538). The EHR NPS score, likelihood to
recommend the Epic EHR, increased by 39 points (−3 pre and 36 post; p < 0.001) for providers and 29 points (8 pre and 37 post; p = 0.001) for staff after the Sprint intervention. Providers (NPS = +53) and staff
(NPS = +47) also indicated a high likelihood to recommend the Sprint to their colleagues
([Fig. 4]).
Fig. 4 Net promoter score (NPS) for 21 clinics undergoing virtual sprints.
There was a trend toward improved EHR usability and proficiency in all domains pre-to-post-Sprint
for both staff (differences of 2–13%) and providers (differences of 6–17%; [Table 2]). Respondents agreed slightly or strongly that improvements in EHR-facilitated communication
with providers outside of UCHealth: a 17% improvement (p < 0.001) for providers and 13% improvement (p = 0.02) for staff. The EHR was also found to facilitate communication with colleagues,
affiliates, and patients and these measures improved after the Sprint. A trend toward
an increase in providers who indicated no/did not feel burnout was achieved post-Sprint
for providers (10%; p = 0.04) and staff (9%; p = 0.13; [Table 2]). Regarding job satisfaction and teamwork, results were more variable ([Table 2]). Post-Sprint, for providers, there was a trend toward improvement in control over
workload (11%; p = 0.02), sufficiency of time for documentation (15%; p < 0.001), and care team efficiency (8%; p = 0.09). For staff, improvements were smaller but present for control over workload
(6%; p = 0.36), sufficiency of time for documentation (7%; p = 0.23), and care team efficiency (5%; p = 0.43).
Table 2
Survey responses
|
Survey question
|
Provider responses
|
Staff responses
|
|
Pre-Sprint (n = 257)
(%)
|
Post-Sprint (n = 195)
(%)
|
Delta %
(p-Value)
|
Pre-Sprint (n = 156)
(%)
|
Post-Sprint (n = 169)
(%)
|
Delta (%)
(p-Value)
|
|
The Epic EHR facilitates communication about patients with members of my clinic.
Agree slightly/strongly
|
218
(85)
|
182
(94)
|
+9
(0.003)
|
133
(87)
|
156
(93)
|
+6
(0.08)
|
|
The Epic EHR facilitates communication about patients with UCHealth colleagues.
Agree slightly/strongly
|
182
(71)
|
156
(81)
|
+10
(0.02)
|
97
(63)
|
125
(75)
|
+11
(0.04)
|
|
The Epic EHR facilitates communication about patients with non-UCHealth clinicians.
Agree slightly/strongly
|
55
(21)
|
74
(38)
|
+17
(<0.001)
|
49
(32)
|
75
(45)
|
+13
(0.02)
|
|
The Epic EHR facilitates communication about patients with patients themselves.
Agree slightly/strongly
|
200
(78)
|
173
(90)
|
+12
(0.002)
|
118
(77)
|
136
(82)
|
+5
(0.36)
|
|
The amount of time I spend on the electronic medical record (EMR) at home is: minimal/none
|
123
(48)
|
105
(54)
|
+6
(0.26)
|
123
(79)
|
135
(81)
|
+2
(0.76)
|
|
My proficiency with EMR use is: good/optimal
|
108
(42)
|
101
(52)
|
+9
(0.06)
|
93
(60)
|
109
(64)
|
+4
(0.47)
|
|
Overall, I am satisfied with my current job.
Agree slightly/strongly
|
217
(85)
|
171
(88)
|
+2
(0.58)
|
132
(85)
|
154
(92)
|
+7
(0.07)
|
|
I feel a great deal of stress because of my job.
Disagree slightly/strongly
|
75
(29)
|
59
(30)
|
+1
(0.88)
|
48
(31)
|
45
(27)
|
-4
(0.50)
|
|
Using your definition of “burnout,” please choose one of the answers below: no burnout/do
not feel burned out
|
150
(59)
|
133
(69)
|
+10
(0.04)
|
89
(57)
|
110
(66)
|
+9
(0.13)
|
|
My control over my workload is: good/optimal
|
68
(26)
|
73
(37)
|
+11
(0.02)
|
65
(42)
|
81
(48)
|
+6
(0.36)
|
|
Sufficiency of time for documentation is: good/optimal
|
43
(17)
|
62
(32)
|
+15
(<0.001)
|
68
(44)
|
84
(51)
|
+7
(0.23)
|
|
Which number best describes the atmosphere in your primary work area?
Calm/in between
|
45
(18)
|
30
(15)
|
−2
(0.58)
|
49
(32)
|
50
(30)
|
−2
(0.84)
|
|
My professional values are well aligned with those of my department leaders.
Agree slightly/strongly
|
176
(69)
|
144
(75)
|
+6
(0.23)
|
105
(69)
|
124
(75)
|
+7
(0.24)
|
|
The degree to which my care team works efficiently together is: good/optimal
|
155
(61)
|
135
(69)
|
+8
(0.09)
|
109
(70)
|
125
(75)
|
+5
(0.43)
|
One illustrative survey comment stated, “This Sprint training is the most productive
event I ever had at UCHealth.” One provider sent this message to the Sprint Team a
few weeks after her clinic Sprint, “I wanted to let you know that I am finally not
spending hours on weekends on Epic!”
Discussion
The COVID-19 pandemic has created unimaginable challenges for health care teams and
society at large, but it has also led to creativity and innovation.[15]
[16] The UCHealth Sprint Team rapidly transitioned an onsite Sprint EHR optimization
and training program to a virtual program without significantly altering cadence,
team composition, number of participants, or curricular content. Despite changes to
the program, the Sprint Team maintained high levels of Sprint and EHR satisfaction
and a trend toward decreased staff and provider burnout.
After virtual Sprint events, EHR satisfaction increased considerably for providers
(increase in NPS of 39) and staff (increase in NPS of 29). Both groups reported improved
EHR proficiency, easier care team communication, and increased sufficiency of time
for documentation. To our knowledge, this is the first study of a fully virtual EHR
training and optimization program. Similar to Kaiser's Pathway to Proficiency onsite
program,[4]
[5] Sprint PI leads conduct peer-to-peer training during kick off, wrap up, and small
group sessions using the Microsoft Teams platform. Unique to our program, Sprint trainers
deliver core, role-based EHR education to all members of the target clinic. We view
user data to help personalize content, like Stanford's Home4Dinner program,[17] but in Sprints, we also use PI and CI knowledge of EHR tools and clinical workflows
to construct core competencies and teach best practice.
Most EHR efficiency programs encourage EHR personalization, but ours is unique in
that Sprint EHR analysts optimize EHR tools during the Sprint event.[12] Analysts work directly with clinical experts and informaticist leaders to clean-up,
repair, and create new build. Sprint events create a sense of urgency due to relatively
short timelines, and as a result, we have a captive audience to test, iterate and
approve specialty-specific EHR tools and workflows leading to true user-centered design.
EHR training and tool design rely on HCW engagement, the antithesis of burnout.[9] It is imperative that we continue to study the impact of interventions on HCW burnout
given its prevalence. Like our onsite program,[8] less burnout was reported post-Sprint compared with pre-Sprint for providers and
staff despite the pandemic. Control over workload, job stress, and workplace atmosphere
have been shown to correlate with staff and provider burnout.[18] While there is a trend toward increased control over workload post-Sprint, we are
unable to demonstrate improvement in job stress and workplace atmosphere. We suspect
that the pandemic played a role in these metrics in several ways including changes
in physical workspaces to accommodate infection control measures, fear of COVID-19
infection, increase in telemedicine visits, altered work–life balance (schools closed
and work from home), and limited staffing due to inpatient and vaccine staffing/redeployment.
Some Sprint participants report feeling overwhelmed by the amount of new EHR knowledge
which could contribute negatively to job stress. For onsite Sprints, stress can be
induced by the physical presence of a large team, while during virtual events, a higher
volume of e-mail communication might lead to perceived increased workload. To fully
understand how virtual Sprint events may impact overall job stress and workplace atmosphere,
we will continue to study Sprints as clinics resume standard operations.
Virtual Transition Challenges and Lessons Learned
Many key Sprint program components, such as daily huddles, request prioritization,
and wrap-up sessions, translated easily to a virtual format, while components such
as workflow observation and group training have proven more difficult to replicate.
We began to schedule multidisciplinary workflow meetings at our 60-day pre-Sprint
meeting. Conducted early in the Sprint, workflow meetings can be subspecialty-based
(i.e., pre–liver transplant team), role based (i.e., physical therapists), or task
based (i.e., scanning documents to orders and anticoagulation tracking). Interestingly,
we observed increased participation and faster time to consensus and decision-making
with virtual workflow meetings than we did with direct observation during onsite Sprints.
We attribute this success to pre-Sprint scheduling, end user screen sharing during
discussion, and the ability for staff and providers to participate remotely. Workflow
sessions also facilitated introductions and personal connections and helped the Sprint
Team identify clinical workflow experts to tap for knowledge during the Sprint.
Our new online-only, topic-specific (medication reconciliation, in basket, problem-based
charting) group training sessions proved to be well received. Sprint informaticists,
trainers, and attendees collaboratively discussed how tips applied to their specialty.
Often, we observed that EHR-proficient end users would support the use of a particular
EHR tool or workflow, augmenting the Sprint team message and contributing to the success
of the session.
Challenges with the virtual format included communication and driving engagement.
It can be difficult to communicate and garner trust in the value of Sprints. On Sprint
Days 1 and 2, there is often a start-up delay, whereafter engagement quickly increases.
The Sprint program managers anticipated that this issue would grow with virtual Sprints
and made several changes to their communication methods. They creatively used subject
lines (“OPTIONAL” vs. “MANDATORY” sessions), visual cues (picture of calendar day
with live link in the body of the e-mail), succinct communication, and specific fonts
and colors to highlight important messages for end users. They also restricted communication
to one e-mail per day from the Sprint team which included the daily schedule and other
high yield tips or messages from the previous day.
During the pandemic, we took a less stringent approach to engagement. With on-site
Sprints, participants were strongly encouraged to attend events in person, and there
were not options for remote participation. The Sprint team is a big investment for
our organization, and the clinic was expected to directly engage to maximize the resource
by showing up in person. During the pandemic, our primary goal became “meet the clinic
where they are.” Clinics needed to maintain visit volumes and support their staff
and providers. While also considering the health and wellbeing of our Sprint team,
we practiced empathy and flexibility. We encouraged clinic participants to join events
remotely, we repeated or shifted group session times, and we adjusted training as
often as needed. In rare cases, we found less participation as clinics responded to
the pandemic, but more commonly, we had increased participation due to the ability
for remote participation and an increased institutional focus on wellness. At UCHealth,
the EHR Sprint Team has been recognized as one of our strongest organizational wellness
initiatives to support providers and staff.
Notably, there were additional benefits to virtual Sprint. There is a high cost to
run an onsite Sprint program that supports clinics across a large geographic region.
The virtual programming decreased Sprint Team travel expenses and overtime. Virtual
Sprints also increased access to training and participation for providers and staff
who were working primarily through telehealth at home or who might want to join Sprint
activities during administrative or research time. Interestingly, daily huddle conversations
are now more robust and easier to follow. Since all participants are remote, those
who call-in no longer struggle to hear or understand in-person conference room dialogue.
Internal Sprint Team communication and accountability is also maximized with the virtual
format.
Limitations
Our data were collected during a global pandemic, and this alone could affect survey
responses and response rate. To avoid precipitating burnout with recurrent e-mails,
we did not send surveys more than two times. The statistical analyses of these survey
responses should be interpreted in the context of some limitations. Nonrespondents
may have different experiences and views than respondents. Pre- and postsurvey responses
were not linked within a particular individual, and adjusted analyses were not performed.
Since this is an exploratory study, we avoid making claims of statistical significance,
and thus no corrections for multiple comparisons were performed.
Additionally, we do not resurvey end users over time. We lack a research budget and
during the pandemic, we did not want to add to HCW burnout. In our recent time-saving
studies,[19] we hypothesize that Sprints likely need to be repeated annually due to EHR upgrades,
as well as changes in clinical practice. KLAS Arch Collaborative data also support
at least 3 hours of provider training annually. More study will be required to determine
if a second or third Sprint event will require the same number of resources per number
of participants to maintain EHR satisfaction and reduced levels of HCW burnout over
time.
As a comprehensive training and optimization program, we are unable to separately
measure the impact of training versus that of EHR optimization. Longhurst et al[6] demonstrated a clear benefit of EHR personalization on end-user EHR satisfaction,
but the impact of user-centered design and software optimization is unknown. Despite
this limitation in our study, we observe benefits to IT teams and clinical users from
EHR clean up, reconfiguration, and new tool build, while we have the attention of
clinical users during Sprint events.
Conclusion
While physician burnout was unacceptably high prior to March 2020, the COVID-19 pandemic
has added uncertainty and workplace chaos to health care.[20] More than ever, it is time to rise to the challenges we face with empathy, innovation,
and teamwork. Clinical informatics teams are in a unique and advantageous position
to implement and showcase high-performing, multidisciplinary teams who can meet clinics
where they are and move them forward with education, training, and EHR optimization.
The Sprint program successfully transformed an onsite EHR optimization and training
program to a virtual program that continues to improve EHR satisfaction and teamwork,
while reducing burnout in ambulatory clinics. We would encourage other organizations
to view their EHR optimization and training programs as key wellness initiatives that
support and foster communication and teamwork in uncertain times.
Clinical Relevance Statement
Clinical Relevance Statement
Ongoing electronic health record training and optimization is a struggle for many
organizations. This paper offers a fully virtual solution, with data supporting its
effectiveness. The solution described herein can be used at organizations of any size
or composition, as long as the size of the Sprint team is adjusted accordingly.
Multiple Choice Questions
Multiple Choice Questions
-
What does the UCH Sprint team do that is unique from other EHR efficiency programs?
-
Trains staff and providers
-
Focuses on one clinic for 4 months
-
Personalized training
-
Specialists-training-specialists
Correct Answer: The correct answer is option a. The UCH Sprint Team is unique in that it consistently
offers training to all clinic staff, not just providers. On average, we spend 2 weeks
with each clinic. We do look at signal data and personalize the training to some extent,
but there is also standard training. Specialists-training-specialists is not part
of the Sprint format.
-
What was added to the Sprint program with the transition from onsite to virtual, to
accommodate for the lack of in person contact?
Correct Answer: The correct answer is option b. We elected not to offer any on-site support, so everything
was 100% virtual. We did add workflow and group sessions to replace missed training
opportunities caused by the virtual format. Kick offs were always part of Sprints
and were continued virtually. We did not change the number of training sessions offered
in our conversion to virtual.
