Adult Learning Principles and Peer Delivery Improve Satisfaction of Electronic Medical Record Onboarding Education

Abstract

Objectives Given the importance of onboarding education in ensuring the safety and efficiency of medical users in the electronic medical record (EMR), we re-designed our EMR curriculum to incorporate adult learning principles, informed and delivered by peers. We aimed to evaluate the impact of these changes based on their satisfaction with the training.

Methods A single site pre- and post-observational study measured satisfaction scores (four questions) from junior doctors attending EMR onboarding education in 2018 (pre-implementation) compared with 2019 (post-implementation). An additional four questions were asked in the post-implementation survey. All questions employed a Likert scale (1–5) with an opportunity for free-text. Raw data were used to calculate averages, standard deviations and the student t-test was used to compare the two cohorts where applicable.

Results There were a total of 98 respondents in 2018 (pre-implementation) and 119 in 2019 (post-implementation). Satisfaction increased from 3.8/5 to 4.5/5 (p < 0.0001) following implementation of a peer-delivered curriculum in line with adult learning practices. The highest-rated factors were being taught by other doctors (4.9/5) and doctors having the appropriate knowledge to deliver training (4.9/5). Ninety-two percent of junior doctors were motivated to engage in further EMR education and 90% felt classroom support was adequate.

Conclusion EMR onboarding education for medical users is a critical ingredient to organizational safety and efficiency. An improvement in satisfaction ratings by junior doctors was demonstrated after significant re-design of the curriculum was informed and delivered by peers, in line with adult learning principles.

Keywords

electronic medical record - adult learning principles - user satisfaction

Background and Significance

High quality electronic medical record (EMR) education for junior doctors (residents, registrars, and fellows) is critical to ensure patient safety and reduce stress and burnout for doctors.[1] Junior doctors interact heavily with the EMR and are expected to perform at full capacity from the day of commencement. A sound understanding of the workflows and tools within the EMR contributes significantly to a junior doctor's ability to operate efficiently and deliver safe care.

Our institution is a large pediatric training hospital in Melbourne, Australia, with approximately 89,500 emergency department presentations, 345,000 outpatient appointments, 52,000 inpatient admissions, and 16,500 surgical operations per year.[2] In April 2016, the organization transitioned from a predominantly paper-based record to an enterprise-wide EMR.

As one of the major pediatric teaching hospitals in the state, each year, large numbers of junior doctors (∼300 per year) join our organization and require onboarding EMR education. Initially, this ongoing demand for onboarding education of junior doctors was met by a traditional EMR training model. Education was provided by EMR technicians with excellent knowledge of the EMR but no experience with medical workflows. The curriculum was designed and developed in accordance with the EMR vendor. The pre-implementation curriculum contained expansive content aimed at equipping users with knowledge regarding most aspects of the EMR on day one but lacked any emphasis on clinical application. The delivery style was highly procedural with many sections of the workbook providing “follow the trainer” activities, where the learner passively imitated the clicks of the trainer.

Adult education literature suggests adults are more self-directed in their learning and have a greater need to understand why they should learn something.[3] [4] Adult learners learn best when learning is seen as relevant to their work, builds on existing knowledge, and requires active engagement. Learning activities should be self-paced using problem-solving exercises covering content that can be applied to real life situations. It is therefore important that the educator knows and understands the learner's needs, and designs learning activities relevant to those needs.[3]

Using these principles, our existing curriculum was overhauled with the aims to reduce the cognitive overload provided by the existing curriculum, and to improve the learning outcomes.

Objective

We aimed to evaluate the impact of these changes to the EMR on-boarding curriculum for junior doctors, based on their satisfaction with the training.

Methods

Curriculum re-design: Given the need to make education relevant and case based, the curriculum had to be informed by junior doctors who had an intimate knowledge of the EMR workflows required for medical staff. The EMR training team employed three junior doctors to review the existing curriculum in each of the major settings: outpatient, inpatient, and surgical, in line with adult learning principles ([Table 1]). Each doctor had extensive knowledge of the workflows in their designated area, having worked with the EMR in our organization for at least 12 months. A senior doctor with a background in medical education provided oversight and consistency between the modules. Existing content was reviewed and classified as “required starting,” “good-to-know” and “expert” EMR knowledge. Based on the content deemed “required starting knowledge,” cases representing real-life scenarios were designed so that the doctor must use the requisite skills to complete the activity. “Pearls” were offered at the end of each activity, based on the “good-to-know” content. Expert level EMR knowledge was excluded from initial training. Training sessions were designed to run over 3 hours each for inpatient and outpatient sessions (with some junior doctors attending both sessions) and a 7-hour course for surgical trainees covering all settings (inpatient, outpatient, and operating theaters).

Table 1 Summary of major changes made to curriculum

From	To
Didactic, passive learning	Exploratory, active learning
Procedural/IT workflows	Case based
Content heavy	Survival-based knowledge
Taught by analysts with strong EMR knowledge	Taught by doctors with strong workflow knowledge
Generic training—based on setting	Relevant to work setting (i.e., inpatient, outpatient, and/or surgical)
Trainers set the pace	Learners set the pace

Abbreviation: EMR, electronic medical record.

In addition, training cohorts were tailored particularly to cater for junior surgical doctors, who had previously undertaken generic medical training with a half hour add-on theater component. A junior surgical doctor was engaged to re-develop the curriculum to provide surgical cases, and to prioritize activities such as theater workflows, theater case bookings, and procedural consent. The surgical doctor also delivered the classroom education to their fellow surgical doctors.

To facilitate a case-based exploration model, interactive e-learnings were created to provide new users with an orientation to the system, and to provide a basic knowledge platform from which they were able to engage in case-based learning activities in the classroom ([Fig. 1]).

Fig. 1 Screenshot from e-learning demonstrating an interactive activity.

Due to the change in format from a “follow the trainer” style to a more active learning style, classroom support was increased with a ratio of one educator for every six learners, previously a 1:10 ratio. The active learning style incorporated the use of exercise booklets, which enabled learners to try out activities, simulating the clinical environment ([Fig. 2]).

Fig. 2 Screenshot from exercise booklet containing an activity that simulates the clinical environment.

An assessment opportunity was provided, with a filmed scenario (i.e., a prerecorded video of a ward round discharge for inpatient training) allowing the learning doctors the opportunity to experience the required workflows in a stress-free environment, and to troubleshoot areas of difficulty before leaving the classroom.

Finally, we continued to offer a 2-week period of floor support to new users. This has always been part of our onboarding process but was restructured to enable both support of new users and to provide “expert” level tips to existing users. Personalization workshops of an hour duration were also offered during the first few weeks of starting work. Calls to the EMR help desk and floor support team were monitored to ensure that the reduction of class content was not leading to critical knowledge gaps.

Data collection: Because it is difficult to measure efficiency and effectiveness, we used learner satisfaction and confidence as surrogate measures of learning success. Retrospective review of junior doctor EMR education satisfaction scores from surveys collected in 2018 (pre-implementation) were compared with satisfaction scores from surveys collected in 2019. There were four questions asked in both 2018 and 2019 and a further four questions asked in 2019 ([Fig. 3]). Satisfaction was rated on a Likert scale of 1 to 5. All learners who underwent training and responded to the surveys did not receive prior training. Raw data was used to calculate mean values and standard deviations, and the student t-test was used to compare the two cohorts.

Fig. 3 Image of survey questions from 2019.

Volume of calls to the EMR Help Desk and average talk-time were compared across both years using the 4-week period following training for each cohort.

Results

A total of 98 survey responses from 2018 (70% response rate), and 119 survey responses from 2019 (85% response rate), were collected following two major intakes of junior medical staff in February and August ([Table 2]). Satisfaction with EMR training ratings increased on a Likert scale from 3.8/5 in 2018 to 4.5/5 in 2019 using the same questions between the 2018 and 2019 trainee cohorts (p < 0.0001).

Table 2 Summary of main findings

	2018	2019	Difference
Number of respondents	98 (70% response rate)	119 (85% response rate)
Trainer: Learner ratio	1:10	1:6
Training duration	4 h
Satisfaction rating (out of 5)	3.8	4.5	p < 0.0001
E-learnings gave the right amount of preparation	3.3	4.2	p < 0.0001
Course was appropriately paced	3.3	4.4	p < 0.0001
I feel equipped/confident to use the EMR on day 1	3.4	3.7 (78 respondents)	p = 0.13
Trainers had the appropriate knowledge		4.8
Able to get support in the classroom		4.8
Motivated to learn further in the EMR.		4.6
It was useful to have a doctor running the training		4.8
Calls to help desk	2,885 calls (total talk time 12,030.50 min)	2,705 calls (total talk time 11,599.15 min)	180 less calls (total talk minutes reduced by 431.4)

Abbreviation: EMR, electronic medical record.

The highest-rated factors contributing to satisfaction after the curriculum re-design were “I found it useful to have a doctor provide the training” (average rating of 4.87/5), and “Doctors have the appropriate knowledge to provide EMR training” (average rating of 4.88/5). Free-text comments included “It was great that most of the time was spent practicing skills—high yield overall,” “easy to get help from trainers who were fantastic,” “it was great to have doctors teaching…they had the knowledge of what would be clinically relevant.”

Classroom support at a ratio of 1:6 educators to learners was rated as 4.85/5 with 92% of respondents rating this 4 or above. Motivation to engage in further EMR education was also rated 4.6/5 with 98% of respondents rating this 4 or above. Pacing of the course was rated 4.4/5 with free-text comments varying from “too long” to “longer session please.”

The lowest-rated factor was “I feel confident to use the system on day 1” (78 respondents) with a rating of 3.7/5. This question was modified for the August cohort in an attempt to better ascertain the primary goal of training to “I feel equipped to use the system on day 1” (41 respondents) with an overall rating of 4.1/5. Because of the wording change, only the January cohort was compared with 2018 ([Table 2]). Overall, 25.5% (n = 28) of junior doctors were neutral (rating of ⅗), while 6.4% (n = 7) and 0.9% (n = 1) rated confidence as ⅖ and ⅕, respectively.

Calls to Help Desk did not increase following the modified training. In the 4 weeks following each training cohort (January/August), there were a total of 2,885 calls to Help Desk in 2018 and 2,705 calls in 2019. The average talk time was 4.17 minutes in 2018 (total talk time 12,030 minutes) and 4.29 minutes (total talk time 11,599 minutes) in 2019.

Discussion

The introduction of the EMR has been one of the largest transformations in the way we deliver medicine in the last decade. For users to be effective and efficient in the EMR, the importance of quality onboarding training cannot be understated. We have adopted adult learning practices and delivered a curriculum designed by doctors, tailored to doctors' needs. In an observational study design, we have demonstrated that satisfaction ratings have improved in response to this approach.

As training content was reduced to allow experiential learning, which is more time intensive; we monitored calls to Help Desk as a surrogate measure of learning gaps. Calls to Help Desk reduced following the modified curriculum suggesting that while content was reduced, the appropriate content had been included and was taught effectively.

EMR education is a rapidly evolving area with limited evidence to date on effective education strategies for the computer learning environment.[5] [6] [7] This is the first publication, to our knowledge, to measure pre- and post-intervention learner satisfaction following the institution of case-based learning informed and delivered by doctors. The findings are consistent with learnings from the Arch Collaborative who has compared education strategies to overall organizational EMR satisfaction. The Arch Collaborative suggests that organizations with higher overall satisfaction provide onboarding education that puts meaning into training, utilizes case-based training with patient scenarios, and provides delivery by clinicians.[7] In addition, our training model is in line with many successful organizations that achieve high EMR satisfaction ratings—a model built on layers of knowledge through initial online training, followed by in-class training, and subsequently supported by floor support and personalization laboratories. The current evidence is for a minimum of 5 to 6 hours broken up into several sessions, which we have mirrored in our adopted training model.[7] We believe approaching the EMR education process beyond training doctors to input data allows us to highlight the benefits of using the EMR well for both doctors and their patients. Our educators emphasize the rationale of why things are required to be done the way that they are, to achieve good patient care and clear communication with other clinicians. Employing doctors that are passionate educators and strong advocates for the EMR as a clinical tool has been critical and is reflected by the free-text comments to our survey. We acknowledge that while this approach was cost-neutral and had no negative impact on staffing or hospital activity for our organization, there are many complex factors that may make this a less feasible approach in other institutions.

While our findings are positive and encouraging, there is room for further growth in this area. A successful EMR user should achieve strong mastery of the EMR, feel a sense of shared ownership, and have the EMR meet their unique needs.[8] There are several aspects within the onboarding process that we can focus on to set up ongoing successful EMR use for clinicians. These include fostering a sense of excitement about the benefits of an EMR, demonstrating what is possible and providing details on how to obtain further training. A key finding that points toward an area for improvement, is the fact that the lowest-rated factor in our survey was “I feel confident to use the system on day 1.” In our experience, online learning prior to in-class training tends to have a poor adherence. Further efforts to encourage learners to engage in the pre-learning activities will enable a higher platform of knowledge on which to build upon during classroom activities. This may have an impact on the level of confidence achieved.

There are several ways that we can continue to improve our specialized and tailored training for doctors. Our institution has adopted other projects not described in this paper, including optimization strategies and specialty-specific builds. These enable users of EMR to see that they are able to shape the changes of EMR, to feel a sense of ownership, and that the EMR is designed for their unique needs.

The conclusions of our study are limited by the small sample size (1 year of data only) and the number of changes being introduced simultaneously to our training program. It should be noted, however, that the case mix of doctors attending training had no major differences between 2018 and 2019. By introducing multiple improvements at the same time, it is difficult to know which factor had the largest impact. For example, it is possible that the improvement in satisfaction ratings was driven predominantly by the surgeons who for the first time had a tailored education. However, we would argue that the combination of curriculum re-design to an interactive case-based learning system, tailoring the education to each craft group, and providing peer educators have been synergistic, and would be difficult to achieve in isolation. In addition, our study has predominantly reported user satisfaction as a surrogate measure for learning success. The in-class assessment ensured basic level efficiency and there was no escalation in calls to the EMR Help Desk; however, a more formal proficiency test would be useful in informing further curriculum design.

Conclusion

Computer-based skill acquisition presents unique challenges to the clinical learner. However, this is critically important for doctors working within EMRs. We provide evidence of one effective strategy to improve the learning experience for new medical users. Further research is required to better understand the cost-effectiveness of this model of training, how to build upon these skills beyond onboarding, and how to motivate doctors to prioritize EMR skills equal to clinical acumen.

Clinical Relevance Statement

Effective EMR onboarding education is critical in the safety of patients and the prevention of burnout of junior doctors. Junior doctors perceive an interactive case-based education session delivered by peers as a superior model to didactic generic classroom teaching.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

Ethical approval not required as this was a quality improvement project.

• References

• 1 Robinson KE, Kersey JA. Novel electronic health record (EHR) education intervention in large healthcare organization improves quality, efficiency, time, and impact on burnout. Medicine (Baltimore) 2018; 97 (38) e12319
• 2 Annual Financial Report 2018–19. Royal Children’s Hospital, Melbourne, Australia. Available at: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwjqu8-KvczrAhUTfSsKHc0jB10QFjAAegQIAhAB&url=https%3A%2F%2Fwww.rch.org.au%2FuploadedFiles%2FMain%2FContent%2Frch%2FRCH-AFR18-19-web.pdf&usg=AOvVaw0JeC58WBDCtumK6aJ_eSeS . Accessed September 9, 2020
• 3 Collins J. Education techniques for lifelong learning: principles of adult learning. Radiographics 2004; 24 (05) 1483-1489
• 4 Arseneau R, Rodenburg D. The developmental perspective. In: Pratt DDM. , ed. Five Perspectives on Teaching in Adults and Higher Education. Melbourne, FL: Krieger Publishing; 1998
• 5 Pantaleoni JL, Stevens LA, Mailes ES, Goad BA, Longhurst CA. Successful physician training program for large scale EMR implementation. Appl Clin Inform 2015; 6 (01) 80-95
• 6 Stevens LA, Pantaleoni JL, Longhurst CA. The value of clinical teachers for EMR implementations and conversions. Appl Clin Inform 2015; 6 (01) 75-79
• 7 Longhurst CA, Davis T, Maneker A. Arch Collaborative. , et al; Local investment in training drives electronic health record user satisfaction. Appl Clin Inform 2019; 10 (02) 331-335
• 8 KLAS. Arch Collaborative Guidebook. Salt Lake City, USA: The Arch Collaborative; 2019

Publikationsverlauf

Eingereicht: 10. November 2019

Angenommen: 30. Juli 2020

Artikel online veröffentlicht:
22. September 2020