Improved Medical Student Engagement with EHR Documentation following the 2018 Centers for Medicare and Medicaid Billing Changes

Abstract

Background Medical student note writing is an important part of the training process but has suffered in the electronic health record (EHR) era as a result of student notes being excluded from the billable encounter. The 2018 CMS billing changes allow for medical student notes to be used for billable services provided that physical presence requirements are met, and attending physicians satisfy performance requirements and verify documentation. This has the potential to improve medical student engagement and decrease physician documentation burden.

Methods Our institution implemented medical student notes as part of the billable encounter in August 2018 with support of our compliance department. Note characteristics including number, type, length, and time in note were analyzed before and after implementation. Rotating medical students were surveyed regarding their experience following implementation.

Results There was a statistically significant increase in the number of student-authored notes following implementation. Attending physicians' interactions with student notes greatly increased following the change (4% of student notes reviewed vs. 84% of student notes). Surveyed students reported that having their notes as part of the billable record made their notes more meaningful and enhanced their learning. The majority of surveyed students also agreed that they received more feedback following the change.

Conclusion Medical students are interested in writing notes for education and feedback. Inclusion of their notes as part of the billable record can facilitate their learning and increase their participation in the note writing process.

Keywords

electronic health records - education, medical, undergraduate - documentation - Centers for Medicare and Medicaid

Background and Significance

Clinical note writing has been an essential aspect of undergraduate medical education for decades.[1] The Association of American Medical Colleges states that medical students must learn to “communicate effectively, both orally and in writing, with patients, patients' families, colleagues, and others with whom physicians must exchange information.”[2] Electronic health records (EHRs) have become the default for documenting patient care in the vast majority of academic settings in the United States, in large part thanks to the Health Information Technology for Economic and Clinical Health Act of 2009.[3] While the EHR brings exciting opportunities to improve healthcare delivery and care coordination, it has presented challenges to medical student experiential learning, especially with respect to participation in the note writing process.[4]

In 2012, the Alliance for Clinical Education presented guidelines that reiterated the importance of medical students learning to document in the EHR era.[5] A survey of medical school deans found that while 90% believed student notes belonged in the medical record, only about half of hospitals allowed some form of student note in the EHR.[6] A similar survey of Emergency Medicine clerkship directors indicated that 37% of the clerkships did not allow medical student document in the EHR, and a major reason cited for this decision was the Centers for Medicare and Medicaid (CMS) regulations that kept healthcare providers from using medical student documentation when billing for professional services.[7] Even if students were allowed to write notes, since the notes were not allowed to be used for billing, the experience of writing the note had diminished authenticity and salience. Furthermore, since the attending physician was required to write or review a second official note, review of the medical student note created additional work for the attending instead of being integrated within and incentivized by the official medical documentation process.

Given these concerns and others related to increased documentation burden on attending physicians, CMS changed their regulations in 2018 as they relate to medical student documentation used for clinical care and billing. The changes allow for teaching physicians to use all student documentation for billable services provided that (1) physical presence requirements are met, and (2) the attending physician satisfies the performance requirement and verifies the documentation.[8]

Given the importance of the note in the medical students' educational experience, our institution set out to implement and evaluate these new CMS regulations in a manner that allows the student to be an active participant in the clinical documentation process. The hope was that this would allow the student to write meaningful notes and get better feedback from their supervisors. In this study, we evaluated the effect of the implementation of these changes on quantity of student note writing, resident/attending review of the notes, and student perspectives of note writing.

Methods

We implemented medical student documentation as part of the legal medical record (LMR) in all acute care and ambulatory settings at Stanford Children's Health in August 2018, after the change in CMS policy. Prior to August 2018, medical student documentation in the EHR was made available only as an educational tool. Students could only create a “student” note type that was not considered part of the LMR. Residents and attending physicians could view their notes and provide verbal feedback to the medical student. However, the supervisors were instructed not to attest or addend the note, as these notes were not part of the official documentation. Subsequent to implementing the new CMS rules in August 2018, medical students were granted security to create standard note types (i.e., history & physical (H&P), progress note, consult note, and discharge summary), and these notes could be edited and cosigned by the resident and the attending. These notes became part of LMR when cosigned.

To assist the attending physician in using the student notes for billing purposes, we created attestation macros with the guidance from the hospital compliance department. There were two workflow scenarios: (1) The student would create a standard note and then send the note to the attending for co-signature. The attending would addend the note, add an attestation, and sign. The attending must be present with the student during the visit/evaluation or independently confirm their findings. (2) Similar to the first workflow, but with a resident involved in the intermediate step, the resident must be present with the student during the visit/evaluation or independently confirm their findings. The attending must be present for key portions. Each of these two scenarios had a different attestation macros. If a student initiated the note and a student attestation was not used, the macro prompted the attending to choose the appropriate attestation statement. Our compliance department agreed that student notes can support most evaluation and management billing charges, but they did not allow for critical care and intensive care charges to use student documentation. Because of this, the “student” note type remained for these situations in which the billing note could not include student documentation. After implementation, we queried our hospital billing department regarding any claims issues or denials related to student notes.

We analyzed the note-writing data pre- and post-implementation from EHR use-logs for students who wrote notes during the 3 months prior to implementation and 3 months following implementation. We looked at the number of notes by type, note length, and also evaluated time spent by residents and attending physicians in viewing and editing the student notes by querying data from EHR time logs. Median time in notes and note length as well as interquartile ranges (IQR) were calculated.

In addition to the above analyses, we also surveyed the medical students who rotated in the core pediatric clerkship between August 2018 and May 2019 about their experience. This survey consisted of four 7-point Likert scale questions and one Net Promoter Score (a scoring tool used to determine a user's likelihood to recommend – percent likely to recommend less percent not likely to recommend). Questions in the survey aimed to assess the effective writing notes that are part of the LMR has had on the students' educational experience, feeling part of the team, and meaningfulness of their work.

Results

In 3 months prior to implementation (May–July 2018), 68 unique medical students created 637 “student” notes (they did not have security to write any other note types). In 3 months following implementation (September–November 2018), 95 students created 1,912 standard notes (a threefold increase; [Table 1]). For students rotating on the core pediatrics clerkship, 27 students created 333 notes (mean = 12.3 notes/student) pre-implementation, while 29 students created 617 notes (mean = 21.3 notes/student) post-implementation (p = 0.012, two-tailed t-test). The vast majority of the notes were progress notes (74%), followed by H&Ps (10%), discharge summaries (9%), and consults and follow-ups (5%).

Table 1 Student Note Volume Pre- and Post-Implementation

	Pre-implementation	Post-implementation
Number of students (all)	68	95
Number of notes (all)	637	1912
Number of students (core pediatrics)	27	29
Number of notes (core pediatrics)	333	617
Number of notes per student (core pediatrics)	12.3[a]	21.3[a]

^a t-test, two-tailed p = 0.012.

Following implementation, students spent a median time of 30 minutes creating/editing a note. When a resident or fellow was involved in editing the student note (50% of the time), the median time spent by resident/fellow was 4.9 minutes (IQR: 11.7 minutes; [Table 2]). When the attending physicians contributed to the student's note (84% of the time), the median editing time for attending physicians was 2.3 minutes (IQR: 5.6 minutes). This was a marked improvement in engagement with the students' notes compared with pre-implementation, when attending physicians edited less than 4% of “student” notes and spent a median time of 0.6 minute editing these notes (IQR: 0.9 minutes).

Table 2 Note Engagement Pre- and Post-Implementation

	Pre-implementation	Post-implementation
Percent of notes edited by resident/fellow	8% (51/637)[a]	50% (956/1912)[a]
Percent of notes edited by attending physician	3.8% (24/637)[a]	84% (1,600/1,909)[a]
Duration of time editing by resident/fellow Interquartile range (IQR)	9.8 min (median) 17.0 min	4.9 min (median) 11.7 min
Duration of time editing by attending physician Interquartile range (IQR)	0.6 min (median) 0.9 min	2.3 min (median) 5.6 min

^a Fisher's exact test, p < 0.0001.

[Fig. 1] displays the note writing and editing time by students, resident/fellows, and attending physicians for the common note types following the implementation. For students, most of the time was spent writing consult notes (median = 81.6 minutes, IQR: 59.5 minutes), followed by H&Ps (60.1 minutes, IQR: 85.7 minutes), discharge summaries (32.6 minutes, IQR: 41.4 minutes), and progress notes (25.0 minutes, IQR: 37.6 minutes). For the attending physicians, most of the time was spent editing student consult notes (6.1 minutes, IQR: 14.6 minute), followed by discharge summaries (2.3 minutes, IQR: 4.7 minutes) and progress notes (2.3 minutes, IQR: 5.5 minutes).

Fig. 1 Duration of writing and editing various note types by role. Box and whisker plot (whiskers represent 1.5 times interquartile range).

We also looked at the note character length, comparing those written by medical students with those written by a resident, fellow, or attending (without student contribution; [Fig. 2]). Notes written by medical student were generally longer (except for H&Ps). For example, for progress notes, the median character length was 5,589 (IQR: 4,591) for medical student notes and 3,927 (IQR: 3,946) for notes without medical student contribution (p < 0.0001, t-test).

Fig. 2 Note character length of those written by medical student and those written by resident/fellow/attending only. Box and whisker plot (whiskers represent 1.5 times interquartile range).

We have not had any issues from the hospital billing department regarding student documentation on Medicare and Medicaid patients.

In total, 20 of the 52 students (38.5% response) who rotated through the core pediatrics clerkship from August 2018 to May 2019 responded to the survey. Survey results indicate that the majority of the students responded positively to medical student documentation being part of the LMR. However, 50% of the respondents said that they would recommend the tool to colleagues (rating of 9 or 10 on a 0 to 10 scale while 20% would not recommend it [rating of 0 to 6], giving it a net promoter score of 30 (NPS can range from −100 to 100).

[Fig. 3] shows the responses to four questions regarding the students' experience with documentation (on a 7-point Likert scale). The vast majority of students responded positively to statements “writing notes in enhances my learning” (95%) and “my work is more meaningful” (90%). The responses to the statement “my notes had a meaningful impact on patient care,” was slightly less positive, though students still largely agreed (85% agreed to some extent). The results were more mixed on the statement: “I receive more feedback from attendings about my notes,” with 65% agreeing to some extent.

Fig. 3 Medical student survey responses following implementation. The numbers in the circles represent the average score on a 7-point Likert scale. Horizontal bars represent the percentage of responses in each scale score. The column on right displays the sum of percentage of “somewhat agree,” “agree,” and “strongly agree” responses.

Discussion

To our knowledge, this is the first report on the effects of the 2018 medical student note CMS billing changes on the medical student experience. This study provides evidence that incorporation of the medical student notes into the LMR resulted in a significant increase in the number of medical student notes filed in the EHR. Our results confirm that students are interested in writing notes for education and feedback. The threefold increase in the number of medical student notes was not due solely to more students writing notes. When we compare the pre- and post-implementation number of notes written by medical students rotating in the core pediatrics clerkship (where the number of student enrolled was stable pre- and post-implementation), there was a near doubling of the number of notes written (333 vs. 617) and a significant increase in the number of notes per student (12.3 vs. 21.3, p = 0.012; [Table 1]). This indicates that students are more engaged in writing notes when they are incorporated in the LMR.

The majority of the surveyed students agreed that the ability to write standard notes in the EHR enhanced their learning, that their notes had a meaningful impact on patient care, and that the process made their work more meaningful. We also noted that majority of students agreed that they received more feedback on their notes after the intervention ([Fig. 3]).

A recent editorial expressed concerns over the modified regulations, worrying that medical student notes would become overly focused on billing, that writing billable notes would lead teaching physicians to spend less time with students, and that shifting documentation to medical students might pose a risk to their education and well-being.[9] In contradiction to these concerns, our results indicate that students felt that these CMS changes had a positive impact on their learning, and our EHR data suggests that physicians are engaging with the student notes more, not less ([Table 2]). These changes have the potential to not only increase the students' participation in the note writing and patient care processes, but also to decrease the note writing burden for the resident and attending providers.

Additionally, in Australia, where billing and documentation requirements differ dramatically from the United States, surveyed medical students have readily engaged with the EHR and found it enhanced their learning and the clinical feedback they received.[10] [11] These finding are in line with key tenets of experiential learning in adult learning theory.[12] Newer pedagogic models have also suggested enhanced value of authentic activity and assessment in fostering meaningful learning,[13] which may also support enhanced experiential learning when the learner engages in activities that have an audience outside the constructed educational environment (other health care providers and CMS in this case). As health information systems become more and more integrated into the practice of medicine, it is important to incorporate education and coaching on how to appropriately use the EHR into the undergraduate medical experience.[14]

Notes initiated by students were substantially longer than those initiated by residents or attendings ([Fig. 1]). This is likely multifactorial, but literature suggests that students may be less discerning in their note content and may have more redundancy and copy-pasted information.[15] Students also spent more time editing notes; most likely secondary to a lower proficiency compared with their resident and attending colleagues. This highlights the importance of adequate EHR training for medical students, in addition to education on note format and content.[16]

It is important to note that the implementation of student notes requires input from local compliance and billing departments to ensure proper processes. Physicians must participate in key portions of the documentation process when medical students are involved, confirming physical exam findings and performing their own medical decision making. Care must be taken to ensure physicians are completing the requirements and attesting accordingly. Anecdotally, we have seen that both trainee and attending physicians seem to be adapting to the change well without complaints. We have had no known issues with billing as it relates to notes with student contribution.

There are several limitations of this study. First, the quantitative results derive from a quasi-experimental pre-post study design with a relatively small cohort size, and a 38.5% response rate which may not be a fully representative sample. Although there was a strong association between the intervention and the desired effects, causation cannot be established. We were also not able to survey students prior to the intervention, so the student perspectives are based on a single survey post-intervention. Second, while surveys may show an increase in satisfaction, they may not lead to improved performance.[17] [18] Third, engagement by attending physicians in reviewing medical student notes was assessed by using EHR audit data, which may not fully capture or reflect attending interactions with student notes and may also be impacted by the accuracy of the EHR time-logs that can have variability in their correlation to true clinician behavior[19] Fourth, while we have anecdotal evidence that attending physicians and residents perceived these changes favorably and did not experience negative effects, we did not formally evaluate their perceptions/experience. Finally, these results reflect the experience of a single academic pediatric institution, which may not be generalizable to all educational settings.

Further research is required to fully evaluate the effect of these changes on total time spent by medical students in the EHR as well as the effects on clinical competency, both to understand how the note writing process may enrich education and to identify possible negative effects of decreased time in other activities such as direct patient engagement.[20] Further work to develop enhanced evaluation methods of medical student notes may provide insight into medical student clinical knowledge and reasoning.[15] Further evaluation should also focus on the effects of these changes on attending documentation burden and quality of documentation to ensure that the desired outcomes are being achieved without introducing new unintended consequences.

Conclusion

Inclusion of medical student notes in the LMR led to increased participation in note writing by medical students, increased engagement in reviewing medical student notes by attendings, and positive perceptions by medical students. Further research is necessary to fully understand the effect of these changes on medical student education and attending physicians' documentation burden.

Clinical Relevance Statement

Note writing plays an important role in training physicians to think critically and communicate with others. New guidelines allow for medical student notes to be integrated into the clinical care spectrum using the electronic health record. These changes can be made easily and allow for improved student engagement in the care process.

Multiple Choice Questions

1. The 2018 CMS billing changes allow for medical student notes to be part of the billable encounter provided that:

   • Physical presence requirements are met.

   • The attending physicians satisfy the performance requirement and verify the documentation.

   • The student's note is separated from the attending physician's note during HIM record requests.

   • All of the above.

   • Options a and b.

   Correct Answer: The correct answer is e. The 2018 CMS billing changes allow for student notes to be part of the billable record if physical presence requirements are met (i.e., confirming physical exam findings), and if the attending physician satisfies the performance requirement and verifies the documentation (i.e., performs their own medical decision-making and reviews and attests the student's note). There is no requirement to separate the student's note from the attending's note.

2. Including medical student notes as part of the billable record has the potential to:

   • Increase attending engagement with notes

   • Enhance student learning

   • Make student work more meaningful to patient care

   • All of the above

   Correct Answer: The correct answer is option d. The students surveyed after making their notes part of the billable record agreed that this practice enhanced their learning and made their work more meaningful. EHR data showed a large change in attending interaction with student notes (4% prior compared with 84% after).

Conflict of Interest

None declared.

Acknowledgments

The authors would like to thank the Stanford Children's Health IS Analytics team for their help in data management.

Protection of Human and Animal Subjects

The Stanford University IRB Board determined that this project does not meet the definition of human subject research as defined in federal regulations 45 CFR 46.102 and is exempted from further IRB review.

• References

• 1 Weed LL. Medical records that guide and teach. N Engl J Med 1968; 278 (11) 593-600
• 2 Learning objectives for medical student education--guidelines for medical schools: report I of the medical school objectives project. Acad Med 1999; 74 (01) 13-18
• 3 Percent of Hospitals. By type, that possess certified health IT. Accessed January 17, 2020 at: https://dashboard.healthit.gov/quickstats/pages/certified-electronic-health-record-technology-in-hospitals.php
• 4 Stephens MB, Gimbel RW, Pangaro L. Commentary: The RIME/EMR scheme: an educational approach to clinical documentation in electronic medical records. Acad Med 2011; 86 (01) 11-14
• 5 Hammoud MM, Dalymple JL, Christner JG. et al. Medical student documentation in electronic health records: a collaborative statement from the Alliance for Clinical Education. Teach Learn Med 2012; 24 (03) 257-266
• 6 Friedman E, Sainte M, Fallar R. Taking note of the perceived value and impact of medical student chart documentation on education and patient care. Acad Med 2010; 85 (09) 1440-1444
• 7 Wittels K, Wallenstein J, Patwari R, Patel S. Medical student documentation in the electronic medical record: patterns of use and barriers. West J Emerg Med 2017; 18 (01) 133-136
• 8 Medicare Learning Network. E/M service documentation provided by students (manual update). Published June 1, 2018. Accessed January 17, 2020 at: https://www.cms.gov/Outreach-and-Education/Medicare-Learning-Network-MLN/MLNMattersArticles/Downloads/MM10412.pdf
• 9 Cassese T, Sharkey MS, Pincavage AT, Schwanz K, Farnan JM. Avoiding pitfalls while implementing new guidelines on student documentation. Ann Intern Med 2019; 170 (03) 193-194
• 10 Downing NL, Bates DW, Longhurst CA. Physician burnout in the electronic health record era: are we ignoring the real cause?. Ann Intern Med 2018; 169 (01) 50-51
• 11 Cheng DR, Scodellaro T, Uahwatanasakul W, South M. An electronic medical record in pediatric medical education: survey of medical students' expectations and experiences. Appl Clin Inform 2018; 9 (04) 809-816
• 12 Merriam SB, Caffarella RS, Baumgartner LM. Learning in Adulthood: A Comprehensive Guide. 3rd edition. Jossey-Bass; 2006
• 13 Newmann FM, Wehlage GG. Five standards of authentic instruction. Educ Leadersh 1993; 50 (07) 8-12
• 14 Tierney MJ, Pageler NM, Kahana M, Pantaleoni JL, Longhurst CA. Medical education in the electronic medical record (EMR) era: benefits, challenges, and future directions. Acad Med 2013; 88 (06) 748-752
• 15 Monahan K, Ye C, Gould E. et al. Copy-and-paste in medical student notes: extent, temporal trends, and relationship to scholastic performance. Appl Clin Inform 2019; 10 (03) 479-486
• 16 Pereira AG, Kim M, Seywerd M, Nesbitt B, Pitt MB. Minnesota Epic101 Collaborative. Collaborating for competency-a model for single electronic health record onboarding for medical students rotating among separate health systems. Appl Clin Inform 2018; 9 (01) 199-204
• 17 Wack M, Puymirat E, Ranque B. et al. Evaluating the impact of computerized provider order entry on medical students training at bedside: a randomized controlled trial. PLoS One 2015; 10 (09) e0138094
• 18 Knight AM, Kravet SJ, Kiyatkin D, Leff B. The effect of computerized provider order entry on medical students' ability to write orders. Teach Learn Med 2012; 24 (01) 63-70
• 19 Hron JD, Lourie E. Have you got the time? Challenges using vendor electronic health record metrics of provider efficiency. J Am Med Inform Assoc 2020; 27 (04) 644-646
• 20 Kumar A, Chi J. CMS billing guidelines and student documentation: a new era or new burden?. J Gen Intern Med 2019; 34 (04) 634-635

Publication History

Received: 27 January 2021

Accepted: 18 May 2021

Article published online:
07 July 2021