A Comparison of Resident Procedure Logs to Data Generated from an Electronic Health Record

Abstract Background  Emergency medicine (EM) residents are required to report procedural competency, often in a database separate from the electronic health record (EHR), in a redundant and time-consuming manner. We hypothesize that, if documented in an appropriate manner, procedural reports generated from an EHR reliably exceed those required by the Accreditation Council for Graduate Medical Education (ACGME) as well as those self-reported by EM residents. Objectives  (1) To compare the number of medical resuscitations recorded in the EHR to the number documented by residents in a separate database. (2) To compare the number of medical resuscitations recorded in the EHR to the ACGME requirement for graduation. Methods  Self-reported numbers of adult medical resuscitation by each resident of the previous three graduating classes of one EM program were compared with those generated from the EHR (Epic Systems, Verona, Wisconsin). There is no discrete documentation of medical resuscitations in the EHR. The ACGME describes a resuscitation as “…patient care for which prolonged physician attention is needed,” and thus, a surrogate was determined to be any patient for which the attending physician documented critical care time. Results  Data generated from the EHR reliably exceeded reported (mean [M] = 165.78, standard deviation [SD] = 45.97) and required (M = 188.09, SD = 30.93) numbers for adult medical resuscitations for 100% of the residents of the past three graduating classes ( n  = 32). Conclusion  In an accredited EM residency program that utilizes a modern EHR with a validated reporting functionality, residents should not need to redundantly log the number of adult medical resuscitations performed. Each resident in this study performed significantly more adult medical resuscitations than they logged and more than required by the ACGME, and thus, the time spent documenting these in a separate database was superfluous. Furthermore, this process increases resident awareness of proper documentation and data stewardship, two skills certain to prevail throughout their careers as modern EM physicians.


Introduction
Medical residents in most specialties are required to keep a record of their procedures performed during residency per guidelines set by the Accreditation Council for Graduate Medical Education (ACGME).These procedure logs serve as a record of procedural competency required for residency graduation and often required for hiring as well.These procedure logs are often completed by individual residents in a database separate from the electronic health record (EHR) in a redundant and time-consuming manner.This dated process has not translated well to the data-heavy era of EHRs.We examined the procedure logs recorded by residents from an emergency medicine (EM) residency program with procedural reports generated by an EHR.In particular, we compared the number of medical resuscitations logged by residents with those captured by the EHR.We hypothesized that, if documented in an appropriate manner, procedural reports generated from an EHR reliably exceed those required by the ACGME as well as those self-reported by EM residents.
Prior studies have explored similar benefits of procedural reports generated from an EHR.Outcomes identified in a study by Seufert et al included an increase in daily mean number of procedures logged, while simultaneously being more detailed and complete than data self-reported by EM residents. 1

Objectives
• To compare the number of medical resuscitations recorded in the EHR to the number documented by residents in a separate database.• To compare the number of medical resuscitations recorded in the EHR to the ACGME requirement for graduation.

Methods
Self-reported numbers of adult medical resuscitations by each resident of the previous three graduating classes of one EM program were compared with those generated from the EHR (Epic Systems, Verona, Wisconsin; ►Table 1).Data were collected via the self-reporting tool SlicerDicer.Resident procedure logs were retrieved from the in-house residency management software (New Innovations, Uniontown, Ohio).These logs include approved real-life cases managed during residency training as well as simulated cases performed during procedural conference.The ACGME describes a resuscitation as "…patient care for which prolonged physician attention is needed," 2 and thus, a surrogate was determined to be any patient for which the attending physician documented critical care time.Resident attribution was accomplished by identifying the "first resident assigned" as recorded in the EHR.This is a discrete data element captured during normal resident workflows.Each chief complaint was reviewed to separate medical resuscitations from trauma resuscitations.These were further delineated into adult and pediatric resuscitations by the patient's age at the time of visit.

Results
Data generated from the EHR reliably exceeded reported (mean [M] ¼ 165.78, standard deviation [SD] ¼ 45.97) numbers for adult medical resuscitations for 100% of the residents of the past three graduating classes (n ¼ 32; ►Table 2).Data generated from the EHR required also exceeded the number required by the ACGME for procedural competency for 100% of the residents (M ¼ 188.09,SD ¼ 30.93; ►Table 2).

Discussion
In the era of EHRs, a large amount of data are being collected in the background that sometimes precludes the need for separate databases that were previously necessary.In this instance of resident procedural logs, the number of medical resuscitations recorded by the EHR reliably exceeded both the number recorded by the residents as well as the number required by the ACGME for graduation for 100% of the residents.
One challenge in this study was developing a surrogate in the EHR for medical resuscitations as there is no discrete entry in the EHR that declares an intervention to be a medical resuscitation.In addition, residents, particularly interns, are often not aware of which interventions qualify as a medical resuscitation given the ambiguity in the ACGME's definition of medical resuscitation.This is in contrast to other procedures such as lumbar punctures and intubations, which are clearly defined.Based on the ACGME's definition of a resuscitation as "…patient care for which prolonged physician attention is needed," 2 we used critical care time documented by the attending physician as a surrogate for medical resuscitation.This appears to be a fair surrogate as many of the same patient abnormalities, such as shock, and therapeutic interventions, such as intravenous vasopressors, recognized as part of critical care require prolonged physician attention.Thus, capturing a resuscitation is dependent on critical care time being documented for patients.Given this dependency on critical care time documentation, we are potentially capturing fewer than the actual number of resuscitations performed by residents if critical care time is not documented by the attending physician.
The necessity for developing a surrogate for medical resuscitations illustrates a principle of data literacy 3 that the type and quality of data output from the EHR is determined by the data input.As there was no discrete documentation of medical resuscitations in the EHR, it was not possible to obtain a direct log of medical resuscitations from the EHR.
There were several self-reported documentation numbers that either equaled or barely exceeded the number medical resuscitations required by the ACGME.This could be due to cessation of reporting after meeting the minimum standard for residency graduation.Hence, the logs obtained from the EHR are more accurate representations of the number of procedures the residents have actually performed.

Conclusion
In an accredited EM residency program that utilizes a modern EHR with a validated reporting functionality, resi-dents should not need to redundantly log the number of adult medical resuscitations performed in a separate database.Each resident in this study performed significantly more adult medical resuscitations than they logged and

Clinical Relevance Statement
In an accredited EM residency program that utilizes a modern EHR with a validated reporting functionality, residents should not need to redundantly log the number of adult medical resuscitations performed in a separate database.The EHR in this case provided a log of medical resusci-tations that was more accurate in capturing the number of medical resuscitations performed by residents than the separate database the residents used for self-reporting.

Table 1
Resident data from SlicerDicer

Table 2
Comparison of average reported versus retrieved adult medical resuscitations more than required by the ACGME, and thus, the time spent documenting these in a separate database was superfluous.This method has the added advantage of being an accurate representation of real-life scenarios instead of simulated patient resuscitations.Furthermore, this process increases resident awareness of proper documentation and data stewardship, two skills certain to prevail throughout their careers as modern EM physicians.Further analysis is being conducted to determine if these findings apply to other procedural reporting required by the ACGME.