Improving ACGME Compliance for Obstetric Anesthesiology Fellows Using an Automated Email Notification System

Abstract

Background The Accreditation Council for Graduate Medical Education establishes minimum case requirements for trainees. In the subspecialty of obstetric anesthesiology, requirements for fellow participation in nonobstetric antenatal procedures pose a particular challenge due to the physical location remote from labor and delivery and frequent last-minute scheduling.

Objectives In response to this challenge, we implemented an informatics-based notification system, with the aim of increasing fellow participation in nonobstetric antenatal surgeries.

Methods In December 2014 an automated email notification system to inform obstetric anesthesiology fellows of scheduled nonobstetric surgeries in pregnant patients was initiated. Cases were identified via daily automated query of the preoperative evaluation database looking for structured documentation of current pregnancy. Information on flagged cases including patient medical record number, operating room location, and date and time of procedure were communicated to fellows via automated email daily. Median fellow participation in nonobstetric antenatal procedures per quarter before and after implementation were compared using an exact Wilcoxon-Mann-Whitney test due to low baseline absolute counts. The fraction of antenatal cases representing nonobstetric procedures completed by fellows before and after implementation was compared using a Fisher's exact test.

Results The number of nonobstetric antenatal cases logged by fellows per quarter increased significantly following implementation, from median 0[0,1] to 3[1,6] cases/quarter (p = 0.007). Additionally, nonobstetric antenatal cases completed by fellows as a percentage of total antenatal cases completed increased from 14% in preimplementation years to 52% in postimplementation years (p < 0.001).

Conclusion Through an automated email system to identify nonobstetric antenatal procedures in pregnant patients, we were able to increase the number of these cases completed by fellows during 3 years following implementation.

Keywords

anesthesiology - education - educational needs - obstetrics/gynecology - surgical and anesthesia information systems

Background and Significance

The Accreditation Council for Graduate Medical Education (ACGME) establishes minimum case experience requirements for trainees in core residency programs and subspecialty fellowship programs.[1] Programs are required to ensure that graduates are “able to competently perform all medical, diagnostic, and surgical procedures considered essential for the area of practice,” in part, through demonstrating compliance with these case minimums. In the subspecialty of obstetric anesthesiology, most case minimums are easily met through routine presence on the labor and delivery suite. This includes vaginal and cesarean deliveries involving high-risk maternal and fetal conditions and obstetric antenatal procedures including cervical cerclage, external cephalic version, and fetal surgery ([Table 1]). In contrast, when pregnant women require nonobstetric antenatal procedures, their surgery and anesthesia care is often delivered in perioperative areas remote from maternal care units and not uncommonly by nonobstetric anesthesiologists.[2] [3] This presents a unique challenge to the fellowship program, as the scheduling and planning of these nonobstetric surgeries must be somehow communicated to fellows to facilitate their participation in and learning from these cases. Historically, programs may have utilized unreliable methods for this communication, including phone calls from schedulers or anesthesia in-room providers to facilitate obstetric anesthesiology fellow presence for these procedures; however, these methods are fraught with multiple opportunities for communication breakdown and failure. Prior work suggests that communication challenges within the health care system can be successfully overcome through the use of informatics-based solutions.[4] [5] [6]

Table 1 Accreditation Council for Graduate Medical Education case minimums for obstetric anesthesiology fellowship

Case types	Minimum required number of cases
High-risk maternal comorbidity
Vaginal delivery	30
Cesarean delivery	30
High-risk fetal condition
Vaginal delivery	30
Cesarean delivery	20
Antenatal procedures	10[a]

^a Limited to no more than five cases accrued from cervical cerclage placement or removal.

Objectives

Thus, we designed and implemented a quality improvement project which created an informatics-based electronic notification system, with the aim of increasing fellow participation in nonobstetric antenatal surgeries performed during pregnancy.

Methods

This quality improvement study was determined to be exempt by the Human Research Protections Program at Vanderbilt University Medical Center. In December 2014 an automated email notification system to inform obstetric anesthesiology fellows of scheduled nonobstetric surgeries in pregnant patients was implemented. Prior to December 2014, cases were identified by nurse practitioners in the anesthesiology preoperative evaluation clinic. If the patient was noted to be pregnant, the nurse practitioner was instructed to notify the obstetric anesthesiology attending on service, who communicated by word-of-mouth to the fellow the patient's information and date and time of surgery. As a backup means of identifying these cases, labor and delivery charge nurses were also requested to notify fellows whenever requests for preoperative fetal monitoring were received from the main operating room.

Following implementation of this quality initiative, nonobstetric cases in pregnant patients were identified via daily automated query of the preoperative evaluation database looking for structured documentation of current pregnancy ([Supplementary Material A], available in the online version). A functionality which identified positive point-of-care urine pregnancy test results and assessed laboratory values for plasma human chorionic gonadotropin indicating pregnancy (value > 2 mIU/mL) was later added in June 2016. Information on flagged cases including patient medical record number, operating room location, and date and time of procedure were communicated to fellows via automated email daily at 7:00 a.m. ([Supplementary Material B], available in the online version).

ACGME case logs for obstetric anesthesiology fellows from July 2013 to July 2018 were accessed, and case numbers for antenatal procedures, including both obstetric and nonobstetric surgeries, were recorded. Median nonobstetric antenatal procedures per quarter completed by fellows before and after implementation of the automated email notification system were compared using an exact Wilcoxon-Mann-Whitney test. The fraction of fellow-logged antenatal cases per year representing nonobstetric procedures was compared using a Fisher's exact test. To assess for possible changes in the overall number of nonobstetric antenatal procedures performed at our institution during the study period, the structured query language was retrospectively applied to records in the pre- and postimplementation years to identify the total number of flagged cases annually.

Results

Prior to implementation of the automated email notification system in December 2014, fellows participated in 1 and 3 nonobstetric antenatal case, respectively, in academic years 2013 to 2014 and 2014 to 2015. Following implementation, fellows participated in 11, 25, and 16 nonobstetric antenatal cases in academic years 2015 to 2016, 2016 to 2017, and 2017 to 2018 ([Fig. 1]). This represents a significant difference in quarterly nonobstetric cases, with fellows performing a median of 0 (interquartile range [IQR] 0,1) cases/quarter (n = 6 quarters) prior to implementation and 3 (IQR 1,6) cases/quarter (n = 14 quarters) following implementation (p = 0.007). Additionally, yearly nonobstetric antenatal cases completed by fellows as a percentage of total antenatal cases completed increased from 14% in preimplementation years to 52% in postimplementation years (p < 0.001). There was no significant difference in total nonobstetric antenatal cases performed at our institution throughout the study period (average 37 cases per year preimplementation vs. 31 cases per year postimplementation, p = 0.34).

Fig. 1 Total number of obstetric and nonobstetric antenatal surgery cases logged by obstetric anesthesiology fellows in each academic year from July 2013 through June 2018. Arrow indicates timing of intervention, with automated email system initiated in December 2014, halfway through academic year 2014 to 2015. Obstetric antenatal cases include cervical cerclage, external cephalic version, and fetal surgery cases.

Discussion

This report describes the successful implementation of an informatics-based solution to a clinical education problem, namely augmenting identification of nonobstetric antenatal surgery cases with the goal of greater obstetric anesthesiology fellow participation in such cases. We observed a considerable increase in the number of nonobstetric antenatal cases logged by fellows in the 3 years following implementation.

Our results indicate that informatics-based communication solutions may be superior to traditional means of communication when approaching obstacles within the health care system. The original case notification system for obstetric anesthesia fellows at our institution had numerous vulnerabilities which likely contributed to its failure, including (1) multiple personnel serving in the various roles involved in communicating cases, (2) infrequent occurrence of cases preventing reinforcement of the communication procedures, and (3) multiple points of potential communication breakdown. By leveraging technology, all of these vulnerabilities were overcome to ensure proper communication of cases to fellows. Prior work has similarly demonstrated improved and more efficient communication between health care entities using automated notification systems. In a study of automated pager notifications following computerized provider order entry, Jacobs et al were able to demonstrate shorter times to order implementation of both radiographic studies and respiratory therapy treatments.[7] In another example, Weingart et al assessed whether automated electronic notifications to providers regarding incomplete imaging tests could improve test completion rates. They found higher rates of completion in the intervention group, although some baseline differences in patient characteristics may account for their findings.[5]

There are also numerous ways in which an informatics solution similar to the one proposed here can be applied to improve processes in graduate medical education. First, the process of identifying educationally high-yield cases or achieving case minimum requirements for trainees can be similarly applied in other specialties. By automating the identification of such cases, program directors can ensure that educational opportunities are recognized in a timely fashion and appropriately communicated to trainees. Second, this type of informatics solution can be applied to other areas of graduate medical education in which electronic capture of data may be more timely or accurate than current practices. For example, automated operative case reporting systems have been shown to increase the capture of operative cases for general surgery interns compared with manual case entry by residents.[8] Similar results have been shown for cardiology fellows, where procedure logs automatically generated from the clinical record increased the number of documented procedures and reduced administrative time required for logging of cases.[9] Patient attribution, or attributing patient-level metrics to providers, can be modeled using electronic health record data to improve the capture of trainee provider–patient interactions, which is vital in providing appropriate feedback and evaluations to trainees.[10] In another anesthesia-specific example, Starnes et al describe the development of an automated case cancellation review system for anesthesiology residents to better understand the causes and consequences of day-of-surgery case cancellations, which is important for their education pertaining to systems-based practice as required by the ACGME.[11] Understanding the value and benefits of informatics-based educational solutions in graduate medical education, other institutions can certainly adapt the methods we present here to a variety of clinical situations using the structured query language included in supplementary material as a possible starting point.

In interpreting these results, it is important to consider multiple limitations to the study design. First, there are other possible reasons for the increase in fellow participation in nonobstetric antenatal cases throughout the study period. In conjunction with the automated email system, communications from the program director also emphasized the importance and educational value of these cases, which could have influenced fellows to seek out these opportunities more actively. Additionally, this increased emphasis could have resulted in more reliable logging of cases by fellows into the ACGME system during the postimplementation years. We also considered whether the observed increase may have been due to an overall increase in the number of nonobstetric antenatal cases performed at our institution during the study period. However, by retrospectively applying the structured query language to records in the pre- and postimplementation years, we found no change in the average annual number of positive query outputs. Another limitation of this work is that we were unable to define a gold standard for identification of a pregnant patient for nonobstetric surgery. This difficulty was structural—performance of using structured data to document a pregnancy was presumably not perfect, laboratory studies were not ordered on all patients, administrative billing and coding data could not be relied on as not all surgical encounters took place within the setting of an inpatient admission, and we did not collect ongoing accuracy data from our fellows. This precluded us from calculating performance characteristics of our automated script.

Post-2018 fellow case log data was not included in our analysis as this email-based alert system was ultimately discontinued with the implementation of Epic (Epic Systems, Madison, Wisconsin, United States) at our institution. As part of the Epic implementation, similar functionality was created as part of the go-live build. Specifically, a status board was developed that displayed cases in the main operating room where the patient was marked as having a current pregnancy. This was achieved using a custom extension which filtered cases with a rule-based approach using documentation of an active pregnancy. Although the effort for this implementation was relatively small, being completed by a physician builder with only several hours of effort, it was not well utilized in the initial implementation due to lack of active alerting of fellows. The workflow was subsequently revised to include inbasket messages sent within Epic that are triggered by the scheduling and rescheduling of these cases via case tracking events that are distributed to an inbasket pool, which includes our fellows. Additionally, efforts were made to encourage structured documentation of active pregnancies for surgical patients as part of the surgical and anesthesia preprocedure workflows. This reimplementation was conducted in a multidisciplinary fashion in coordination with our education personnel and took approximately 6 months to complete.

Conclusion

In conclusion, through the use of an easy-to-implement, low-cost, automated email system to identify nonobstetric antenatal surgical procedures, we were able to demonstrate an increase in the number of these cases completed by fellows during the 3 years following implementation.

Clinical Relevance Statement

Modern training institutions are large and intricate, and informatics systems are uniquely suited to accomplish complex educational initiatives which require the transfer of information within those systems. Importantly, these automated informatics solutions do not add, but rather reduce, human resource burdens, placing pertinent information directly into the hands of stakeholders with the greatest interest—in this case, trainees and program directors.

Multiple Choice Questions

1. Which of the following is a benefit of an informatics-based notification system over a personal communication-based system?

   • More accurate transfer of information.

   • More timely transfer of information.

   • Lower resource burden.

   • All of the above.

   Correct Answer: The correct answer is option d.

   Each of the answers a to c represent benefits of an informatics-based system.

2. Which of the following outcomes has been shown to result from implementation of an informatics-based notification or reporting system?

   • Decreased operative case cancellations.

   • Decreased logging of ACGME-required cases.

   • Increased trainee involvement in educational cases.

   • None of the above.

   Correct Answer: The correct answer is option c. The work presented here supports the use of an automated query and email system to identify nonobstetric antenatal cases during pregnancy. With the implementation of the system, trainee involvement in those cases rose appreciably. Automated case reporting has been shown to increase logging of ACGME-required cases in general surgery interns. An automated case cancellation review system was previously shown to improve resident education in systems-based practice (one of the core ACGME competencies), but has not been shown to decrease case cancellation rates.

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

This quality improvement study was determined to be exempt by the Human Research Protections Program at Vanderbilt University Medical Center. No human and/or animal subjects were included.

• References

• 1 Education ACfGM. Accessed December 8, 2020 at: https://www.acgme.org/What-We-Do/Accreditation/Common-Program-Requirements
• 2 Huang SY, Lo PH, Liu WM. et al. Outcomes after nonobstetric surgery in pregnant patients: a nationwide study. Mayo Clin Proc 2016; 91 (09) 1166-1172
• 3 Bonnet MP. Sedation and anaesthesia for non-obstetric surgery. Anaesth Crit Care Pain Med 2016; 35 (Suppl. 01) S35-S41
• 4 Dalal AK, Schaffer A, Gershanik EF. et al. The impact of automated notification on follow-up of actionable tests pending at discharge: a cluster-randomized controlled trial. J Gen Intern Med 2018; 33 (07) 1043-1051
• 5 Weingart SN, Yaghi O, Barnhart L. et al. Preventing diagnostic errors in ambulatory care: an electronic notification tool for incomplete radiology tests. Appl Clin Inform 2020; 11 (02) 276-285
• 6 Williams C, Rao A, Ziemba JB, Myers JS, Patel N. Text messaging real-time COVID-19 clinical guidance to hospital employees. Appl Clin Inform 2021; 12 (02) 259-265
• 7 Jacobs BR, Crotty E, Conway E. et al. Computerized provider order entry with pager notification improves efficiency in STAT radiographic studies and respiratory treatments. Appl Clin Inform 2010; 1 (01) 19-31
• 8 Naik ND, Abbott EF, Aho JM. et al. The ACGME case log system may not accurately represent operative experience among general surgery interns. J Surg Educ 2017; 74 (06) e106-e110
• 9 Anyanwu EC, Mor-Avi V, Ward RP. Automated procedure logs for cardiology fellows: a new training paradigm in the era of electronic health records. J Grad Med Educ 2021; 13 (01) 103-107
• 10 Mai MV, Orenstein EW, Manning JD, Luberti AA, Dziorny AC. Attributing patients to pediatric residents using electronic health record features augmented with audit logs. Appl Clin Inform 2020; 11 (03) 442-451
• 11 Starnes JR, McEvoy MD, Ehrenfeld JM, Sandberg WS, Wanderer JP. Automated case cancellation review system improves systems-based practice. J Med Syst 2015; 39 (11) 134

Publication History

Received: 24 February 2021

Accepted: 19 April 2021

Article published online:
26 May 2021