Understanding CancelRx: Results of End-to-End Functional Testing, Proactive Risk Assessment, and Pilot Implementation

Abstract

Background CancelRx allows prescribers to send electronic cancellation messages to pharmacies when medications are discontinued. Little is known about its functionality and impact on clinical workflows.

Objectives To understand CancelRx functionality, its potential impact on workflows and medication safety risks, and to develop mitigating strategies for risks introduced by implementation.

Methods We conducted direct observations and semi-structured interviews to develop CancelRx use cases and assessed CancelRx in an end-to-end test environment, proactive risk assessment, and pilot implementation from April 16 to July 15, 2018.

Results E-cancellations were sent upon discontinuation of e-prescriptions written within the electronic health record (EHR), but not other medications (e.g., printed prescriptions) and could be initiated by nonprescribers. In our proactive risk assessment, CancelRx implementation eliminated five of seven failure modes in outpatient prescribing to Johns Hopkins pharmacies, but introduced new risks, including (1) failure to act if an e-cancellation was not sent or was unsuccessful; (2) failure to cancel all prescriptions for a medication; (3) errors in manual matching; and (4) erroneous medication cancellations. We identified potential mitigation strategies for these risks. During pilot implementation, 92.4% (428/463) of e-cancellations had confirmed approval by the receiving pharmacy, while 4.5% (21/463) were denied, and 3.0% (14/463) had no e-cancellation response. Among e-cancellations received by the pilot pharmacy, 1.7% (7/408) required manual matching by pharmacy staff. Based on performance in testing, 73.4% (340/463) of completed e-cancellations would be expected to generate an in-basket message, including 21 (6.2%) denials and 319/340 (93.8%) approvals with a note from the pharmacy.

Conclusion CancelRx is an important functionality with the potential to decrease adverse events due to medication errors. However, changes in implementation in our EHR and pharmacy software and enhancements in the CancelRx standard are needed to maximize safety and usability. Further studies are needed to evaluate the impact of e-cancellation on medication safety.

Keywords

electronic prescribing - medication errors - risk assessment

Background and Significance

Adverse drug events (ADE) are frequent in ambulatory care, leading to an estimated 4.5 million patient visits each year.[1] [2] [3] Medication errors are a leading cause of preventable ADE and patient harm in ambulatory settings.[4] [5] Although computerized provider order entry with clinical decision support can decrease medication errors,[6] [7] there is increasing recognition of safety risks associated with electronic prescribing.[8] [9] [10] [11] [12] [13] [14]

Safe prescribing in ambulatory care is a complex process that entails coordination among multiple stakeholders, including prescribers, pharmacy staff, and patients. The vast majority of prescriptions in the United States are now sent electronically from electronic health records (EHRs), through an intermediary, most commonly Surescripts, to pharmacies.[15] However, most EHRs do not currently notify pharmacies when prescribers discontinue or change medications; instead, prescribers are expected to call or fax pharmacies to communicate this information. An analysis of a sample of new e-prescriptions identified messages to discontinue another prescription in 0.7% of transactions, or an estimated 10 million e-prescriptions annually,[16] which likely represents a small fraction of medication discontinuations.

Failure to communicate medication changes to pharmacies is a significant contributing factor to medication errors and preventable ADE; without communication, an estimated 1.5 to 5% of prescriptions are filled after intended discontinuation by a clinician.[15] [17] Erroneous continuation of these medications, including duplicate therapies and medications prescribed in error, has led to serious adverse events, including clinically significant drug side effects[15] [18] and inadequate supply of critical medications,[19] contributing to hospital admimssions.[17] [19] Improved communication between prescribers and pharmacies, including electronic communication of medication discontinuation, is a critical safety priority.

The National Council for Prescription Drug Programs' SCRIPT standard (version 10.6) for e-prescribing supports a functionality called CancelRx, which allows prescribers to send electronic cancellation messages, or “e-cancellations”, through Surescripts or another intermediary to pharmacies when medications are discontinued or changed. The importance of CancelRx for safe e-prescribing has been recognized by multiple stakeholders.[20] [21] Despite the availability of CancelRx in the SCRIPT standard since 2010, adoption has been limited. The Medicare Access and CHIP Reauthorization Act of 2015 (MACRA) required EHRs to include electronic cancellation messaging to pharmacies, but prescribers and pharmacies must also certify their capacity to handle CancelRx messages.[21] While the number of CancelRx transactions on the Surescripts network is increasing,[21] the impact of CancelRx on the workflow of key stakeholders and patient outcomes remains largely unknown.

Similar to some other large health systems, Johns Hopkins uses two different information technology systems for its EHR (Epic, Verona, Wisconsin, United States) and outpatient pharmacy management software (EnterpriseRx, v. 8.0.0.6 and 8.3.0.5, McKesson Corporation, San Francisco, California, United States).[19] As a result, e-prescriptions written by Johns Hopkins clinicians are transmitted externally through the Surescripts network to Johns Hopkins outpatient pharmacies, using the same exact mechanism as external pharmacies. In the absence of CancelRx, Johns Hopkins outpatient pharmacies were not automatically notified of medication changes made by Johns Hopkins clinicians, leading to potential dispensing errors and ADEs. Given the likely benefit but limited understanding of CancelRx functionality and the multiple stakeholders involved in implementation, we sought to understand the impact of CancelRx on workflows and medication safety risks to inform safe implementation of CancelRx at Johns Hopkins.

Objectives

The objectives of this study were (1) to understand CancelRx functionality and its potential impact on the workflows of key stakeholders in ambulatory care; (2) to assess the potential impact of CancelRx on safety risks related to communication of medication discontinuation; and (3) to develop mitigating strategies for remaining risks in communication of medication discontinuation and those introduced by CancelRx implementation.

Methods

Our primary study team included the Johns Hopkins Medicine Health Information Technology (HIT) Safety Officer (A.R.C.), HIT leads from Epic ambulatory (A.T.M. and L.P.D.) and pharmacy (B.A.T.), a pharmacy safety officer (K.G.D.), a human factors psychologist (S.K.) and a human factors engineer (Y.Y.), a HIT Safety project manager (N.B.), and a general internist and ambulatory patient safety researcher (S.I.P.). To attain our objectives, we developed a series of use cases for CancelRx, established an end-to-end test environment for functional testing, conducted a proactive risk assessment of medication discontinuation within ambulatory care, and implemented CancelRx in a pilot practice and pharmacy at Johns Hopkins. The Johns Hopkins Bartlett Specialty Practice, an adult infectious disease specialty practice, and the Bartlett Pharmacy were selected due to the close working relationship between the practice and pharmacy, the established safety infrastructure in the practice, including a comprehensive unit based safety program team,[22] [23] and the frequent use of high-risk medications, such as highly active antiretroviral therapy (HAART).[24]

Development of Use Cases

A study team member conducted four direct observations of pharmacy staff to understand their workflows within the pharmacy, and specifically within the EnterpriseRx pharmacy management software. The team member took hand-written notes which were used to create a process map of pharmacy workflows. As observed discontinuations were infrequent, we then conducted 11 semi-structured interviews, including 6 with Bartlett Specialty Practice prescribers and 5 with Bartlett pharmacists and pharmacy technicians to understand current communications and processes around medication discontinuation and changes. Using an interview guide developed by the team, two team members conducted the interviews to identify whether and how providers communicated with the pharmacy for five scenarios, including adverse drug reaction, medication written in error, change in medication regimen, change in dose or frequency, or completion of therapy. The interviewer took written notes, which were transcribed for analysis. The study team conducted thematic analysis of the interview notes, guided by the Systems Engineering in Patient Safety (SEIPS) human factors framework. The SEIPS model describes a health care work system in which patients and providers interact using tools (i.e., CancelRx) to perform a task (e.g., discontinue medications) within a work system (i.e., Johns Hopkins) informed by organization and societal context (e.g., regulations, policies).[25] [26]

Creation of End-to-End Test Environment and Test Scenarios

In collaboration with Surescripts (Arlington, Virginia, United States), we established an end-to-end test environment which simulated the Johns Hopkins production environment for e-prescribing. In this environment, e-prescription and “e-cancellation” messages could be transmitted via Surescripts from our Epic test environment (2017, Verona, Wisconsin, United States) to our McKesson EnterpriseRx (v. 8.0.0.6 and 8.3.0.5) test environment. We evaluated three stages of CancelRx implementation: (1) transmission of the e-cancellation message; (2) processing of the e-cancellation and response by the pharmacy; and (3) communication of e-cancellation status back to the prescriber.

We developed test scenarios based on the intentions of prescribers and clinical workflows in order to examine e-cancellation under various conditions including: the presence or absence of an e-prescription; whether an e-prescription was active or inactive; the location of the e-prescription in the pharmacy management system; and whether the pharmacy was able to accept e-cancellation messages. To simulate pilot implementation, we used two test prescribers, one with and one without the e-cancellation functionality, and modified the test environment order transmittal rule to restrict the e-cancel functionality to a single pilot practice by evaluating the encounter department of the original e-prescription.

Proactive Risk Assessment

We conducted a failure mode and effect analysis to proactively identify potential risks related to medication errors in ambulatory prescribing to Johns Hopkins pharmacies and their severity, frequency, and detectability prior to e-cancellation implementation. We recruited stakeholders, including physicians, nurse practitioners, nurses, pharmacists, and pharmacy technicians across Johns Hopkins to participate with the core team in a series of nine 60 to 90 minute focus groups for the proactive risk assessment. A total of 17 people including the core team participated in at least one of the focus groups. In the initial focus groups, stakeholders reviewed and confirmed the workflows and use cases previously developed by the study team. In subsequent focus groups, stakeholders identified potential medication safety risks in the existing workflow. Participants in the final focus groups were asked to score each failure mode by assigning a likelihood from 1 to 4 to the severity of the associated risk, the frequency of occurrence, and the detectability of an event, with higher scores indicating greater risk, frequency, and difficulty of detection, respectively. We assigned the final score for each based on discussion and consensus. The total score, or risk prediction number, for each failure mode was determined by multiplying these scores (severity × frequency × detectability); a higher score indicates a greater potential risk. The study team then presented the e-cancellation functionality identified from testing of the scenarios in the test environment. The stakeholder focus groups evaluated which existing risks were mitigated by e-cancellation implementation and identified and scored any new medication safety risks.

Pilot Implementation and Evaluation

We subsequently implemented e-cancellation with 14 pilot prescribers in the infectious disease specialty practice and pharmacy. Pilot prescribers were given the cancel order/request service level within Epic, and CancelRx was enabled for the pilot pharmacy. As any reorder of a medication results in the discontinuation of the old prescription and generation of a new prescription, we elected to suppress e-cancellation messages for prescriptions that were discontinued when a medication was reordered. As in the test environment, we also updated a portion of the enterprise order transmittal rule in the EHR to have it evaluate the encounter department of the original e-prescription in order to restrict the functionality to the pilot practice.

We analyzed 3 months of transactions following CancelRx implementation (April 16 to July 15, 2018). Our primary outcome was confirmed prescription deletion or deactivation by the pharmacy, measured as the proportion of e-cancellations that resulted in an electronic response indicating approval by the receiving pharmacy. We used an orders report to identify medications that were discontinued in the EHR by pilot prescribers and an EHR interfaces report to identify which of these discontinuations resulted in an e-cancellation. We determined the pharmacy response to the e-cancellation (i.e., approved or denied) using pharmacy management software data for the Johns Hopkins pharmacy and Surescripts data for external pharmacies. If neither were available, we conducted chart review to determine if a pharmacy message was received within Epic.

Secondary outcomes included the proportion of medication discontinuations that generated an e-cancellation, the number of in-basket messages that would be generated, based on performance in the test environment and the response to e-cancellation from the pharmacy, and time to e-cancellation response by pharmacies. Time from e-cancellation to pharmacy response was calculated by Surescripts as the difference in time between the receipt of each message at Surescripts. All analyses were conducted using SAS version 9.4 (SAS Institute Inc., Cary, North Carolina, United States). This study was acknowledged as a quality improvement initiative by the Johns Hopkins School of Medicine Institutional Review Board.

Results

Current Processes and Use Cases

E-prescriptions enter McKesson EnterpriseRx through the Inbound Communications Queue (ICQ) and progress through a series of queues (e.g., data entry, dispensing, and verification) as they are processed by pharmacy staff. Within the pharmacy management software, successful manual cancellation of an unfilled prescription resulted in deletion of the prescription from the patient profile. In contrast, successful manual cancellation of a previously dispensed prescription results in the deactivation of the prescription. The deactivated prescription remains in the patient profile with a status of “discontinued” and cannot be used to initiate a refill request.

At baseline, if communication of medication discontinuation was attempted, prescribers used a variety of methods to communicate with pharmacy staff, including placing a phone call themselves, delegating communication to a staff member who would call or send a fax, sending an in-basket message to a designated pharmacist within the EHR, and using the “note to pharmacy” field on a different prescription. Pharmacy staff confirmed that prescribers would communicate medication changes or discontinuation through a variety of methods, but indicated that prescribers infrequently contacted them. Instead, pharmacists were highly vigilant for errors and would access the EHR directly to clarify patients' medication regimens, an option unavailable to external pharmacies. Pharmacy staff reported initiating the great majority of communications with prescribers as a result of identification of discrepancies.

We created a framework to understand prescriber intentions, reasons for medication discontinuation, and successful pharmacy outcomes of discontinuation ([Table 1]). A successful pharmacy outcome depended on intent of the prescriber, and in some cases, on the specific reason for medication discontinuation (e.g., adverse drug reaction). A successful pharmacy outcome might require deactivation of more than a single prescription.

Table 1 Prescriber intentions, reasons for medication discontinuation, and successful pharmacy outcomes

Prescriber intention	Reason	Successful pharmacy outcome
Prevent an initial fill	Prescribing error—wrong medication, patient, pharmacy, dose, or frequency	Deletion of prescription
	Alternate therapy, e.g., change in decision making or due to cost	Deletion of prescription
Prevent a subsequent fill	Adverse drug reaction	Deactivation of all prior prescriptions for this medication; pharmacy notified of allergy
	Alternate therapy, e.g., due to effectiveness or cost	Deactivation of all prior prescriptions for this medication
	Dose adjustment	Deactivation of all prior prescriptions for this medication
	Therapy completed	Deactivation of all prior prescriptions for this medication
Ensure only one prescription is filled	Duplicate therapy	Deletion or deactivation of one or more prescriptions, retaining a single active prescription
Medication reconciliation without intent to prevent a fill	Therapy completed but may request refill	Maintenance of prescription on profile as completed but able to request refill

Performance of CancelRx

E-cancellation Transmission from the EHR

In our test environment, an e-cancellation message was only sent when discontinuing a medication originally e-prescribed in the EHR to a pharmacy that accepted e-cancellation messages ([Table 2]). In testing of the pilot scenario, an e-cancellation was only sent when the original e-prescription was written by a CancelRx enabled prescriber from a CancelRx enabled practice ([Table 3]). In addition, an e-cancellation could be initiated by a nonprescriber, such as a certified medical assistant, as long as the original prescriber and practice were enabled for CancelRx. By default, e-cancellation messages were sent for all reasons for medication discontinuation, including medication reorder, which discontinues an existing e-prescription and generates a new e-prescription.

Table 2 Results of CancelRx functional testing scenarios

Scenario	EHR outcome	Pharmacy outcome
Active e-prescription in EHR and pharmacy
E-prescription written by pilot prescriber	E-cancellation sent to pharmacy	E-prescription in Inbound Communications Queue (ICQ): • E-cancellation overwrites the e-prescription in ICQ • Only option is to remove from ICQ
		E-prescription in subsequent queue (e.g., data entry, dispensing, verification): • E-cancellation is received in ICQ • E-prescription remains in queue but is automatically matched with the e-cancellation and marked with “Rx unable to be processed” • Only option is to remove e-cancellation in ICQ and e-prescription in subsequent queue
		E-prescription in process within a queue: • E-cancellation is received in ICQ • E-prescription remains in queue but is not automatically matched with the e-cancellation • E-prescription can continue to move forward to the next queue until manually matched by pharmacy staff • E-cancellation can be erroneously matched (e.g., wrong patient or wrong drug) without any warning
		E-prescription filled and ready for patient pick-up: • E-cancellation is received in ICQ • E-prescription is marked as “You cannot Rx status change this Rx. The Rx status is not active” • Unable to confirm in test environment whether medication could be sold to patient
E-prescription written by nonpilot prescriber	No e-cancellation sent to pharmacy
Active e-prescription in EHR but not in pharmacy
Prescription transferred	E-cancellation sent to pharmacy	• No e-cancellation is received in ICQ • E-cancel is automatically denied with a note “Rx was transferred”
Active e-prescription in pharmacy, but not local EHR (prescription originated in other EHR)
Patient or physician reported medication	No e-cancel sent
Prescription reconciled from outside EHR	No e-cancel sent
No e-prescription
Print and “no print” prescriptions	No e-cancel sent
Inactive e-prescription
Prescription expired	Unable to test
Other
Pharmacy does not accept CancelRx	E-cancel message attempted but fails
Prescription not found	Unable to test

Abbreviation: EHR, electronic health record.

Table 3 CancelRx message generation and transmission under pilot conditions

Original prescriber	Pharmacy accepts CancelRx	Cancellation generated by the EHR	Cancellation transmitted to pharmacy	Cancel messaging outcome
Pilot user in pilot DEP[a]	Yes	Yes	Yes	E-cancellation sent
	No	Yes	No	Failure message that receiving partner does not accept CancelRx
Nonpilot user in pilot DEP	Yes	Yes	No	Failure message that CancelRx not supported by sender
	No	Yes	No	Failure message that CancelRx not supported by sender
Pilot user in nonpilot DEP	Yes/No	No	NA	No e-cancellation generated
Nonpilot user in nonpilot DEP	Yes/No	No	NA	No e-cancellation generated

Abbreviation: EHR, electronic health record.

^a DEP indicates the clinical department within the EHR.

During the 3 months following pilot implementation, Johns Hopkins users sent 463 e-cancellations, 292 resulting from medication discontinuation by pilot participants, and 159 resulting from discontinuation of medications originally prescribed by pilot participants by nonpilot users, consistent with the performance in the test environment. Most (713 of 1005, 70.9%) medication discontinuations by pilot providers did not result in an e-cancellation, most commonly because the medication was not e-prescribed in the EHR (n = 374, 37.2% of medication discontinuations), the e-prescription was written by a nonpilot prescriber (n = 258, 25.7%), or the pharmacy did not accept e-cancellation transactions (n = 67, 6.6%).

Pharmacy Processing and Response

In most scenarios in our test environment, e-cancellations were automatically matched to an existing e-prescription and presented in the ICQ of the pharmacy management software for review by pharmacy staff. However, manual matching of the e-cancellation in the ICQ with an existing e-prescription was necessary if the e-prescription was in use in a work queue at the time of receipt of the e-cancellation message. When manual matching was required, the prescription remained active and could move forward through the pharmacy workflow until matched to the e-cancellation message. Once a match was completed, either automatically or manually, the pharmacy management system generated a pharmacy response message. If a matching e-prescription could not be found by the system or the prescription had been transferred to another pharmacy, the pharmacy system automatically generated a denial message for that e-cancellation; the e-cancellation did not enter the ICQ and was not seen by pharmacy staff.

Of all 463 e-cancellation messages sent during our pilot, 428 (92.4%) had confirmation of e-cancellation approval by the pharmacy, 21 (4.5%) were denied by the pharmacy, primarily because the prescription had been transferred (18/21, 85.7%), and 14 (3.0%) had no e-cancellation response. Among the 408 e-cancellations received by the pilot Johns Hopkins pharmacy, seven (1.7%) required manual matching by pharmacy staff. Surescripts data were available for time to cancellation response on 413 (89.2%) of 463 e-cancellations. Of these, 21 were not interpretable as they indicated a pharmacy response up to 16 seconds prior to the e-cancellation. In the remaining 392, response time was less than 5 seconds for 95.4% (374/392).

Communication of e-Cancellation Status within the EHR

In our test environment, the status of the e-cancellation transaction was primarily communicated to the prescriber through the use of in-basket messages. These were generated if the e-cancellation was denied, the e-cancellation was accepted but included a note from the pharmacy, or the pharmacy did not accept e-cancellations. In addition, in the pilot scenario, in-basket messages were also generated when a pilot participant discontinued prescriptions that were not written by a prescriber with e-cancellation capacity. If the prescription had been filled prior to the e-cancellation, the pharmacy software returned this information as a note in the pharmacy response. The pharmacy response was visible in the medication order report in the EHR, but was not visible directly from the medication list; regardless of e-cancellation status, the medication list would display “discontinued.” Based on performance in testing, 73.4% (340/463) of completed e-cancellations from pilot implementation would be expected to generate an in-basket message, including 21 (6.2%) denials and 319/340 (93.8%) approvals with a note from the pharmacy. In addition, we would expect 67 additional in-basket messages from discontinuation of e-prescriptions to pharmacies that did not accept e-cancellation and 258 messages generated by discontinuation of prescriptions written by a nonpilot prescriber.

Proactive Risk Assessment

Our proactive risk assessment revealed the need for improved communications between Johns Hopkins prescribers and pharmacies ([Appendix Table A]). We identified seven failure modes with risk prediction numbers up to 32 (of possible 64). Despite ongoing messaging to clinicians that medication discontinuation in Epic did not inform the pharmacy, the greatest risk identified by our stakeholders was prescribers' lack of communication with pharmacy staff when medications were discontinued or changed, which was identified as the norm. Even when communication of change or medication discontinuation did occur, medication safety risks remained due to the possibility of delays or errors in communication. Following implementation of CancelRx, we anticipated that only two of the original seven failure modes would remain. However, we identified novel failure modes associated with CancelRx implementation ([Table 4]), including: (1) failure to recognize when an e-cancellation message is not sent or is unsuccessful; (2) failure to cancel medications when multiple prescriptions exist (e.g., prescriptions with different doses or at multiple pharmacies), as a prescriber cannot indicate that all prescriptions of a medication should be e-cancelled; (3) delay or error in e-cancellation due to erroneous manual matching of the e-cancellation with the e-prescription; and (4) erroneous cancellation of a necessary medication, either when an e-cancellation message is sent with a medication renewal request or as a result of an error in medication reconciliation. We identified potential mitigation strategies for these risks for both pilot and future implementation, including changes within the EHR, pharmacy management software, and the data standard ([Table 4]).

Appendix Table A Risks associated with discontinuation of medications e-prescribed in ambulatory care within Johns Hopkins—before and after CancelRx implementation

Failure mode	Scenario	Severity[a]	Detectability[b]	Before CancelRx implementation		After CancelRx implementation
				Frequency[c]	Risk prediction number[d]	Frequency	Risk prediction number
No communication with pharmacy attempted	Wrong patient or medication	4	4	2	32	–	–
	Duplicate therapy	3	2	4	24	–	–
	Alternate medication[e]	2	3	3.3	19.8	–	–
	Adverse drug event[f]	4	2	2	16	–	–
	Change in dose or frequency[e]	2	2	3.6	14.5	–	–
	Wrong pharmacy	1	2	4	8	–	–
Pharmacist misses/misunderstands instructions in “notes to pharmacy”		2	3	4	24	–	–
Pharmacist does not read in-basket message prior to medication dispensing		4	3	2	24	–	–
Pharmacist misses duplicate alert for medication[e]		2.9	3	2	17.3	2	17.3
Prescriber contacts pharmacist out of workflow on medication[e]		2.9	2	2	11.5	–	–
Pharmacist does not hear voicemail prior to medication dispensing		4	2	1	8	–	–
Pharmacist unable to reach prescriber about medication concern[e]		2.9	1	2	5.8	2	5.8
User cannot specify that all prior prescriptions of medication should be discontinued		3	3	–	–	3	27
Pharmacist matches e-cancellation to wrong prescription		4	3	–	–	2	24
No notification to user by EHR when an e-cancellation is not addressed by a pharmacy		2	4	–	–	2	16
User does not recognize that an in-basket message indicates an e-cancellation failure		2	2	–	–	2	8
User sends e-cancellation in error during medication reconciliation[g]		NS[h]	NS	–	–	NS	NS
Pharmacist cancels active medication when e-cancellation is sent with a renewal request[g]		NS	NS	–	–	NS	NS

^a Frequency categories: 1, unlikely; 2, several times; 3, frequently; 4, continually.

^b Severity categories: 1, less than minor injury; 2, impairment or injury requiring medical intervention; 3, permanent impairment or life-threatening injury; 4, permanent disability or death.

^c Detectability categories: 1, certain detection; 2, significant probability of detection; 3, minor probability of detection; 4, cannot detect.

^d Risk prediction number is the product of frequency, severity, and detectability.

^e Severity, detectability, frequency, and risk prediction number are the geometric mean for three scenarios—for high, medium, and low risk medications.

^f Severity, detectability, frequency, and risk prediction number are the geometric mean for two scenarios—patient receiving all medications within Johns Hopkins and patient receiving medications from within and outside of Johns Hopkins.

^g This failure mode was identified after the initial proactive risk assessment and was not scored.

^h NS, not scored.

Table 4 Failure modes in CancelRx, mitigation strategies in pilot implementation, and future mitigation opportunities

Failure mode	Mitigation strategies for pilot implementation	Recommendations for development
Prescriber does not recognize when an e-cancellation is not sent	• Train prescribers in functionality of CancelRx	• Increase visibility of cancellation status
Prescriber does not recognize that an in-basket message indicates an e-cancellation failure	• Route in-basket messages to trained RN • Train prescribers to locate status in order report	• Increase visibility of cancellation status • Reduce in-basket messages that are not actionable
EHR does not notify prescriber when an e-cancellation is not addressed by a pharmacy	• Monitor frequency	• Notify prescriber when an e-cancellation is not addressed by a pharmacy
Pharmacist cancels active medication when e-cancellation is sent with a renewal request	• Suppress cancellation with prescription renewals	• Transmit cancellation reason
User sends e-cancellation in error during medication reconciliation	• Train prescribers in functionality of CancelRx	• Increase visibility of cancellation status • Control by discontinuing user
Pharmacist matches e-cancellation to wrong prescription	• Monitor frequency	• Reduce manual matches • Provide decision support for manual matches
Prescriber cannot specify if all prior prescriptions of medication should be discontinued—one to one match only	• Assign responsibility for managing e-cancellation messages to pharmacists	• Allow prescriber to specify if all prior prescriptions of medication should be discontinued • Transmit cancellation reason (e.g., adverse drug event) • Consider transmission to multiple pharmacies

Abbreviations: EHR, electronic health record; RN, registered nurse.

Discussion

In our study, the ability to send electronic messages informing the pharmacy when medications are changed or discontinued addressed a clear unmet need for communication between prescribers and pharmacies. When e-cancellations were sent, over 90% were rapidly approved by pharmacies. However, we identified important limitations to implementation of e-cancellation in our EHR. First, there was no mechanism to send e-cancellation messages for medications that were not e-prescribed in the local EHR, including medications reconciled from outside sources, printed prescriptions, and patient-reported medications. Second, for medications that could generate an e-cancellation, determination of whether an e-cancellation was sent was based on the service level and context of the original prescriber rather than the discontinuing user; as a result, it was difficult to restrict the personnel that could generate e-cancellations (e.g., prescribers only). Third, there was little visibility of the e-cancellation transaction and outcome within a patient's medical record in the EHR; feedback to prescribers instead relied on in-basket messages, but most of these messages were not actionable. In addition, the CancelRx standard did not communicate the intent of the prescriber or allow for e-cancellation at the drug level (e.g., to prevent dispensing of all prior prescriptions of a medication).

While the current e-cancellation functionality is an important improvement, expansion to include all patient medications and all pharmacies is necessary for patient-centered care and maximal usefulness. Currently, prescribers using Epic who implement CancelRx are only able to e-cancel a subset of their patients' medications—those e-prescribed within their practice or health system to cancellation-enabled pharmacies. The impact of the restriction to e-prescribed medications was significant in our pilot—over one-third of medications discontinued by pilot prescribers could not be e-cancelled for this reason. However, this will vary by patient and across clinical practices or health systems, depending on the extent to which patients receive care from outside providers with different EHRs. This limitation might be particularly critical for some clinicians, such as emergency department and primary care clinicians, who might be more likely to discontinue medications prescribed by clinicians from other practices or health systems. The existing SCRIPT standard does not restrict CancelRx to e-prescribed medications in a given EHR. However, sufficient information about the medication would need to be sent in the e-cancellation to accurately identify the patient, prescription, and dispensing pharmacy.

In comparison, a relatively low proportion of medication discontinuations could not be e-cancelled because receiving pharmacies did not accept e-cancellations. Although several large chains recently began accepting e-cancellations, we anticipate the proportion of successful e-cancellation will be lower in other settings, as the majority of medications in our pilot were e-prescribed to the Johns Hopkins CancelRx-enabled pilot pharmacy. While our EHR vendor added functionality to fax a cancellation to pharmacies that do not accept e-cancellation, this upgrade was not available within our EHR at the start of our pilot. Further understanding of the risks and benefits of fax cancellation and its impact on pharmacy workflows is needed.

Currently, there are no automated safeguards in place to require an e-cancellation to be authorized by a prescriber; as the EHR assesses the service level, or permission, of the original prescriber, rather than the discontinuing user, any staff member with access to remove a medication from the medication list may trigger an e-cancellation. To improve the safety of CancelRx implementation, EHRs should readily allow control of permissions for e-cancellation. Given the current CancelRx implementation in Epic, removal of e-prescriptions from patients' medication lists should be completed only by prescribers with appropriate training in medication reconciliation.

Given the limitation that some but not all medication discontinuations in the EHR trigger a cancellation and with limited pharmacy adoption of CancelRx, it is critical for prescribers to have clearly visible, real-time information not only on whether an e-cancellation was successfully sent but also if the prescription was successfully deleted or deactivated in the pharmacy system. This feedback would allow the discontinuing prescriber to take appropriate action during the patient encounter if an alternative method of communication to the pharmacy is needed. In our test environment and pilot implementation, discontinuing users did not receive immediate feedback within the patient encounter, but received an in-basket message. However, these notifications were generated for any successful cancellation that included a note from the pharmacy, including successful cancellations for which the pharmacy reported the number of prior fills, when the pharmacy did not accept e-cancellation, and when the pharmacy denied an e-cancellation. Handling these messages requires a separate workflow, creating additional work and potentially leading to delays and omissions in communication to pharmacies.

Our proactive risk assessment identified novel risks with implementation resulting from these limitations in e-cancellation functionality. E-cancellation will transform erroneous discontinuation of an e-prescription in the EHR from a latent error (removal from the medication list) to an active error (prevention of dispensing of a needed medication by the pharmacy), increasing the potential for harm from these errors. The limited visibility of the e-cancellation transaction might contribute to an increased risk of unintended e-cancellations.

We identified a number of potential mitigation strategies for these novel risks, including improvements to implementation of the standard, such as presentation within the EHR to ensure communication of e-cancellation status; reducing manual matching and providing decision support within pharmacy software to reduce errors; and modification of the standard to include transmission of additional information in the e-cancellation, such as the reason for medication discontinuation and whether all prior prescriptions of a medication should be discontinued. Transmission of the reason for discontinuation was identified both by prescribers and pharmacists as an important gap in the current functionality. By providing pharmacists with a greater understanding of prescriber intentions, it might reduce the risk of erroneous e-cancellation, allow pharmacists to more readily identify safety risks (e.g., additional medications that should be discontinued in the case of an ADE), and potentially improve pharmacists' ability to counsel patients on changes in their medication regimens.

While one-to-one matching between an e-cancellation and e-prescription, as CancelRx is currently implemented, is straightforward and precise, the communication needs between prescribers and pharmacists around medication discontinuation are more complicated. Until there is widespread adoption of e-cancellation, including e-cancellation of prescriptions from sources outside of the local EHR, there will be a backlog of prescriptions that have been discontinued but remain active at the pharmacy. A mechanism that allows the prescriber to indicate intent to discontinue all prior prescriptions of a medication could more quickly eliminate redundancies. In addition, although e-cancellation of prescriptions from sources outside the local EHR would present challenges, this would move us closer to a unified and correct patient medication record and potentially prevent harm when a patient should not continue to receive a medication.

Safe HIT implementation is a critical national patient safety priority.[27] The Office of the National Coordinator for Health Information Technology sponsored the development of the SAFER guides[28] to serve as a resource for organizations to improve the safety of HIT implementation. Central to these and other HIT safety recommendations are user-centered design, involvement of multiple stakeholders in implementation, identification of ideal workflows, incremental testing within the context of implementation to confirm anticipated performance, identification of unintended consequences, and postimplementation monitoring to ensure the system performs as expected.[29] [30] [31] [32] Pilot implementation promotes these critical HIT safety recommendations, but incremental implementation within our EHR test environment was technically challenging and created unintuitive restrictions on the CancelRx functionality. Improving the ability to conduct controlled, incremental implementation of e-cancellation would allow for safer implementation, particularly in larger health systems where medication reconciliation workflows vary across clinical settings (e.g., inpatient vs. ambulatory). Support for incremental implementation would also allow for institutional learning through a phased rollout and might lead to improved adoption rates, better risk mitigation strategies, and accelerated innovation.[33]

Best practices for CancelRx implementation within EHRs and by pharmacies will need to be developed on the basis of workflow and outcomes analyses. It some situations, it is not clear if an e-cancellation should be sent. For example, a medication renewal in the EHR discontinues the prior prescription and creates a new prescription for the medication. In the absence of CancelRx, if the old and new prescriptions are not linked in the pharmacy management software, prescribers may continue to receive automated refill requests for the old prescription, even though a renewal has been sent. Sending an e-cancellation of the prior prescription at the time of prescription renewal might eliminate duplicate refill requests and the associated risk of duplicate prescriptions. However, it might lead to confusion at the pharmacy as to the intent of the prescriber. Including the reason for discontinuation, such as renewal, may clarify that the new prescription is in fact replacing the prior, older prescription. In our pilot implementation, we opted not to send e-cancellations with medication renewals due to risk of confusion and because the primary solution to erroneous refill requests should occur within the pharmacy, including correct prescription linkage within the pharmacy system.

CancelRx is an important tool with the potential to decrease errors in medication prescribing and dispensing by facilitating cancellation of erroneous orders immediately upon recognition and communication of changes in medication regimens from prescribers to pharmacies. This function may be complementary with other health information technology (IT) tools to reduce prescribing errors, such as indications-based prescribing[34] [35] or to facilitate communication between pharmacies and prescribers, such as RxChange, which allows pharmacists to request changes of e-prescriptions from prescribers.[36] However, how e-cancellation interacts with these and other functions of e-prescribing within the EHR has yet to be fully determined. CancelRx should be implemented in the context of robust medication reconciliation to reduce the risk of inadvertent cancellation of necessary medications, and questions remain about optimal implementation and workflows. Further understanding of the information needs of key stakeholders in e-prescribing and e-cancellation, including patients, prescribers, and pharmacy staff, could inform the development of future health IT standards and might improve care coordination around medication changes.

Our study benefited from a multidisciplinary collaboration between prescribers, pharmacy staff, health IT experts, and patient safety researchers to systematically and rigorously examine the entire process of e-cancellation. Although this study is limited to a single EHR and pharmacy management software, over 200 million patients have records with this EHR vendor,[27] [37] illustrating the significance of the issues we identified in this study. Larger scale evaluation is needed to determine how often CancelRx messaging results in successful medication discontinuation at pharmacies and to evaluate its impact on medication errors and ADEs. In addition, future research is needed to examine the performance of CancelRx implementation in other EHRs and pharmacy management systems.

Conclusion

Electronic cancellation messaging, or CancelRx, is important functionality that has the potential to decrease adverse events due to errors in both medication prescribing and dispensing. However, changes in implementation in our EHR and pharmacy software and enhancements to the CancelRx standard are needed to maximize safety and usability. Further studies are needed to evaluate the impact of e-cancellation on medication safety.

Clinical Relevance Statement

Safe medication discontinuation in ambulatory care is a complex process that entails coordination among multiple stakeholders, including prescribers, pharmacy staff, and patients. Electronic cancellation messaging, or CancelRx, is an important tool that allows prescribers to communicate discontinuation of a medication to the dispensing pharmacy. CancelRx has significant potential to decrease adverse events due to medication errors, but the functionality in our test environment was limited to medications e-prescribed in the local EHR to participating pharmacies.

Multiple Choice Questions

1. Discontinuation of which of the following resulted in an e-cancellation message in the pilot?

   • Medications reconciled from outside sources, patient reported medications, printed prescriptions, and e-prescribed medications.

   • Prescriptions generated from within the EHR (printed prescriptions and e-prescribed medications).

   • E-prescribed medications only.

   • None of the above.

   Correct Answer: The correct answer is option c. In our test environment, the EHR generated an e-cancellation message when a medication was discontinued if the original medication order was an e-prescription. If the receiving pharmacy did not accept e-cancellation transactions, the discontinuing prescriber received an in-basket message indicating that the e-cancellation had failed. No e-cancellation messages were generated for medications on the EHR medication list that were reconciled from outside sources, patient reported, or prescribed within the EHR but printed.

2. Which of the following was not identified as an area for improvement of CancelRx implementation?

   • Increased visibility of the e-cancellation transaction in the EHR.

   • Decision support for prescribers.

   • Decision support for pharmacy staff.

   • Greater ability to implement CancelRx incrementally in large institutions.

   Correct Answer: The correct answer is option b. In our test environment, prescribers did not receive visible feedback on whether an e-cancellation was successfully sent, and if sent the outcome (e.g., successfully removed from the patients' active prescriptions). Instead, discontinuing prescribers received in-basket messages to communicate transaction outcomes, which required a separate workflow. Given the limitation that some but not all medication discontinuations in the EHR trigger a cancellation and with limited pharmacy adoption of CancelRx, it is critical for prescribers to have clearly visible, real-time feedback to allow them to take timely action if an alternative method of communication to the pharmacy is needed.

   Most e-cancellation messages were automatically matched to the appropriate prescription by the pharmacy management software. However, some e-cancellation and e-prescription messages needed to be manually matched by pharmacy staff. Decision support to assist in identification of the correct e-prescription could reduce the risk of error in this process.

   Finally, incremental implementation within our test environment was technically challenging and created unintuitive restrictions on the CancelRx functionality. Improving the ability to conduct controlled, incremental implementation of e-cancellation would allow for safer implementation, particularly in larger health systems where medication reconciliation workflows vary across clinical settings (e.g., inpatient vs. ambulatory).

Conflict of Interest

None declared.

Acknowledgements

We would like to thank Matthew Lengel, Pharm.D., and Kisten Nolan, RN, for their leadership within the Bartlett pharmacy and practice, and Jim Greene, MJ McMullen, and Shahriar Ahmed from Surescripts for their assistance in the development of the end-to-end test environment and provision of data on Johns Hopkins CancelRx transactions.

Protection of Human and Animal Subjects

This study did not involve the use of clinical data requiring human subjects review.

• References

• 1 Gandhi TK, Weingart SN, Borus J. , et al. Adverse drug events in ambulatory care. N Engl J Med 2003; 348 (16) 1556-1564
• 2 Sarkar U, López A, Maselli JH, Gonzales R. Adverse drug events in U.S. adult ambulatory medical care. Health Serv Res 2011; 46 (05) 1517-1533
• 3 Taché SV, Sönnichsen A, Ashcroft DM. Prevalence of adverse drug events in ambulatory care: a systematic review. Ann Pharmacother 2011; 45 (7–8): 977-989
• 4 Panesar SS, deSilva D, Carson-Stevens A. , et al. How safe is primary care? A systematic review. BMJ Qual Saf 2016; 25 (07) 544-553
• 5 Thomsen LA, Winterstein AG, Søndergaard B, Haugbølle LS, Melander A. Systematic review of the incidence and characteristics of preventable adverse drug events in ambulatory care. Ann Pharmacother 2007; 41 (09) 1411-1426
• 6 Ranji SR, Rennke S, Wachter RM. Computerised provider order entry combined with clinical decision support systems to improve medication safety: a narrative review. BMJ Qual Saf 2014; 23 (09) 773-780
• 7 Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008; 15 (05) 585-600
• 8 Koppel R, Metlay JP, Cohen A. , et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; 293 (10) 1197-1203
• 9 Ash JS, Sittig DF, Dykstra RH, Guappone K, Carpenter JD, Seshadri V. Categorizing the unintended sociotechnical consequences of computerized provider order entry. Int J Med Inform 2007; 76 (Suppl. 01) S21-S27
• 10 Office of the National Coordinator for Health Information Technology. Health information technology adverse event reporting analysis of two databases. Washington, DC: 2014. . Available at: https://www.ecri.org/resource-center/center-for-health-it-safety-and-innovation . Accessed April 17, 2019
• 11 Schiff GD, Amato MG, Eguale T. , et al. Computerised physician order entry-related medication errors: analysis of reported errors and vulnerability testing of current systems. BMJ Qual Saf 2015; 24 (04) 264-271
• 12 Schiff GD, Hickman TT, Volk LA, Bates DW, Wright A. Computerised prescribing for safer medication ordering: still a work in progress. BMJ Qual Saf 2016; 25 (05) 315-319
• 13 Brown CL, Mulcaster HL, Triffitt KL. , et al. A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care. J Am Med Inform Assoc 2017; 24 (02) 432-440
• 14 Amato MG, Salazar A, Hickman TT. , et al. Computerized prescriber order entry-related patient safety reports: analysis of 2522 medication errors. J Am Med Inform Assoc 2017; 24 (02) 316-322
• 15 Allen AS, Sequist TD. Pharmacy dispensing of electronically discontinued medications. Ann Intern Med 2012; 157 (10) 700-705
• 16 Yang Y, Ward-Charlerie S, Kashyap N, DeMayo R, Agresta T, Green J. Analysis of medication therapy discontinuation orders in new electronic prescriptions and opportunities for implementing CancelRx. J Am Med Inform Assoc 2018; 25 (11) 1516-1523
• 17 Copi EJ, Kelley LR, Fisher KK. Evaluation of the frequency of dispensing electronically discontinued medications and associated outcomes. J Am Pharm Assoc (2003) 2018; 58 (4S): S46-S50
• 18 U.S. Food and Drug Administration. Computerized prescriber order entry medication safety (CPOEMS): Uncovering and learning from issues and errors. Sliver Spring, MD: 2015. . Available at: https://www.fda.gov/drugs/medication-errors-related-cder-regulated-drug-products/computerized-prescriber-order-entry-medication-safety-cpoems . Accessed April 17, 2019
• 19 Lourenco LM, Bursua A, Groo VL. Automatic errors: a case series on the errors inherent in electronic prescribing. J Gen Intern Med 2016; 31 (07) 808-811
• 20 Schiff G, Mirica MM, Dhavle AA, Galanter WL, Lambert B, Wright A. A prescription for enhancing electronic prescribing safety. Health Aff (Millwood) 2018; 37 (11) 1877-1883
• 21 Fischer S, Rose A. Responsible e-prescribing needs e-discontinuation. JAMA 2017; 317 (05) 469-470
• 22 Pitts SI, Maruthur NM, Luu NP. , et al. Implementing the comprehensive unit-based safety program (CUSP) to improve patient safety in an academic primary care practice. Jt Comm J Qual Patient Saf 2017; 43 (11) 591-597
• 23 Agency for Healthcare Research and Quality. The CUSP Method. Secondary The CUSP Method March 2018 2012 . Available at: https://www.ahrq.gov/professionals/education/curriculum-tools/cusptoolkit/index.html . Accessed April 17, 2019
• 24 Institute for Safe Medical Practices. ISMP list of high-alert medications in community/ambulatory healthcare. Secondary ISMP list of high-alert medications in community/ambulatory healthcare 2011 . Available at: https://www.ismp.org/sites/default/files/attachments/2017-11/highAlert-community.pdf . Accessed April 17, 2019
• 25 Holden RJ, Carayon P, Gurses AP. , et al. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics 2013; 56 (11) 1669-1686
• 26 Carayon P, Schoofs Hundt A, Karsh BT. , et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006; 15 (Suppl. 01) i50-i58
• 27 Institute of Medicine. Health IT and Patient Safety: Building Safer Systems for Better Care. Washington, DC: The National Academies Press; 2011
• 28 Office of the National Coordinator for Health Information Technology. Safer guides: secondary safer guides. April 4, 2018 . Available at: https://www.healthit.gov/topic/safety/safer-guides . Accessed April 17, 2019
• 29 Walker JM, Carayon P, Leveson N. , et al. EHR safety: the way forward to safe and effective systems. J Am Med Inform Assoc 2008; 15 (03) 272-277
• 30 Wright A, Aaron S, Sittig DF. Testing electronic health records in the “production” environment: an essential step in the journey to a safe and effective health care system. J Am Med Inform Assoc 2017; 24 (01) 188-192
• 31 Sittig DF, Singh H. Eight rights of safe electronic health record use. JAMA 2009; 302 (10) 1111-1113
• 32 Sittig DF, Classen DC. Safe electronic health record use requires a comprehensive monitoring and evaluation framework. JAMA 2010; 303 (05) 450-451
• 33 Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004; 82 (04) 581-629
• 34 Schiff GD, Seoane-Vazquez E, Wright A. Incorporating indications into medication ordering--time to enter the age of reason. N Engl J Med 2016; 375 (04) 306-309
• 35 Kron K, Myers S, Volk L. , et al. Incorporating medication indications into the prescribing process. Am J Health Syst Pharm 2018; 75 (11) 774-783
• 36 Surescripts. RxChange FAQs, 2016 . Available at: https://surescripts.com/docs/default-source/PressRelease-Library/rxchange-faqs.pdf . Accessed April 17, 2019
• 37 Epic Systems Corporation. About us. 2018 . Available at: https://www.epic.com/about . Accessed April 17, 2019