Using EHR Data to Detect Prescribing Errors in Rapidly Discontinued Medication Orders

Abstract

Background Previous research developed a new method for locating prescribing errors in rapidly discontinued electronic medication orders. Although effective, the prospective design of that research hinders its feasibility for regular use.

Objectives Our objectives were to assess a method to retrospectively detect prescribing errors, to characterize the identified errors, and to identify potential improvement opportunities.

Methods Electronically submitted medication orders from 28 randomly selected days that were discontinued within 120 minutes of submission were reviewed and categorized as most likely errors, nonerrors, or not enough information to determine status. Identified errors were evaluated by amount of time elapsed from original submission to discontinuation, error type, staff position, and potential clinical significance. Pearson's chi-square test was used to compare rates of errors across prescriber types.

Results In all, 147 errors were identified in 305 medication orders. The method was most effective for orders that were discontinued within 90 minutes. Duplicate orders were most common; physicians in training had the highest error rate (p < 0.001), and 24 errors were potentially clinically significant. None of the errors were voluntarily reported.

Conclusion It is possible to identify prescribing errors in rapidly discontinued medication orders by using retrospective methods that do not require interrupting prescribers to discuss order details. Future research could validate our methods in different clinical settings. Regular use of this measure could help determine the causes of prescribing errors, track performance, and identify and evaluate interventions to improve prescribing systems and processes.

Keywords

adverse drug event - medication errors - patient safety - electronic health records - error management and prevention

Background and Significance

Electronic health record (EHR) systems can be equipped with features to prevent prescribing errors. Forcing functions like order sets and sentences are designed to prevent prescribing errors at the point of initiating medication orders, and active interruptive clinical decision support (CDS) can prevent errors from being submitted or from reaching the patient if interruptive alerts occur in verification processes.[1] Despite these features and the rapid growth of EHR systems, prescribing errors are still prevalent and have the potential to result in patient harm.[2] [3] However, prescribing errors can be difficult to detect, as they are often not voluntarily reported unless they result in harm. The use of trigger tools, which are defined as “a data element present within the health record which may represent the presence of an adverse drug event (ADE) or error which has occurred,[4]” is considered one of the most effective methods of adverse event detection,[4] [5] [6] [7] identifying events not detected through other techniques. Using an EHR system improves the efficiency of the trigger tool technique by enabling the selective query of data elements that constitute a trigger (e.g., selected medications and laboratory values).

While triggers are traditionally designed to detect patient harm, the information gathered from the investigations of near-miss safety events can also be significant in improving patient safety.[8] [9] [10] [11] Koppel et al revised an existing trigger, abrupt medication stop, to detect no harm, near-miss prescribing errors from rapidly discontinued medication orders.[12] In their study, Koppel et al provided substantial evidence to support their hypothesis that medication orders stopped within 120 minutes of original submission are often prescribing errors. However, their approach of having an individual dedicated to monitoring computerized prescription order entry (CPOE) systems in real time for rapidly discontinued medication orders and following up by interviewing prescribers to determine if an error occurred, has practical challenges for use as a detection method in routine practice.

Objectives

This study complements the research of Koppel et al by assessing retrospective techniques for the detection of prescribing errors in rapidly discontinued medication orders, which would allow for data to be reviewed regularly (e.g., weekly or monthly) without interrupting patient care practices. The aims of this study were (1) to develop methods for retrieving rapidly discontinued medication orders from pediatric EHR data, (2) to assess the ability to retrospectively detect prescribing errors from the retrieved EHR (similar to the application of using trigger tools to detect ADEs), and (3) to characterize the identified errors and identify practical applications of the approach for detecting prescribing errors.

Methods

Study Design

The study was reviewed and approved by the hospital's institutional review board. A random number generator was used to identify 28 days during 2012 in which medication orders from the EHR system would be queried. To address the potential of the “July effect” on errors, the random number generator was repeated until only 2 days from July were selected.[13] The first developed query identified all orders within the 28-day span, and a subsequent one extracted orders that were altered within 120 minutes of initial entry. All inpatient and outpatient medication orders within this timeframe on these days were included in the study. Inpatient orders were defined as those to be administered in the hospital, incorporating admitted inpatients and those visiting the hospital for outpatient clinical services. Outpatient orders were defined as prescription medications to be administered outside the hospital. Fields collected in the data query were medical record number, patient medical service, primary diagnosis, unique order identification number, date/time of original order entry, name and position of prescriber, medication, therapeutic class, date/time of order discontinuation, and name and position of the person who discontinued the original order.

Prescribing error was defined as “a failure in the prescription writing process that results in a wrong instruction about one or more of the normal features of a prescription” where “normal features” of a prescription would include, but not be limited to, the identity of the recipient; the identity of the drug; the formulation and dose; and the route, timing, frequency, and duration of administration.[14] The positive predictive value (PPV) of the detection method was calculated as the fraction of rapidly discontinued orders that were determined to likely be errors. This method of calculating PPV was used previously.[12]

Similar to Koppel et al, we excluded orders discontinued within 1 minute of the initial submission to avoid immediately self-corrected mistakes, typographic, or selection/drop-down menu picking errors.[12] Because medication orders can be stopped by using various CPOE functions, the following actions were included in the query: cancel, complete, delete, discontinue, and modify. Initial review indicated that data captured by the order action modify largely represented routine pharmacy tasks that do not represent prescribing errors, such as dose rounding per hospital policy or adding the appropriate diluent. Therefore, modify order actions were removed from subsequent analyses. Also, the query returned no orders attributed to the order action delete. Complete is an action often used by mid-level providers and pharmacists, it essentially has the same function as discontinue yet does not require a physician cosignature. Therefore, the final data set contained only medication orders that were canceled, completed, or discontinued within 1 to 120 minutes of original entry.

Setting

The study hospital is a tertiary referral center for children with pediatric malignancies. It does not offer emergency services or have a neonatal intensive care unit. The hospital provides inpatient and outpatient medications for patients, and all prescriptions are entered through CPOE. For inpatient and outpatient services, the hospital has clinical pharmacists who are responsible for optimizing all drug therapies, generating all intravenous nutrition orders, and providing pharmacokinetic consultation services.

The study hospital has fully implemented an EHR system with CPOE (Millennium system, Cerner Corporation, Kansas City, Missouri, United States) for all aspects of inpatient and outpatient care, including orders, documentation, laboratory, and pharmacy.[15] During the study period, the hospital had active and passive CDS systems. Interruptive CDS alerts for drug–drug interactions, drug–allergy, and dose range checking were active for the study period. For passive CDS, roughly 3,000 order sets were active and more than 30,000 order sentences were available to guide prescribing. To manage alert fatigue caused by interruptive CDS alerts, a team made proactive decisions in 2005, such as limiting drug interaction alerts to the highest level (based on the Multum content database) and disabling duplicate therapy (i.e., therapeutic class duplication) alerts.

Data Analysis

Two pharmacists independently reviewed the data by examining the queried data files and also seeking additional details from EHR patient chart records to determine whether the discontinued orders were due to a prescribing error. The orders were categorized as most likely a prescribing error, most likely not a prescribing error, or not enough information to determine whether an error occurred. Discovered errors were further characterized by clinical significance as being either potentially clinically significant or not significant through independent review by two study authors. Clinical significance was defined as likely to have caused harm if it reached the patient. Percentage agreement and Cohen's kappa were used to evaluate interrater reliability among the pharmacists reviewing the data. For the orders that were identified as likely errors, the interruptive CDS alert log files were reviewed to determine whether an alert designed to prevent the prescribing error occurred during the ordering process.

To assess the performance of the error detection method throughout the 2-hour timeframe, the data were divided into 15-minute intervals, and percentage of errors with confidence intervals was calculated for each segment. Errors discovered during the analysis were cross-referenced with records from the hospital's voluntary electronic event reporting system.

To identify potential system improvements, the data were analyzed by prescriber-type and established error categories.[16] To compare the prevalence of errors across prescriber types, error proportions were calculated by dividing the number of observed errors by the total number of orders entered during the study period (i.e., not limited to those discontinued within 120 minutes) and expressed as an error rate per 1,000 orders. Pearson's chi - square was used to compare rates of errors across prescriber types.

Results

A total of 305 medication orders (corresponding to 176 patients) that were canceled, completed, or discontinued within 120 minutes of being submitted were reviewed as potential prescribing errors. The most common primary diagnosis of the patients was acute lymphocytic leukemia (n = 73, 23.9%). Analgesics were the most common class of medication associated with rapid discontinuations (n = 46, 15.1%). The average time needed to review each order was 2 minutes. Interrater agreement across the three determination options (e.g., most likely a prescribing error, most likely not a prescribing error, and not enough information to determine whether an error occurred) was initially 65% (κ = 0.46), which corresponds to a moderate level of agreement.[17] However, the reviewers reached 100% consensus after meeting to discuss initial assessments. Disagreement among reviewers was most often because one reviewer did not initially notice a key piece of information in the health record. Once this was brought to the other reviewer's attention, consensus was reached for all orders.

Roughly half (n = 147, 48%) of the canceled, completed, or discontinued orders were determined to be most likely a prescribing error ([Table 1]). One hundred forty-three of the orders were for inpatient contexts. Most of the prescribing errors were self-corrected (121, 82.3%), pharmacists corrected 7 (4.7%) errors during their review processes, and the remaining 19 errors were corrected by clinicians other than the original prescriber (e.g., an attending physician correct an error made by a physician in training). Analysis of the rate of errors at 15-minute time points indicated that the trigger was most predictive for orders stopped within the first 90 minutes (PPV = 0.54 versus 0.21 for 91–120 minutes). Most of the identified prescribing errors occurred within 15 minutes of being submitted (n = 88, 59.9%), and more than three-fourths occurred within the first 45 minutes (n = 121, 77.6%). None of the detected errors had been reported to the hospital's voluntary electronic event reporting system.

Table 1 Reasons for discontinuation of medication orders within 120 minutes of original submission

Reason for discontinuation	0–15 min (n)	16–30 min (n)	31–45 min (n)	46–60 min (n)	61–75 min (n)	76–90 min (n)	91–105 min (n)	106–120 min (n)	Total (N)
Error	88	16	10	17	6	6	3	1	147
Nonerror	9	4	7	8	2	2	5	14	51
Undetermined	39	26	9	5	7	5	7	9	107
Error % (95% CI)	65 (55–71)	34 (22–50)	38 (21–59)	56 (38–74)	40 (17–67)	46 (20–74)	20 (5–49)	4 (0–23)	48 (43–54)

Abbreviation: CI, confidence interval.

Note: Error corresponds to cases determined to be most likely a prescribing error; Nonerror to most likely not a prescribing error; and Undetermined to not enough information to determine whether an error occurred.

Independent review of the errors to categorize them as being either potentially clinically significant or not significant yielded an initial 82.3% (κ = 0.42), agreement between the two reviewers. After meeting to discuss individual assessments, 100% consensus was achieved. In all, 15.6% (n = 24) of the errors were determined to be potentially clinically significant had they reached the patient. Included in these were significant opioid analgesic overdoses (e.g., 2.3 times hydromorphone overdose) and two underdoses (e.g., changing a pain crisis patient's medication from morphine to hydromorphone). Among the potentially clinically significant errors, the lag times between initial order submission and discontinuation were not normally distributed (Kolmogorov–Smirnov = 0.24, p < 0.001), with a median time of 10 minutes (range = 1–107). Of the potentially significant errors, 54% (n = 13) of the orders were discontinued within 15 minutes.

Of the orders that were identified as likely errors, three relevant interruptive CDS alerts occurred on three separate orders during the prescribing process, but these alerts did not prevent the error. Two of the alerts were for previously documented patient drug allergy histories (i.e., a penicillin allergy alert was documented in the chart, yet the medication was still ordered), and one alert occurred because the medication dose was out of the dosing range.

The number of errors detected in the study was compared with the distribution of overall prescriptions written by practitioner type at our hospital ([Table 2]). A total of 16,687 medication orders were submitted during the study period, which corresponded to error rates ranging from 2.6 to 15.9 errors per 1,000 orders ([Table 2]). Nurse practitioners and physician assistants made the most prescribing errors and submitted the most orders ([Table 2]). However, physicians in training had a higher error rate than did all other types of prescribers (p < 0.001). Most of the 147 identified prescribing errors were categorized as duplicate orders (n = 45, 30.6%) ([Table 3]).

Table 2 Prescribing errors by prescriber type (N = 147)

Prescriber type	% of errors	% of overall prescribing[a]	Error rate/1,000 orders (95% CI)
Nurse practitioner/Physician assistant	43	46	9.2 (7.1–11.6)
Physician in training	37	20	15.9 (11.9–20.7)[b]
Physician	16	27	5.8 (3.7–8.5)
Other (e.g., Pharmacist)	4	7	2.6 (0.5–7.5)

Abbreviation: CI, confidence interval.

^a % of institution's overall medication orders written by staff during the study period.

^b Larger error rate compared with that of other prescriber types (p < 0.001).

Table 3 Frequency of different types of prescribing errors (N = 147)

Error type	n	%
Duplicate order	45	31
Wrong route	22	15
Wrong drug	21	14
Ordered incorrectly	14	10
Wrong dosage form	9	6
Underdose	8	5
Overdose	5	3
Wrong frequency	4	3
Wrong patient	1	1
Other	18	12

Discussion

Similar to previous research, our study revealed that many rapidly discontinued medication orders are likely prescribing errors.[12] Roughly half of the reviewed orders were determined to be errors (48%) and error rates were as high as 1.6% for certain prescriber types.

The work by Koppel et al was important foundational research that supported the hypothesis that medication orders discontinued within 2 hours of original entry are often errors. However, the prospective methods of their study would require someone to regularly interrupt prescribers to discuss medication orders and to determine whether the discontinued order was due to an error, which could have a variety of detrimental effects.[18] [19] [20] Their method also requires someone to monitor the EHR system in near real time to determine whether an order is discontinued within 120 minutes. Therefore, the challenges of sustaining the prospective approach and potential patient safety risks of routinely interrupting prescribers of the may hinder the method's practicality for regular use.[18] [19] [20] Our research has demonstrated that similar results can be achieved by using automated, trigger tool–type, retrospective methods, which are potentially more efficient and pose less risks. Our results also cross-validate the method for a pediatric patient population, as the initial research was conducted in an adult setting.

Given that none of the detected errors were reported to our hospital's voluntary reporting system, our method provided information for discovering improvement opportunities that would have likely been missed otherwise. This finding compares similarly with other research that has highlighted the low reporting rates for prescribing errors.[21] [22] It also empirically reinforces previously expressed sentiments that multiple detection methods are needed to fully understand the extent of errors and adverse events in a health care setting.[4] [23] Further, only 7 of 147 (5%) of the orders were discontinued and corrected by pharmacists during the standard pharmacy order verification process. Therefore, this method appears to be finding prescribing errors that are not simply being recorded by other means such as intervention databases often maintained by pharmacists.

Examining the trend of errors over the time from original order submission revealed a drop-off in the rate of errors after 90 minutes. Approximately 55% of the prescribing orders before 90 minutes were errors versus 21% in the 91 to 120 minute segment. Also, 47% of the order discontinuations evaluated in the study occurred within 15 minutes of initially being submitted. The orders within this time period also composed most of the errors detected in the study. Furthermore, the rate of errors within the first 15 minutes was 15% higher than the rate overall, and most of the potentially significant errors also occurred within this timeframe. On the basis of our data, particular attention should be focused on medication orders canceled, completed, or discontinued within 15 minutes, yet extending this timeframe to 90 minutes would still yield reasonable rates of identified errors.

In contrast to findings in previous research that has used “abrupt medication stop” as a trigger to detect patient harm,[4] [5] [24] [25] [26] the errors discovered in our study are “near-miss” patient safety events.[27] That is, these prescribing errors did not reach patients and are distinct from ADEs that result in harm, which are the primary focus of trigger tools. Although these errors were caught and corrected before reaching the patients and only 15% of them were potentially serious, there is value in understanding these events to inform process improvement in CPOE and medication use more broadly. Moreover, near-miss safety events have been shown to occur up to 100 times more than adverse events.[21]

Other contextual details of the detected errors should reveal additional improvement opportunities. For example, potentially clinically significant errors were identified in our study, including dosing errors for opioid analgesics and chemotherapy, which are high-alert medications. This information could prompt a review of prescribing practices and policies pertaining to high-alert medications. Differences in the number of prescribing errors by prescriber type can highlight systematic performance gaps, prompting prescribing training and other interventions. Improvement efforts can also be directed by examining the most frequent types of errors, which were duplicate therapy orders in our study. An example of a direct practical application of this particular result is that it has prompted our hospital to reexamine its use of interruptive duplicate therapy CDS alerts. Because we found that three interruptive CDS alerts did not prevent prescribing errors, this event detection approach may also help better understand the effectiveness of interruptive alerts.

Preventing patient harm is the ultimate goal of eliminating prescribing errors, but such errors also place an unnecessary demand on resources. The moment a medication order is submitted, structured work cycles begin. Pharmacists begin checking orders for correctness, pharmacy technicians prepare medications, and inpatient orders are delivered and administered. Errors in prescribing that are caught and changed can result in double work and the loss of often costly medications. Thus, detecting prescribing errors can provide feedback on efficiency. Inefficiency from prescribing errors becomes particularly significant considering the potential number of errors over an entire year versus our 28-day study period. To emphasize this point, we queried all of the medication orders from our study year (2012) that would qualify as rapidly discontinued, completed, or canceled within 120 minutes. The queries returned 5,837 orders, which equates to 2,685 prescribing errors given our current result of a 48% percentage of errors. Extrapolating this rate from an initial validation data set to a larger one has been done in a similar context and used to evaluate the performance of interventions to prevent wrong patient orders.[27] [28]

Limitations and Future Directions

The initial interrater agreement for determining if the reviewed medication orders were errors was relatively low (i.e., agreement was 65%, Cohen's kappa, 0.46). Despite both reviewers researching full consensus upon discussing their determinations, this initial rate of disagreement threatens the reliability of the measure. One explanation for this disagreement is that the reviewers were not the initial prescribers and that the design of the study was retrospective. Prospective designs for detecting medication errors have advantages over retrospective ones in their ability to be certain if an order was actually a mistake and to clarify context in which an error occurred. For example, one recently published paper utilized a prospective design to study reasons prescribers discontinued medication orders. While this study yielded illuminating results, the retrospective design described on our study has the potential to be used more regularly for continuous data collection to inform improvement.[29] Our study should be repeated, with a keen interest on increasing the reliability of the measure, possibly through additional methods to selectively target critical EHR data needed to determine if an order's discontinuation was due to error. Also, given the unique focus of the study hospital, the results of our study may not be generalizable to other settings, and further research should validate this error detection method in different clinical settings.

The feasibility of reproducing this method in health care settings that do not use the Cerner EHR could also be explored. Although the average time to determine whether a rapidly discontinued medication order was an error was relatively low at roughly 2 minutes, future research could investigate ways to fully automate prescriber error identification. Additionally, we excluded the review of modify order actions because many of these were clinically appropriate actions taken by pharmacists, and it may be possible to more selectively retrieve modified orders so they could be assessed as potential prescribing errors. Although our queries did not return any orders with the order action delete, other settings may use this action more widely, which may represent prescribing errors. Future research could also use this method to test the effectiveness of interventions to reduce errors.

Conclusion

Our research demonstrated that it is possible to identify prescribing errors retrospectively from electronically submitted medication orders that were rapidly discontinued in a pediatric patient population. Given that this measure was originally developed using prospective methods that require interrupting prescribers to discuss order details, validating it using retrospective methods enables it to be used more regularly. Extending the practical utility of this error detection measure allows for gathering more critical information that can assist in determining the root causes of prescribing errors, tracking performance, and identifying and evaluating interventions to improve prescribing systems and processes.

Clinical Relevance Statement

This study demonstrated that prescribing errors can be systematically identified in electronic health record data by targeting medication orders that were rapidly discontinued. Previous research has suggested this is possible through prospective designs, but this study extended the use of this technique to retrospective methods. The identification of errors is critical to improving health care systems, and the methods described in this article may enable administrators and researchers identify errors that were previously unknown.

Multiple Choice Question

• Which method of detection is most likely to identify harmful patient safety events?

  • Direct observation

  • Voluntary event reporting

  • Manual chart reviews

  • Trigger tools

  Correct Answer: The correct answer is d. While each method of detection may identify harmful patient safety events, trigger tool's effectiveness is largely predicated on the occurrence of harm. For example, harm from an overdose of opioid medications could be detected through review of the use of naloxone. This post hoc logic is consistent through most triggers, including “rescue drugs” or abnormal laboratory values. However, our study's method was designed to detect near-miss prescribing errors that were corrected before reaching patients.

Conflict of Interest

None.

Acknowledgment

The authors thank Vani Shanker, PhD, for copyediting the manuscript.

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by the St. Jude Institutional Review Board.

• References

• 1 Reckmann MH, Westbrook JI, Koh Y, Lo C, Day RO. Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assoc 2009; 16 (05) 613-623
• 2 Slight SP, Howard R, Ghaleb M, Barber N, Franklin BD, Avery AJ. The Prevalence and Causes of Prescribing and Monitoring Errors in UK Primary Care. Proceedings of the 36th Annual Meeting of the Society-of-General-Internal-Medicine; 2013 April; Denver, Colorado: Springer; 2013:S208–S208
• 3 Westbrook JI, Reckmann M, Li L. , et al. Effects of two commercial electronic prescribing systems on prescribing error rates in hospital in-patients: a before and after study. PLoS Med 2012; 9 (01) e1001164
• 4 Classen DC, Resar R, Griffin F. , et al. ‘Global trigger tool’ shows that adverse events in hospitals may be ten times greater than previously measured. Health Aff (Millwood) 2011; 30 (04) 581-589
• 5 Kirkendall ES, Kloppenborg E, Papp J. , et al. Measuring adverse events and levels of harm in pediatric inpatients with the Global Trigger Tool. Pediatrics 2012; 130 (05) e1206-e1214
• 6 Meyer-Massetti C, Cheng CM, Schwappach DLB. , et al. Systematic review of medication safety assessment methods. Am J Health Syst Pharm 2011; 68 (03) 227-240
• 7 Sharek PJ, Horbar JD, Mason W. , et al. Adverse events in the neonatal intensive care unit: development, testing, and findings of an NICU-focused trigger tool to identify harm in North American NICUs. Pediatrics 2006; 118 (04) 1332-1340
• 8 Barach P, Small SD. Reporting and preventing medical mishaps: lessons from non-medical near miss reporting systems. BMJ 2000; 320 (7237): 759-763
• 9 Callum JL, Kaplan HS, Merkley LL. , et al. Reporting of near-miss events for transfusion medicine: improving transfusion safety. Transfusion 2001; 41 (10) 1204-1211
• 10 Grant MJC, Larsen GY. Effect of an anonymous reporting system on near-miss and harmful medical error reporting in a pediatric intensive care unit. J Nurs Care Qual 2007; 22 (03) 213-221
• 11 Lundy D, Laspina S, Kaplan H, Rabin Fastman B, Lawlor E. Seven hundred and fifty-nine (759) chances to learn: a 3-year pilot project to analyse transfusion-related near-miss events in the Republic of Ireland. Vox Sang 2007; 92 (03) 233-241
• 12 Koppel R, Leonard CE, Localio AR, Cohen A, Auten R, Strom BL. Identifying and quantifying medication errors: evaluation of rapidly discontinued medication orders submitted to a computerized physician order entry system. J Am Med Inform Assoc 2008; 15 (04) 461-465
• 13 Phillips DP, Barker GE. A July spike in fatal medication errors: a possible effect of new medical residents. J Gen Intern Med 2010; 25 (08) 774-779
• 14 Aronson JK. Medication errors: definitions and classification. Br J Clin Pharmacol 2009; 67 (06) 599-604
• 15 Hoffman JM, Baker DK, Howard SC, Laver JH, Shenep JL. Safe and successful implementation of CPOE for chemotherapy at a children's cancer center. J Natl Compr Canc Netw 2011; 9 (03) (Suppl. 03) S36-S50
• 16 Dean B, Barber N, Schachter M. What is a prescribing error?. Qual Health Care 2000; 9 (04) 232-237
• 17 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33 (01) 159-174
• 18 Trbovich PL, Griffin MC, White RE, Bourrier V, Dhaliwal D, Easty AC. The effects of interruptions on oncologists' patient assessment and medication ordering practices. J Healthc Eng 2013; 4 (01) 127-144
• 19 Weigl M, Müller A, Vincent C, Angerer P, Sevdalis N. The association of workflow interruptions and hospital doctors' workload: a prospective observational study. BMJ Qual Saf 2012; 21 (05) 399-407
• 20 Weigl M, Müller A, Zupanc A, Glaser J, Angerer P. Hospital doctors' workflow interruptions and activities: an observation study. BMJ Qual Saf 2011; 20 (06) 491-497
• 21 Aspden P, Corrigan JM, Wolcott J, Erickson SM. Patient Safety: Achieving a New Standard for Care. Washington, DC: National Academies Press; 2004
• 22 Morimoto T, Gandhi TK, Seger AC, Hsieh TC, Bates DW. Adverse drug events and medication errors: detection and classification methods. Qual Saf Health Care 2004; 13 (04) 306-314
• 23 Naessens JM, Campbell CR, Huddleston JM. , et al. A comparison of hospital adverse events identified by three widely used detection methods. Int J Qual Health Care 2009; 21 (04) 301-307
• 24 Classen DC, Lloyd RC, Provost L, Griffin FA, Resar R. Development and evaluation of the Institute for Healthcare Improvement Global Trigger Tool. J Patient Saf 2008; 4 (03) 169-177
• 25 Resar RK, Rozich JD, Classen D. Methodology and rationale for the measurement of harm with trigger tools. Qual Saf Health Care 2003; 12 (Suppl. 02) ii39-ii45
• 26 Takata GS, Mason W, Taketomo C, Logsdon T, Sharek PJ. Development, testing, and findings of a pediatric-focused trigger tool to identify medication-related harm in US children's hospitals. Pediatrics 2008; 121 (04) e927-e935
• 27 NCC MERP. Taxonomy of medication errors. Available at: http://www.nccmerp.org/sites/default/files/indexColor2001-06-12.pdf . Accessed August 31, 2017
• 28 Adelman JS, Kalkut GE, Schechter CB. , et al. Understanding and preventing wrong-patient electronic orders: a randomized controlled trial. J Am Med Inform Assoc 2013; 20 (02) 305-310
• 29 Hickman TT, Quist AJL, Salazar A. , et al. Outpatient CPOE orders discontinued due to ‘erroneous entry’: prospective survey of prescribers' explanations for errors. BMJ Qual Saf 2017; 0: 1-6