The Impact of Technology on Prescribing Errors in Pediatric Intensive Care: A Before and After Study

Abstract

Background Increased use of health information technology (HIT) has been advocated as a medication error reduction strategy. Evidence of its benefits in the pediatric setting remains limited. In 2012, electronic prescribing (ICCA, Philips, United Kingdom) and standard concentration infusions (SCIs)—facilitated by smart-pump technology—were introduced into the pediatric intensive care unit (PICU) of an Irish tertiary-care pediatric hospital.

Objective The aim of this study is to assess the impact of the new technology on the rate and severity of PICU prescribing errors and identify technology-generated errors.

Methods A retrospective, before and after study design, was employed. Medication orders were reviewed over 24 weeks distributed across four time periods: preimplementation (Epoch 1); postimplementation of SCIs (Epoch 2); immediate postimplementation of electronic prescribing (Epoch 3); and 1 year postimplementation (Epoch 4). Only orders reviewed by a clinical pharmacist were included. Prespecified definitions, multidisciplinary consensus and validated grading methods were utilized.

Results A total of 3,356 medication orders for 288 patients were included. Overall error rates were similar in Epoch 1 and 4 (10.2 vs. 9.8%; p = 0.8), but error types differed (p < 0.001). Incomplete and wrong unit errors were eradicated; duplicate orders increased. Dosing errors remained most common. A total of 27% of postimplementation errors were technology-generated. Implementation of SCIs alone was associated with significant reductions in infusion-related prescribing errors (29.0% [Epoch 1] to 14.6% [Epoch 2]; p < 0.001). Further reductions (8.4% [Epoch 4]) were identified after implementation of electronically generated infusion orders. Non-infusion error severity was unchanged (p = 0.13); fewer infusion errors reached the patient (p < 0.01). No errors causing harm were identified.

Conclusion The limitations of electronic prescribing in reducing overall prescribing errors in PICU have been demonstrated. The replacement of weight-based infusions with SCIs was associated with significant reductions in infusion prescribing errors. Technology-generated errors were common, highlighting the need for on-going research on HIT implementation in pediatric settings.

Keywords

clinical information systems - intensive and critical care - pediatrics - computer-assisted decision-making - medical order entry systems - prescriptions - medication errors - infusion pumps

Background and Significance

Patients in a pediatric intensive care unit (PICU) are at heightened risk from medication errors.[1] Error reduction strategies include the use of health information technology (HIT) interventions such as electronic prescribing or computerized provider order entry (CPOE).[2] Pediatric-specific clinical decision support (CDS) and functionality are necessary to optimize safety.[3] Evidence for the benefits of CPOE in PICU is limited. Prgomet et al's 2017 systematic review of CPOE in critical care, despite reporting an 85% overall error reduction, found no reduction on subgroup analysis of the four pediatric studies.[4] Two of these were the only PICU-based CPOE studies included in a recent systematic review of pediatric dosing errors—of which neither reported a significant change.[5] [6] [7]

In PICU, infusion errors are of particular concern due to the routine use of multiple, high-risk medications across a 100-fold (<1– >100 kg) weight range.[1] [8] Recommendations from both safety agencies and governmental bodies include the replacement of traditional individualized weight-based infusions with standard concentration infusions (SCIs), and the use of smart-pump technology.[9] [10] [11] Although various national projects to standardize infusions are ongoing,[12] [13] [14] the use of weight-based infusions and traditional infusion pumps remain common in many European pediatric and neonatal intensive care units.[14] [15] [16] Heavily reliant on mathematical calculations, with dilution and manipulation of adult dosage forms commonly required, serious risk of infusion error remains.[10] [17] [18] [19]

Technology-generated errors (TGEs) are one of the unintended consequences of HIT implementation.[20] Systematic reporting of TGEs, many of which are site- and system-specific, supports shared learning and system enhancement.[4] [21] [22] Diversity in TGE terminology is adding to the recognized difficulties in comparing medication error studies.[21] [23] IntelliSpace Critical Care and Anesthesia (ICCA, Philips, United Kingdom) is a commercially available clinical information management system, widely used in both adult and pediatric hospitals in Ireland and the United Kingdom (Personal Communication, Philips, September 2019).[24] To date little research has been conducted on this system.

Objective

We undertook a single-site study to determine the rate of PICU prescribing errors before and after the implementation of electronic prescribing and a smart-pump drug library of SCIs. Secondary objectives were to investigate the effect on error severity and to describe TGEs.

Methods

Setting

This study was conducted in a 23-bed PICU in an Irish tertiary pediatric hospital. Prior to 2012, like many hospitals in Ireland and the UK, all medications were prescribed on paper. Most infusions were prescribed using individual weight-based calculations, most commonly based on the “rule of six” mathematical equation: 6 × (body weight [kg]) = amount of drug (mg) added to 100 mL to deliver 1 μg/kg/minute at 1 mL/hour.[18] Each infusion prescription required a “statement of rate”, that is, “1 mL/hour  = X dose/weight/time” to direct pump programming and dose adjustment using traditional or “nonsmart” infusion pumps.

In May 2012, a locally devised smart-pump drug library of SCIs across four weight bands (≤5, >5– ≤ 10, >10– ≤ 20, and >20 kg) was uploaded onto “smart” syringe drivers and large-volume infusion pumps (B. Braun Space pumps, Melsungen, Germany). Most medications had a standard and a high-strength option to balance excessive infusion volumes with titratability. The delivery of SCIs via smart-pumps removed the requirement to include a “statement of rate” on infusion prescriptions. Six months later, paper prescriptions were replaced with electronic prescribing using the ICCA clinical information management system. The pediatric drug files for both the smart-pumps and ICCA were developed by local multidisciplinary teams. Clinical decision support , although limited, was optimized by the extensive use of preconfigured weight-based limits and prepopulated “standard” orders. Standard orders were configured for all SCIs; commonly used medications; and those requiring age-, indication-, or formulation-specific dosing information ([Fig. 1]). “Soft” limits triggering a color change and “hard” limits preventing order completion were set for all parameters, for example, absolute dose, dose per weight, and concentration.[25] Weight-band specific “order sets” containing up to 17 standard orders were created for several specific indications, for example, “5 to 10 kg postcardiac surgery, “ < 5 kg general admission”. Medications or doses not facilitated by the drug file are ordered as “freeform” orders; there are no limits associated with these. Dual-prescribing processes (paper plus “test” electronic orders) were in place from July 2012 until electronic prescribing “go-live” in November 2012. During this period, paper prescriptions were considered the primary order and were used for documentation of both administered doses and clinical pharmacist interventions.

Fig. 1 Samples of drop-down menus for standard orders.

A unidirectional interface enabled autopopulation of near “real-time” infusion pump data onto the nursing flowsheet. This was reliant on manual assignment by nursing staff of the pump to the corresponding infusion order. Barcode-assisted medication administration tools were not employed.

On week days, clinical pharmacists review and verify all medication orders. They routinely insert instructions onto the order to direct administration, therapeutic drug monitoring, or improve clarity. They also liaise directly with clinicians and nurses. These clinical pharmacist interventions (CPIs), a proportion of which involve identification of prescribing errors, are routinely recorded in a CPI database. A weekend pharmacy service is not available; however, orders created during the weekend are reviewed the following week. Other than electronic verification, postimplementation pharmacy review processes were unchanged.

Study Design

A retrospective, before and after study design, was employed. Data were collected for 24 study weeks, evenly divided across four time periods (Epochs; [Fig. 2]). With the exception of Epoch 3 (6 consecutive weeks immediately postimplementation), study weeks were randomized. To control for seasonal effects, study weeks were matched for Epochs 1 and 4, and where possible for Epoch 2 (May–November only). Purposive sampling of study weeks to align with biannual (January and July) rotation of PICU registrars (equivalent to residents in the United States) was not done.

Fig. 2 Study time periods.

Preliminary CPI database review and the results from similar studies suggested a baseline error rate of 5% and a postimplementation rate of 2.5%.[7] [26]

Error rates were calculated as:

It was determined a sample size of 1,080 orders (infusions and non-infusions) in each of Epochs 1, 3, and 4 would detect this error reduction with 80% power and a significance level of 5%. Post hoc analysis of Epoch 1 error rates, using the same parameters, determined 233 orders were required in Epoch 2 (infusion orders only). Working sequentially through a randomized patient list for each epoch, medication orders (paper or electronic) and the CPI database were retrospectively reviewed until the sample size was reached. Patient demographic data, including severity of illness scores on admission (PIM2), were extracted from ICCA.[27] Orders not reviewed by a clinical pharmacist were excluded.

Definitions and Grading

Two pharmacists independently categorized all CPIs and extracted those involving prescribing errors; a third pharmacist was consulted where disagreement existed. To reduce subjectivity, a commonly used prescribing error definition and a list of included error scenarios compiled from published lists of pediatric medication error scenarios were used.[28] [29] The complete list of included scenarios can be seen in [Table 1].

Table 1 List of included prescribing error scenarios

Error Summary	Error details	Reference source
Incorrect dose	Dose incorrect: incorrect adjustment for altered renal/hepatic impairment	Ghaleb et al[29] [a]
	Dose incorrect: incorrect adjustment to achieve/maintain therapeutic drug levels	Ghaleb et al[29] [a]
	Dose incorrect: other (±10% correct dose)	Local consensus
	Dose incorrect: PRN with no max daily dose (where prescribed PRN dose and interval could exceed max daily)	Local consensus, Ghaleb et al[29] [a]
	Dose incorrect: wrong standard order/age category chosen	Local consensus
	Inappropriate completion of “max dose” field (removing autofilled dose on eMAR, causing potential to administer doses outside dose/weight limits)	Howlett et al[28]
	Unintentionally prescribing a medication order for the incorrect medication (transcription error)	Local consensus
	Unintentionally prescribing a medication order for the incorrect medication (other)	Local consensus
Clarity	Medication name incorrect-freeform details unclear	Local consensus
	Writing an ambiguous prescription	Ghaleb et al[29]
	Writing illegibly	Ghaleb et al[29]
Duplication	Medication not cancelled (after change made)	Local consensus
	Prescription duplication (same medication twice)	Howlett et al[28]
	Prescribing two different medications for the same indication when only one of the medications is necessary	Local consensus
Incomplete order	Prescribing a medication order without specifying one or more elements required: medication, dose, dosage units, frequency, and route	Local consensus, Ghaleb et al[29] [a]
	Omission of the prescriber's signature	Ghaleb et al[29]
	Incomplete/ambiguous information	Local consensus, Ghaleb et al[29]
Altering order	Alteration of an existing order (electronic) resulting in incorrect supplementary instructions	Howlett et al[28]
	Alteration of a standard order from a dropdown menu resulting in incongruous supplementary instructions	Howlett et al[28]
	Not rewriting a prescription (paper) in full if a change has been made to it	Ghaleb et al[29]
Statement of rate	Writing an incorrect statement of rate (type A): expressing rate as X mL (rather than X mL/hour) = dose/weight/time	Howlett et al[28]
	Writing an incorrect statement of rate (type B): expressing rate using incorrect unit of time e.g., “per min' instead of “per hour”	Howlett et al[28]
	Writing an incorrect statement of rate: combination of both type A and type B error	Howlett et al[28]
Incorrect formulation	Prescribing a dose regimen (dose/frequency) that is not that recommended for the formulation prescribed	Ghaleb et al[29]
	Selection of an incorrect formulation (caused by failure to amend default formulation)	Howlett et al[28]
Incorrect Units	Prescribing a medication using the incorrect units	Local consensus
Incorrect Route	Prescribing a medication to be administered via the incorrect route	Local consensus
Unmeasurable	Prescribing a dose that cannot readily be administered using available dosage forms (solid dosage forms only[a])	Local consensus, Ghaleb et al[29] [a]
Interaction	Prescribing a medication without taking into account a potentially significant drug interaction	Ghaleb et al[29]
Diluent	Prescription unclear-Incomplete information (diluent)	Local consensus
	Prescribing a medication to be given by intermittent intravenous infusion in a diluent that is incompatible with the drug prescribed	Ghaleb et al[29]
Duration	Continuing a prescription for a longer duration than necessary (non-infusions)	Ghaleb et al[29] [a]
	Medication stopped/cancelled in error	Local consensus
Incorrect Conc-entration	Ordering an infusion in the wrong concentration for a patient without valid clinical rationale	Howlett et al[28]
Contra-indication	Prescribing a medication for a patient who has a specific contraindication to its use	Ghaleb et al[29]
Omission	Unintentionally not prescribing a medication for a clinical condition for which medication is indicated	Ghaleb et al[29]

Abbreviation: eMAR, electronic medication administration record.

^a Slightly amended version of the published error scenario.

A technology-generated error was defined as “an error caused by the implemented technology, which could not have occurred if the order was not electronically created or intended to be administered via the smart-pump drug library.” A paper-generated error was defined as “an error that would be, or would be likely to be, prevented by the implemented HIT”.

Errors were graded for severity of actual harm caused using the “National Coordinating Council for Medication Error Reporting and Prevention (NCCMERP) Index for categorizing medication errors”.[30] Errors that failed to reach the patient (NCCMERP B) were further assessed for potential to cause harm by a five-person multidisciplinary panel using a validated 10-point severity grading scale.[30] [31] Scores of less than 3 are considered “minor”, those between 3 and 7 “moderate” and those above 7 to be “severe.”

Statistical Analysis

Patient demographics and medication orders were described using standard descriptive statistics. STATA (Stata 13.1, StataCorp LLC, Texas, United States) was used for all analyses. Significance for all comparisons was defined as p < 0.05, with a Bonferroni correction for multiple comparisons. Differences in proportions were determined using ANOVA tests for continuous normal variables; Kruskal–Wallis's tests for nonnormal variables, with Dunn's pairwise comparison testing for post hoc analysis; and Chi-squared (or Fisher's exact where samples were ≤5) tests for categorical variables and error rates.

Ethics

The Children's Health Ireland at Crumlin Research Ethics Committee determined this study to be an audit of existing medication error records, and therefore Ethics Committee approval was not required.

Results

Medication Orders and Patient Demographics

A total of 3,356 medication orders reviewed by a pharmacist (74.9% of all orders) for 288 patients were included over the four epochs. Due to incomplete clinical pharmacy records, only 70% (752 of 1,080) of Epoch 3 sample size was included. Almost one third (30.6%) of patients were neonates or preterm infants. No significant differences between epochs were identified for any recorded demographic. However, post hoc analysis of PIM2 scores indicated Epoch 3 patients were marginally less ill (p = 0.06) and had fewer medication orders (p = 0.001). Order data and patient demographics are presented in [Table 2].

Table 2 Summary of medication orders and patient demographics

	Epoch 1	Epoch 2	Epoch 3	Epoch 4	Total	p-Value
Total orders (n)	1,606	293	993[a]	1,586	4,478
Orders reviewed (n)	1,202	246 (infusions only)	752[a]	1,156	3,356
% Reviewed	74.8%	84.0%	75.7%	72.8%	74.9%	0.001 (0.2[b])
Patients (n)	74	64	67	83	288
Age category (n, %)						0.07
Preterm	5 (6.8%)	7 (10.9%)	2 (3.0%)	7 (8.4%)	21 (7.3%)
Neonate	15 (20.3%)	20 (31.3%)	15 (22.4%)	17 (20.5%)	67 (23.3%)
1 month–1 year	35 (47.3%)	18 (28.1%)	37 (55.2%)	35 (42.2%)	125 (43.4%)
> 1 year–40 kg	17 (23.0%)	17 (26.6%)	13 (19.4%)	17 (20.5%)	64 (22.2%)
> 40 kg	2 (2.7%)	2 (3.1%)	0 (0.0%)	7 (8.4%)	11 (3.8%)
Gender (male/female)	39/35	33/31	43/24	46/37	161/127	0.4
PIM2 score (median, IQR)	2.7 (5.2)	2.2 (6.4)	2.3 (3.4)	3.6 (6.8)	2.6 (3.4)	0.10 (0.06[c])
LOS in days (median, IQR)	9 (20)	10 (22)	10 (13)	8 (26)	8 (22)	0.99
Ventilated days (median, IQR)	6 (19)	5 (10)	3 (11)	4 (16)	4 (14)	0.32

Abbreviations: IQR, interquartile range; LOS, length of stay; PIM, pediatric index of mortality.

^a Lower due to missing data.

^b Excluding Epoch 2.

^c Epoch 3 and 4 only.

Prescribing Error Rates

A CPI was recorded against 15.6% of orders (n = 439); 71.1% were categorized as prescribing errors (n = 312). Overall error rates were similar pre- (Epoch 1) and post- (Epoch 4) implementation (10.2 vs. 9.8%, p = 0.80). Error rates were significantly lower in Epoch 3 (5.3%, p-adjusted 0.02). The number of errors identified and associated error rates can be seen in [Table 3].

Table 3 “All orders' and “infusion only' errors and error rates

	Epoch 1	Epoch 2	Epoch 3	Epoch 4	p-Value (adjusted)
All orders
Orders (n)	1,202	n/a	752	1,156
Errors (n)	123	n/a	40	113
Error rate (95% CI)	10.2% (8.6–12.1%)	n/a	5.3% (3.9–7.2%)	9.8% (8.2–11.6%)	0.08 (all) 0.02 (Epoch 3 and 4) 0.99 (Epoch 1 and 4) 0.02 (Epoch 1 and 3)
Infusion orders
Orders (n)	138	246	86	214
Errors (n)	40	36	4	18
Error rate (95% CI)	29% (22.0–37.1%)	14.6% (10.7–19.6%)	4.7% (1.8–11.4%)	8.4% (5.3–12.8%)	<0.001 with Epoch 1 >0.05 exc. Epoch 1

Abbreviation: CI, confidence interval.

In Epoch 2, significant reductions in infusion-related prescribing errors were found on SCI implementation (29.0% [Epoch 1] vs. 14.6% [Epoch 2]; p < 0.001). After implementation of electronically generated infusion orders, further infusion error rate reductions identified in Epoch 4 (8.4%, p = 0.32) and Epoch 3 (4.7%, p > 0.05) failed to reach statistical significance.

Postimplementation, the types of error identified were substantially altered (p < 0.001). A comparison of errors from Epochs 1 and 4 is provided in [Table 4]. Lack of clarity, incomplete, and incorrect unit errors reduced; dosing, altered orders, and duplicate errors increased. In Epoch 1, 78% (n = 96) of errors were deemed likely to be eliminated by electronic prescribing and categorized as paper-generated errors. Higher PGE rates (97%) were identified in Epoch 2, primarily involving “statement of rate” errors. In Epochs 3 and 4, 45 and 27% of errors were technology-generated errors (TGEs), respectively. Details of Epoch 4 TGEs are presented in [Table 5].

Table 4 Summary of error types for infusions and non-infusions in the preimplementation (Epoch 1) and postimplementation (Epoch 4) periods

	Preimplementation (Epoch 1)		Postimplementation (Epoch 4)
Error category	Errors (n, % errors)	Paper-generated (n, % category)	Errors (n, % errors)	Technology-generated (n, % category)
Dose	29 (23.6)	16 (55)	48 (42.5)	6 (13)
Clarity	26 (21.1)	26 (100)	6 (5.3)	2 (33)
Duplication	10 (8.1)	–	21 (18.6)	2 (10)
Incomplete order	22 (17.9)	22 (100)	1 (0.9)	1 (100)
Altering paper/standard order	7 (5.7)	7 (100)	12 (10.6)	12 (100)
Statement of rate	15 (12.2)	15 (100)	–	–
Incorrect formulation	1 (0.8)	1 (100)	7 (6.2)	6 (86)
Incorrect route	3 (2.4)	–	3 (2.7)	–
Interaction	–	–	6 (5.3)	–
Unmeasurable	1 (0.8)	1 (100)	4 (3.5)	–
Incorrect units	3 (2.4)	3 (100)	–	–
Duration	–	–	3 (2.7)	–
Diluent	2 (1.6)	2 (100)	–	–
Other	4 (3.3)	2 (50)	2 (1.8)	2 (100)
Grand total	123 (100.0)	96 (78)	113 (100.0)	31 (27)

Table 5 Overview of Epoch 4 postimplementation technology-generated errors

Error category/medication	Order type	Detail	Errors (n)
Autoscheduling			1
Co-trimoxazole	Non-Infusion	Incorrect scheduling, resulting in Saturday/Sunday dosing being given Sunday/Monday	1
Altering existing order			3
Lansoprazole	Non-Infusion	Altered dose, causing incongruous pharmacy instructions on unmeasurable dose from original order	1
Vancomycin	Non-Infusion	Altered dose/frequency order based on TDM leaving pharmacy instructions to adjust dose	2
Altering selected standard order			9
Amiodarone	Infusion	Altered order with “suitable for peripheral line” order instructions to central line concentration	1
Ciprofloxacin	Non-Infusion	Oral formulation instructions on IV order	1
Dexamethasone	Non-Infusion	Peri-extubation standard order for nonextubation	1
Magnesium oral	Non-Infusion	IV instructions on oral formulation order	3
Paracetamol	Non-Infusion	Per rectum instructions on IV order	1
Phenytoin	Non-Infusion	IV instructions on oral formulation order	1
Sodium valproate	Non-Infusion	Tablet instructions on oral Liquid order	1
Lack of clarity			2
Miconazole gel	Non-Infusion	Dose as 1 “application”, dose in milliliters not specified	1
Sodium phosphate	Non-Infusion	Order instructions contradict frequency ordered	1
Dose			6
Chloral hydrate	Non-Infusion	Inappropriate use of max dose field while weaning	1
Diazoxide	Non-Infusion	Precision setting prevented correct dose (7.75 mg)	1
Heparin	Non-Infusion	Treatment standard order for prophylactic dose	1
Melatonin	Non-Infusion	5 × stat orders, instead of nocte regular	1
Morphine	Non-Infusion	Nonneonatal dose range for neonatal patient	2
Duplication			2
Milrinone	Infusion	Diluent changed, original order not stopped	1
Morphine	Infusion	Diluent changed, original order not stopped	1
Formulation			6
Amlodipine	Non-Infusion	Oral solution being used on tablet order	1
Chlorphenamine	Non-Infusion	Tablets being used on oral solution order	1
Glycerol	Non-Infusion	Enema selected instead of suppository	1
Heparin	Infusion	Non-ECLS heparin order selected for ECLS	1
Miconazole cream	Non-Infusion	“Oral gel” as supplementary instruction	1
Spironolactone	Non-Infusion	Oral solution being used on tablet order	1
Incomplete			1
Vancomycin	Non-Infusion	Freeform order, with no units specified on dose	1
Concentration			1
Vasopressin	Infusion	50 units/50 mL SCI chosen for 5 units/50 mL	1
Total errors			31

Abbreviations: ECLS, extra-corporeal life support; IV, intravenous; SCI, standard concentration infusion; TDM, therapeutic dose monitoring.

A total of 49 (2.6%) orders were freeform orders; none involved an infusion order. In Epoch 4, freeform orders were significantly more likely to have an error than non-freeform orders (21.9 vs. 9.2%, p = 0.01). The freeform error rate was not significantly different to that for non-freeform orders in Epoch 3. (11.8 vs. 5.1%, p = 0.2). Errors in clarity, with misspelt drug names and unclear dosing instructions, were most common (n = 7).

Postimplementation, the risk of infusion errors reduced consistently across therapeutic classes. For non-infusions, errors with antiinfective agents and drugs classed as “other” increased ([Table 6]).

Table 6 Relative error risks before (Epoch 1) and after (Epoch 4) implementation by therapeutic drug class

Therapeutic drug class	Relative risk with 95% confidence interval for infusion errors (Epoch 1 vs. Epoch 4)
	Non-Infusion	Infusions
Analgesics/Sedatives	0.61 (0.34–1.1, p = 0.09)	0.11 (0.05–0.29, p < 0.001)
Cardiovascular agents	1.12 (0.42–3.02, p = 0.82)	0.41 (0.21–0.78, p < 0.01)
Other	2.61 (1.39–4.92, p < 0.01)	0.00
Electrolytes	0.92 (0.43–1.99, p = 0.83)	n/a
Antiinfectives	2.96 (1.39–6.30, p < 0.01)	n/a

Prescribing Error Severity

A total of 97.4% of errors (n = 312) did not cause patient harm or require increased monitoring or intervention. Error severity differences were only identified for infusions, with fewer errors reaching the patient (NCCMERP C) in Epoch 4 (n = 7) than Epoch 1 (n = 28), p < 0.01; no infusion errors caused harm ([Fig. 3]). Potential harm from NCCMERP B errors was the same in all epochs (mean = 5; standard deviation = 1.3). Seven of eight errors which required intervention to prevent harm were identified on electronic orders. None were TGEs and 75% (n = 6) involved incorrect doses of nephrotoxic medications.

Fig. 3 Severity of errors pre- (Epoch 1) and postimplementation (Epoch 4).

Discussion

Error Rates

Overall prescribing error rates (10%) remained unchanged on implementation of electronic prescribing and SCIs into a PICU. Although other studies have identified similar error rates, comparing evidence on the impact of HIT in different settings is challenging.[32] [33] Unfortunately, studies specific to both PICU and either electronic prescribing or CPOE are scarce.[2] [4] [5] [34] Reported error rates are strongly influenced by both study definitions and methodologies.[35] Hospital electronic prescribing/CPOE systems are diverse, with local implementation and system maintenance strongly affecting performance.[21] [36] Some studies have reported significant error reductions in PICU: both Potts et al and Kadmon et al report overall error rate reductions of 39.1 to 1.6%, and 8.2 to 1.4%, respectively on CPOE implementation.[6] [37] Substantial differences in system functionality—as discussed below—are likely to explain the contrast between the results and our findings.

Only one previous study has evaluated the impact of the same clinical information management system on medication errors in PICU. Warrick et al conducted a small study (4-day time periods) after implementation of ICIP (an earlier version of ICCA) into a UK PICU.[7] Reporting pre- and postimplementation rates similar to ours of 8.8 and 4.6% respectively; they were also unable to demonstrate significant error rate reductions.

Considering the high-risk nature of PICU infusions, the significant reduction (p < 0.001) in infusion-related prescribing errors is an important finding. The magnitude of the error reductions achieved by replacing weight-based infusions with hand-written SCIs is in line with similar studies.[19] The safety benefits of this relatively low-cost intervention may be more pronounced in nonspecialist settings, where staff are less familiar with pediatric infusions. Combining SCIs and electronic orders did not provide further benefits in Epochs 3 and 4. Increased duplicates and failure to power specifically for infusion orders may have contributed to the failure to reach statistical significance. Synergistic effects of these interventions have previously been identified.[38] [39] Some errors triggered reconfiguration of SCI standard orders, for example: broadening dinoprostone dose ranges and creation of specific neonatal and nonneonatal morphine orders. Post hoc analysis of Epoch 4 infusion error rates, where those errors prevented by the reconfiguration are excluded, produces a further error rate reduction (5.1%, p = 0.01); this highlights both the increased safety and flexibility of electronic SCI orders, but also the difficulty in assessing continually evolving HIT interventions.

Error Types

Elimination of unclear and incomplete orders is a commonly recognized benefit of electronic prescribing systems.[4] [40] Reducing errors in the complex pediatric setting, which requires robust dose-range checking and CDS, is more difficult.[3] [5] Warrick et al also reported a higher rate of dosing errors after implementation of an earlier version of ICCA.[7] Potts et al, despite reporting overall error reductions (39.1–1.6%) using the WizOrder CPOE system (a precursor to Horizon Expert), found no reduction in dosing errors.[6] Walsh et al proposed the limited ability of their commercial CPOE system (Sunrise Clinical Manager System, Eclipsys) to deal with pediatric weight-based dosing contributed to lack of dosing error reductions.[41]

The importance of pediatric CDS and ongoing system maintenance is highlighted by Kadmon et al. Using MetaVision's iMDsoft CPOE system—which in contrast to ICCA— appears to provide CDS for dosing frequency and renal/hepatic failure adjustments, they reported overall medication errors reductions (8.2–1.4%) in an initial study reported in 2009.[37] Eight years later, having reviewed their system due to increasing error rates, further error reductions were identified on addition of weight-based dosing limits and default doses.[36]

In ICCA, the inability to set frequency limits and lack of clinical parameter or drug level alerts are likely contributors to the increased antiinfective prescribing error risk identified in Epoch 4 (relative risk: 2.96, 95% confidence interval: 1.39–6.3). Other possible explanations include: mis-selection from standard order dropdown menus, transcribing orders on admission to PICU, obligatory formulation selection, and poor dose-rounding functionality.

The increase in duplicate orders (8.1% [Epoch 1] to 18.6% [Epoch 4]) is notable and a commonly reported unintended consequence of electronic systems.[42] [43] [44] Several contributory factors are likely, including: the need to actively discontinue individual electronic orders, where entire paper-prescription sheets are easily cancelled; the ability to view/alter medication orders from multiple screens; the need for temporary duplicate infusion orders to ensure continuous pump-interface data after a diluent or concentration change; order set functionality which requires individual orders to be proactively deselected—a previously identified risk factor for duplicate orders.[45] [46] In addition to the risks of missed or extra doses, the risk of incorrect manual pump-order assignment with duplicate infusion orders is a novel TGE.[28] [47]

Technology-Generated Errors

The frequency of TGEs (33%) mirrors many studies, but direct comparison is difficult due to disparate systems, levels of CDS, and local customization. Heterogeneity of TGEs definitions is also problematic, with differences in: inclusion criteria, for example, duplicate orders; and terminology, for example, “CPOE-related incidents” or “system-related errors” and use of terms such as “selection errors” which require knowledge of the intention of the prescriber.[20] [22] [42] A recent systematic review by Korb-Savoldelli et al reported that 6.1 to 77.7% of prescription errors were CPOE related.[20] In contrast, Potts et al reported no TGEs or duplicate orders with their commercially available CPOE system.[6]

As seen in [Table 5], alteration of both existing orders and standard orders, resulting in incongruous information from “additional information” fields being carried forward onto new orders, were the most commonly identified TGEs. Inability to predefine fields to be cleared on order amendment is a current system-limitation. Poor differentiation of supplementary text may also have contributed. Published data on these error types are limited.[48] Westbrook et al describe similar problems with autocompleted ancillary information and edited orders with two commercial CPOE systems.[22] Singh et al identified higher rates of errors of “inconsistent communication” in prescriptions with free-text comments.[49] The use of “free-text” was involved in several “lack of clarity” TGEs in both Epochs 3 and 4; three involved instructions to either “give PRN and max 4-hourly” on 8-hourly regular orders, or “give 8-hourly PRN” on a 12-hourly order.

Autoscheduling, particularly for medications with complex scheduling, can lead to wrong time errors.[22] [50] Default system-rules, such as scheduling first doses of regular medications for the next hour, can produce wrong time errors and may require local customization. TGE identification, both during and since this study, has triggered targeted reconfigurations. For example, a single “enteral” formulation has replaced certain “tablet/capsule” and “oral liquid” options to reduce formulation errors and enable nursing staff and pharmacists to select appropriate formulations. The freeform order structure has been amended to reduce “dosing unit” errors.

Error Severity

The infrequent nature of clinically significant errors impedes interpretation of the impact of HIT on adverse events.[40] [51] The significant reduction in infusion error severity, although a positive finding, fails to adequately capture the safety benefits of SCIs. The elimination of previously common “statement of rate” errors, although deemed to be “minor”, should not be disregarded. The potential for serious adverse outcomes remains, particularly in settings less familiar with pediatric infusions. The need for more sophisticated CDS in PICU is highlighted by the higher proportion of NCCMERP D and E errors with electronic non-infusions orders. Nephrotoxic antiinfective agents, which require frequent dose adjustments based on changing clinical parameters, were most commonly involved. Although no clinically significant errors were deemed to be TGEs, suggesting ICCA had a limited role, these findings emphasize the need for vigilance on HIT implementation.

Limitations

Although a commonly used outcome measure, prescribing errors provide a limited evaluation of the broader impact of the implemented HIT.[52] [53] It is likely that the use of CPIs, a commonly used error detection method, will have underestimated the actual number of medication errors.[54] Variability in the manner and judgement of different clinical pharmacists recording interventions in different epochs is a potential source of bias. Although interrater reliability was not measured, the use of predefined error lists and robust consensus methods were employed to minimize subjectivity. In our institution, the limitations of “real-world” implementation precluded use of a control group. Hence, despite our attempts to control for seasonal effects, the impact of other temporal changes cannot be discounted. The dual-prescribing phase and the missing data from Epoch 3 impeded identification of a learning curve. Additionally, its occurrence at the end of the registrars’ 6-month rotation may have been a contributory factor; Walsh et al found error rates in PICU were twice as high at the beginning than the end of the academic year.[41] The study was underpowered, with only Epoch 2 powered specifically for infusions and sample size based on inaccurate baseline rates. Exclusion of some common paper-based errors, for example, use of “mcg” and other abbreviations, may also have underestimated baseline rates.

Generalizability may be limited by the niche nature of PICU, diversity in electronic prescribing systems, and local configuration. HIT systems continually evolve where actively managed; repeat studies are likely to produce different results.

Conclusion

This study has described the impact of electronic prescribing and standardizing pediatric infusions on prescribing errors in the high-risk PICU setting. Valuable insight into the impact of ICCA, a commonly used clinical information system, has been provided. Although reductions in overall error rates were not demonstrated, the benefits of SCIs—used in conjunction with smart-pumps—in both electronic and paper-based settings has been highlighted. These findings will support ongoing standardization projects and elimination of weight-based infusion practices. The growing body of knowledge on TGEs has been expanded, enabling ongoing system enhancements. The importance of system maintenance and the complexities and need for sophisticated CDS in the PICU setting have been highlighted. The difficulty in assessing complex and continuously evolving HIT interventions and the need for ongoing research in both general pediatric and intensive care settings has been demonstrated.

Clinical Significance Statement

The insight gained from this research has expanded the limited knowledge base on the impact of HIT in the pediatric setting, optimizing the care of critically ill pediatric patients in an era of increasing HIT use. The finding that almost 1 in 10 orders continue to have a medication error highlights the need for continued vigilance, training, and targeted system optimization. The positive findings from this study regarding the replacement of weight-based infusions with SCIs has been a key driver in an ongoing project to standardize pediatric and neonatal infusions at a national level.

Multiple Choice Questions

1. When implementing electronic prescribing the following outcome is commonly reported:

   • Medication errors are significantly reduced in all settings.

   • System-related errors are rarely identified.

   • Duplication of orders is eradicated.

   • New system-related errors are frequently introduced.

   Correct Answer: The correct answer is option d. In contrast to the adult setting, the evidence base for the benefits of electronic prescribing on reducing pediatric medication errors is limited. A recurring theme in the literature on electronic prescribing or computer provider order entry, including several recent systematic reviews, is the emergence of new or TGEs. Increased incidence of duplicate orders is one of the more commonly reported unintended consequences of these systems.

2. Recommendations for the delivery of pediatric and neonatal infusions include which of the following:

   • Individual weight-based infusions utilizing a mathematical equation known as “the rule of six.”

   • Individual weight-based infusions utilizing a mathematical equation known as “the rule of three.”

   • The use of standardized concentration infusions.

   • The use of adult infusion solutions.

   Correct Answer: The correct answer is option c. Although the traditional method for preparing pediatric infusions has been based on the “rule of six”, this method is recognized as being error prone. The US Joint Commission for Accreditation of Healthcare Organizations and safety agencies including the Institute for Safe Medication Practices have been advocating the use of standard concentration infusions for over 10 years.

Conflict of Interest

None declared.

Acknowledgments

We would like to acknowledge the National Children's Research Centre for funding this study and for providing biostatistical support. We would also like to thank Prof. Paul Gallagher and Dr. Fiona Boland from the Royal College of Surgeons in Ireland for methodological and statistical advice, Erika Brereton and Ian Dawkins, PICU data managers for assistance in data extraction and management, Maura O'Connor, medication safety officer for her expertise, and the pharmacy and PICU staff of Children's Health Ireland at Crumlin for their support.

Protection of Human and Animal Subjects

No human/animal subjects were involved in this study.

• References

• 1 Steering Committee on Quality Improvement and Management and Committee on Hospital Care. Policy statement--principles of pediatric patient safety: reducing harm due to medical care. Pediatrics 2011; 127 (06) 1199-1210
• 2 Manias E, Kinney S, Cranswick N, Williams A, Borrott N. Interventions to reduce medication errors in pediatric intensive care. Ann Pharmacother 2014; 48 (10) 1313-1331
• 3 Dufendach KR, Eichenberger JA, McPheeters ML. , et al. Core Functionality in Pediatric Electronic Health Records. Rockville, MD: Agency for Healthcare Research and Quality (US); 2015
• 4 Prgomet M, Li L, Niazkhani Z, Georgiou A, Westbrook JI. Impact of commercial computerized provider order entry (CPOE) and clinical decision support systems (CDSSs) on medication errors, length of stay, and mortality in intensive care units: a systematic review and meta-analysis. J Am Med Inform Assoc 2017; 24 (02) 413-422
• 5 Gates PJ, Meyerson SA, Baysari MT, Westbrook JI. The prevalence of dose errors among pediatric patients in hospital wards with and without health information technology: a systematic review and meta-analysis. Drug Saf 2019; 42 (01) 13-25
• 6 Potts AL, Barr FE, Gregory DF, Wright L, Patel NR. Computerized physician order entry and medication errors in a pediatric critical care unit. Pediatrics 2004; 113 (1 Pt 1): 59-63
• 7 Warrick C, Naik H, Avis S, Fletcher P, Franklin BD, Inwald D. A clinical information system reduces medication errors in pediatric intensive care. Intensive Care Med 2011; 37 (04) 691-694
• 8 Manrique-Rodríguez S, Sánchez-Galindo AC, Fernández-Llamazares CM, Calvo-Calvo MM, Carrillo-Álvarez Á, Sanjurjo-Sáez M. Safe intravenous administration in pediatrics: A 5-year pediatric intensive care unit experience with smart pumps. Med Intensiva 2016; 40 (07) 411-421
• 9 Rich DS. New JCAHO medication management standards for 2004. Am J Health Syst Pharm 2004; 61 (13) 1349-1358
• 10 Institute for Safe Medication Practices, Vermont Oxford Network. Standard concentrations of neonatal drug infusions 2011. Available at: https://www.ismp.org/recommendations/standard-concentrations-neonatal-drug-infusions . Accessed November 25, 2019
• 11 Institute for Safe Medication Practices. ISMP. 2018–2019 Targeted medication safety best practices for hospitals. Available at: https://www.ismp.org/sites/default/files/attachments/2019-01/TMSBP-for-Hospitalsv2.pdf . Accessed November 25, 2019
• 12 American Society of Health-System Pharmacists. ASHP. Standardize 4 safety. Available at: https://www.ashp.org/pharmacy-practice/standardize-4-safety-initiative . Accessed November 25, 2019
• 13 The making it safer together (MiST) pediatric patient safety collaborative. Available at: http://www.mist-collaborative.net/ . Accessed January 25, 2020
• 14 Howlett M, Curtin M, Doherty D, Gleeson P, Sheerin M, Breatnach C. Pediatric standardised concentration infusions - a national solution. Arch Dis Child 2016; 101 (09) e2
• 15 Oskarsdottir T, Harris D, Sutherland A, Wignell A, Christiansen N. A national scoping survey of standard infusions in pediatric and neonatal intensive care units in the United Kingdom. J Pharm Pharmacol 2018; 70 (10) 1324-1331
• 16 Lyons I, Furniss D, Blandford A. , et al. Errors and discrepancies in the administration of intravenous infusions: a mixed methods multihospital observational study. BMJ Qual Saf 2018; 27 (11) 892-901
• 17 Parshuram CS, Ng GY, Ho TK. , et al. Discrepancies between ordered and delivered concentrations of opiate infusions in critical care. Crit Care Med 2003; 31 (10) 2483-2487
• 18 McLeroy PA. The rule of six: calculating intravenous infusions in a pediatric crisis situation. Hosp Pharm 1994; 29 (10) 939-940
• 19 Lehmann CU, Kim GR, Gujral R, Veltri MA, Clark JS, Miller MR. Decreasing errors in pediatric continuous intravenous infusions. Pediatr Crit Care Med 2006; 7 (03) 225-230
• 20 Korb-Savoldelli V, Boussadi A, Durieux P, Sabatier B. Prevalence of computerized physician order entry systems-related medication prescription errors: a systematic review. Int J Med Inform 2018; 111: 112-122
• 21 U.S. Food and Drug Administration Office of Surveillance and Epidemiology - Centre for Drug Evaluation and Research. Computerized Prescriber Order Entry Medication Safety (CPOEMS): uncovering and learning from issues and errors. Secondary Computerized Prescriber Order Entry Medication Safety (CPOEMS): Uncovering and learning from issues and errors. Available at: http://www.fda.gov/downloads/Drugs/DrugSafety/MedicationErrors/UCM477419.pdf . Accessed January 25, 2020
• 22 Westbrook JI, Baysari MT, Li L, Burke R, Richardson KL, Day RO. The safety of electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated with two commercial systems in hospitals. J Am Med Inform Assoc 2013; 20 (06) 1159-1167
• 23 Lisby M, Nielsen LP, Brock B, Mainz J. How are medication errors defined? A systematic literature review of definitions and characteristics. Int J Qual Health Care 2010; 22 (06) 507-518
• 24 Philips Critical Care and Anesthesia. Available at: https://www.philips.ie/healthcare/product/HCNOCTN332/intellispace-critical-care-and-anesthesia . Accessed January 20, 2020
• 25 Butler E, Howlett MM. Building a drug file for a clinical information management system: grand designs or room to improve?. Arch Dis Child 2013; 98 (06) e1
• 26 Kaushal R, Bates DW, Landrigan C. , et al. Medication errors and adverse drug events in pediatric inpatients. JAMA 2001; 285 (16) 2114-2120
• 27 Slater A, Shann F, Pearson G. ; Pediatric Index of Mortality (PIM) Study Group. PIM2: a revised version of the Pediatric Index of Mortality. Intensive Care Med 2003; 29 (02) 278-285
• 28 Howlett MM, Cleary BJ, Breatnach CV. Defining electronic-prescribing and infusion-related medication errors in pediatric intensive care - a Delphi study. BMC Med Inform Decis Mak 2018; 18 (01) 130
• 29 Ghaleb MA, Barber N, Dean Franklin B, Wong ICK. What constitutes a prescribing error in pediatrics?. Qual Saf Health Care 2005; 14 (05) 352-357
• 30 National Coordinating Council for Medication Error Reporting and Prevention. NCCMERP. NCC MERP index for categorizing medication errors. Available at: http://www.nccmerp.org/sites/default/files/algorColor2001-06-12.pdf . Accessed January 25, 2020
• 31 Dean BS, Barber ND. A validated, reliable method of scoring the severity of medication errors. Am J Health Syst Pharm 1999; 56 (01) 57-62
• 32 Ghaleb MA, Barber N, Franklin BD, Wong ICK. The incidence and nature of prescribing and medication administration errors in pediatric inpatients. Arch Dis Child 2010; 95 (02) 113-118
• 33 Cimino MA, Kirschbaum MS, Brodsky L, Shaha SH. ; Child Health Accountability Initiative. Assessing medication prescribing errors in pediatric intensive care units. Pediatr Crit Care Med 2004; 5 (02) 124-132
• 34 Rinke ML, Bundy DG, Velasquez CA. , et al. Interventions to reduce pediatric medication errors: a systematic review. Pediatrics 2014; 134 (02) 338-360
• 35 Meyer-Massetti C, Cheng CM, Schwappach DL. , et al. Systematic review of medication safety assessment methods. Am J Health Syst Pharm 2011; 68 (03) 227-240
• 36 Kadmon G, Pinchover M, Weissbach A, Kogan Hazan S, Nahum E. Case Not closed: prescription errors 12 years after computerized physician order entry implementation. J Pediatr 2017; 190: 236-240.e2
• 37 Kadmon G, Bron-Harlev E, Nahum E, Schiller O, Haski G, Shonfeld T. Computerized order entry with limited decision support to prevent prescription errors in a PICU. Pediatrics 2009; 124 (03) 935-940
• 38 Hilmas E, Sowan A, Gaffoor M, Vaidya V. Implementation and evaluation of a comprehensive system to deliver pediatric continuous infusion medications with standardized concentrations. Am J Health Syst Pharm 2010; 67 (01) 58-69
• 39 Vaidya V, Sowan AK, Mills ME, Soeken K, Gaffoor M, Hilmas E. Evaluating the safety and efficiency of a CPOE system for continuous medication infusions in a pediatric ICU. AMIA Symposium 2006:1128. Available at: https://www.ncbi.nlm.nih.gov/pubmed/17238747 . Accessed March 13, 2020
• 40 National Quality Forum. Identification and prioritization of health IT patient safety measures - final report secondary identification and prioritization of health IT patient safety measures - final report 2016. Available at: https://psnet.ahrq.gov/resources/resource/29853 . Accessed January 25, 2020
• 41 Walsh KE, Landrigan CP, Adams WG. , et al. Effect of computer order entry on prevention of serious medication errors in hospitalized children. Pediatrics 2008; 121 (03) e421-e427
• 42 Amato MG, Salazar A, Hickman TT. , et al. Computerized prescriber order entry-related patient safety reports: analysis of 2522 medication errors. J Am Med Inform Assoc 2017; 24 (02) 316-322
• 43 Wetterneck TB, Walker JM, Blosky MA. , et al. Factors contributing to an increase in duplicate medication order errors after CPOE implementation. J Am Med Inform Assoc 2011; 18 (06) 774-782
• 44 Burlison JD, McDaniel RB, Baker DK. , et al. Using EHR data to detect prescribing errors in rapidly discontinued medication orders. Appl Clin Inform 2018; 9 (01) 82-88
• 45 Abraham J, Kannampallil TG, Jarman A. , et al. Reasons for computerised provider order entry (CPOE)-based inpatient medication ordering errors: an observational study of voided orders. BMJ Qual Saf 2018; 27 (04) 299-307
• 46 Walsh KE, Adams WG, Bauchner H. , et al. Medication errors related to computerized order entry for children. Pediatrics 2006; 118 (05) 1872-1879
• 47 Dornan T, Ashcroft D, Heathfield H. , et al. An in-depth investigation into causes of prescribing errors by foundation trainees in relation to their medical education: EQUIP study. London: General Medical Council; 2009: 1-215 . Available at: https://www.gmc-uk.org/-/media/documents/FINAL_Report_prevalence_and_causes_of_prescribing_errors.pdf_28935150.pdf . Accessed March 13, 2020
• 48 Tolley CL, Forde NE, Coffey KL. , et al. Factors contributing to medication errors made when using computerized order entry in pediatrics: a systematic review. J Am Med Inform Assoc 2018; 25 (05) 575-584
• 49 Singh H, Mani S, Espadas D, Petersen N, Franklin V, Petersen LA. Prescription errors and outcomes related to inconsistent information transmitted through computerized order entry: a prospective study. Arch Intern Med 2009; 169 (10) 982-989
• 50 Hoonakker PL, Carayon P, Brown RL, Cartmill RS, Wetterneck TB, Walker JM. Changes in end-user satisfaction with Computerized Provider Order Entry over time among nurses and providers in intensive care units. J Am Med Inform Assoc 2013; 20 (02) 252-259
• 51 Gates PJ, Meyerson SA, Baysari MT, Lehmann CU, Westbrook JI. Preventable adverse drug events among inpatients: a systematic review. Pediatrics 2018; 142 (03) e20180805
• 52 Seidling HM, Bates DW. Evaluating the impact of health IT on medication safety. Stud Health Technol Inform 2016; 222: 195-205
• 53 Sheikh A. Realising the potential of health information technology to enhance medication safety. BMJ Qual Saf 2020; 29 (01) 7-9
• 54 Lewis PJ, Dornan T, Taylor D, Tully MP, Wass V, Ashcroft DM. Prevalence, incidence and nature of prescribing errors in hospital inpatients: a systematic review. Drug Saf 2009; 32 (05) 379-389