Electronic Medication Management Systems: Analysis of Enhancements to Reduce Errors and Improve Workflow

Keywords

order entry systems - safety - medication errors - workflows

Background and Significance

The introduction of electronic medication management (eMM) systems in hospitals (also referred to as computerized provider order entry systems) has been transformative in health care, with research showing that implementation of eMM reduces medication errors.[1] [2] An eMM, often one component of an electronic medical record (eMR),[3] allows clinicians to prescribe and review medications, as well as reconcile and record their administration. In addition, the embedding of clinical decision support (CDS) into an eMM system provides information to users in real time on potential medication-related harms by, for example, alerting clinicians to known allergies or drug interactions.[4]

Given the complex nature of medication management in hospitals, the interaction between eMM systems, the tasks required to be performed by their users, and existing workflows can give rise to unintended consequences.[5] A key example of this is the introduction of new safety risks that were previously not possible with the use of paper records. Research has shown that new types of medication errors can occur as a direct consequence of using electronic systems, errors referred to as system-related errors.[6] [7] [8]

In a recent systematic review that synthesized evidence of the effectiveness of eMM to reduce medication error rates and associated patient harms, 12 of the 18 included studies reported the emergence of system-related errors. In the four studies quantifying these types of errors, they reported that between 1 and 35% of all medication errors were system-related.[2] Examples of system-related errors described in these papers included medication errors resulting from the incorrect selection of order components,[9] [10] [11] the failure to modify incorrect default options,[12] and misuse of system functionalities, including CDS.[13] [14]

Another systematic review providing further insight into how and why these new errors emerge identified eight key areas that contribute to eMM-related prescribing errors, such as the computer display and system configuration, unintuitive and automated task processes, and current user workflows.[15] There is now little doubt that system-related errors do not result purely from technical issues, but rather incompatibilities between system design and user factors.[16] [17] Users frequently report that eMM systems introduce additional steps to complete tasks compared with paper-based records, and identify a range of usability issues with systems, often leading clinicians to adopt workarounds.[18] [19] For example, the inflexible design of structured order templates has led clinicians to use free-text boxes to communicate prescribing information, limiting the system's ability to detect possible drug interactions and contributing to inconsistent order information, both of which can lead to significant errors.[20] [21] [22]

Objective

This research provides us with a good foundation for understanding the types and prevalence of new medication errors that arise with the use of eMM systems, but some clear evidence gaps exist. We know very little about the longitudinal effects of system use on system-related errors (i.e., whether errors change over time?),[23] and about modifications made to eMM systems to mitigate system-related errors.[24] Following the implementation of eMM, the system is continuously updated in response to the identification of glitches, errors, workflow blocks, and user feedback,[25] but to date, no research has specifically examined the changes made to eMM systems to mitigate risks and streamline clinician workflow. In this study, we aimed to (1) identify potential system-related errors or workflow blocks which were the target of eMM system updates, including the types of medications involved, and (2) describe and classify the system updates made to target these new risks.

Methods

Design and Setting

This retrospective qualitative study reviewed and classified updates made to the eMM component of a commercially available eMR (Cerner Millennium) at three acute public hospitals within a local health district (LHD) in New South Wales (NSW), Australia. The NSW State Government (Australia) guidance recommends documenting all updates made to an eMM and the rationale for the changes.[3] This study was approved by the districts' Human Research Ethics Committee.

A staged roll-out of the eMM occurred in the first hospital between November 2007 and May 2015. The other two sites introduced the eMM system hospital-wide in September 2017 and March 2019 respectively, over a 2-week period.

Information and communications technology (ICT) services are delivered by a central district-wide information management and technology division (IM&TD), as well as facility-based ICT support teams and specialist staff. For this reason, eMM system updates typically occur at a LHD level. When a clinician requests an eMM change, the application team determines what is possible, builds the change into the testing domain of the eMM, and seeks feedback from the clinician. Once the clinician approves the change, wider group approval is sought from affected stakeholders (e.g., changes to antimicrobial prescribing require consultation with the infectious diseases team) and the health informatics medical, nursing, and pharmacy teams. Once approved, users complete further testing and the change is released on the eMM system, while the ICT team prepares the monthly document detailing recent changes.

Data Collection

Documents detailing key system updates and new features in the eMR across the LHD, published from November 2015 to December 2019, were reviewed. This time period was selected as it commenced with the regular monthly updates made within the district and concluded prior to the COVID-19 pandemic. These documents are compiled by the district's IM&TD staff approximately once a month and distributed to staff via the intranet. Each document was read thoroughly and all updates relating to the medication management process were included in the analysis. Medication-related updates were excluded if they described improvements related to clinical information systems external to the three hospitals. Documents generally followed a similar format with subcomponents of the eMR highlighted by headings (e.g., eMM). However, compared with recent reports, earlier documents were less detailed and structured. Updates ranged from a single sentence, with or without an image, to a comprehensive update specifying multiple individual changes, with detailed descriptions and images of each change ([Fig. 1]).

Fig. 1 Examples of medication-related updates in the electronic medication record (eMR), detailing (A) one system change made to the options available in a drop-down list for reasons a medication was voided and (B) multiple system changes for two high-risk medications, including changing the font to red and the addition of an icon and alert.

Classification

Initially, an attempt was made to categorize medication-related updates and the reason for updates using three existing classifications,[6] [26] [27] [28] including a classification tool for health service organizations based on pioneering work by Westbrook et al[6] [12] and Magrabi et al.[16] [29] [30] [31] [32] However, when mapping eMM updates to categories, many were classified into the broad category of “problems with clinical information system functionality,” which provided limited insight into the nuances of system enhancements.

As no suitable pre-existing classification could be identified, medication-related updates were classified according to “rationale for change” ([Table 1]) and “change made to the system” ([Table 2]; see Appendix A [[Supplementary Material], available in the online version] for full classifications with definitions and examples). This classification system was iteratively developed using cases as they emerged. Specifically, an initial sample of 10 updates was independently classified by three researchers with expertise in psychology, human factors, and clinical informatics (M.K., M.B., and W.Y.Z.). Researchers met to review assigned codes, discuss disagreements, and develop the classification framework. In developing the categories, researchers ensured they described general changes and concepts that could be applied to other settings. The remaining updates were then classified by one researcher (M.K.), with all complicated or unclear updates discussed initially with the other researchers, and if still unclear, with a specialized eMM pharmacist (L.M.H.) from one of the hospital sites, to ensure consistent and credible results.

Table 1 The rationale and most frequent medication-related changes made to the system for each rationale

Rationale for change (%)[a]	Definition	Most frequent changes
Prevent error (24.3)	To directly or indirectly reduce the likelihood of a medication error occurring	• Alert/s added  • Extra information made available  • Font/background changed  • Component/s of an order sentence modified
Support “work as done” (16.4)	To ensure the system supports practices that were previously completed on paper, for example by capturing the range of possible order components and regimens used by clinicians	• Option/s added to list  • Field/s added  • Use of free text data entry broadened
Optimize workflow (22.1)	Capitalizing on the capacity of the electronic system to facilitate more efficient and streamlined workflow, including supporting decision making, providing a better overview of the patient or patient group, or reducing the number of actions required by the user	• Extra information made available  • MPage/tab added  • PowerPlan/Care Set added  • Option/s added to list
Improve documentation (8.6)	To maintain accurate and thorough records of use, for example when completing medication reconciliation	• Field/s added  • Option/s added to list  • Option/s removed from list
Improve monitoring (5.0)	To capture and monitor the use of the system	• Report added
Avoid confusion or misinterpretation (5.7)	To reduce the likelihood of users being confused about system functions, for example by improving terminology and/or phrasing	• Wording and/or phrasing modified  • Option/s removed from list  • Alert/s removed
Support the expansion of eMM use (13.6)	To enable the broadening of eMM use, for example to ensure consistency across the district when eMM use expands to additional sites or to support expanded functionality of the eMM to other patient wards	• Wording and/or phrasing modified  • PowerPlan/Care Set removed  • PowerPlan/Care Set added  • Order sentence/s added
Improve compliance with policies or guidelines (4.3)	To ensure staff are adhering to hospital-, district-, state- or nation-wide rules as determined by policies or guidelines	• Forced review  • PowerPlan/Care Set removed

Abbreviation: eMM, electronic medication management.

^a Percentages reflect the proportion of changes made for each rationale.

Table 2 A classification of updates made to an eMM specifying changes made to the system

Category	Area of change	Change made on the system	Number of changes	% of total changes[a]
Change to the visual display	Design	Font/background changed	5	3.4%
		Icon added	3	2.0%
		Order of information modified	3	2.0%
	Content	Extra information made available	11	7.5%
		Wording and/or phrasing modified	11	7.5%
Category total			33	22.4%
Change to the options available	PowerPlans/Care Sets	PowerPlan/Care Set added	9	6.1%
		PowerPlan/Care Set removed	4	2.7%
		Component/s of a PowerPlan/Care Set modified	1	0.7%
		Use of PowerPlan/Care Set broadened	1	0.7%
	Order sentences	Order sentence/s added	4	2.7%
		Order sentence/s removed	1	0.7%
		Component/s of an order sentence modified	7	4.8%
		Filter added for order sentence/s	1	0.7%
	Order form fields	Field/s added	4	2.7%
		Field/s removed	1	0.7%
		Field/s combined	1	0.7%
		Field/s modification restricted	1	0.7%
	Lists	Option/s added to list	20	13.6%
		Option/s removed from list	4	2.7%
		Use of option/s broadened	1	0.7%
	Free text data entry	Use of free text data entry broadened	3	2.0%
Category total			63	42.9%
Change to clinical decision support	Alerts	Alert/s added	7	4.8%
		Alert/s removed	7	4.8%
		Alert/s content modified	3	2.0%
		Alert/s use broadened	3	2.0%
	MPages/tabs	MPage/tab added	7	4.8%
		MPage/tab removed	2	1.4%
	Other	Task automation or calculation	3	2.0%
Category total			32	21.8%
Adding a forcing function	Forced review		5	3.4%
	Forced selection		2	1.4%
Category total			7	4.8%
Improved information transmission	Between eMM and other eMR modules		2	1.4%
	Between eMR and other HIS		2	1.4%
Category total			4	2.7%
Other	eMM use broadened		1	0.7%
	Report added		7	4.8%
Category total			8	5.4%
Total			147	100%

Abbreviations: eMM, electronic medication management; eMR, electronic medical record; HIS, health information system.

^a Percentages may not add up to their category totals due to rounding.

Results

Overview of System Updates

The sample included 43 documents with 117 updates, totaling 147 individual changes made to the eMM system over the 4-year period.

We identified between one and three reasons for each update, with a total of 140 reasons for the changes made in our sample. Eight broad categories of reasons for the changes made to the eMM system were identified in the dataset: prevent error, support “work as done,” optimize workflow, improve documentation, improve monitoring, avoid confusion or misinterpretation, support the expansion of eMM use, and improve compliance with policies or guidelines ([Table 1]). Across the timeframe (November 2015 to December 2019), the most common rationale for an update to the eMM system was to prevent medication errors (24% of all rationales). Of the 34 updates that were made to prevent errors, the addition of an alert was the most common change (13% of the changes that were made to prevent errors). For instance, an alert was added to inform prescribers of an existing active anticoagulant order when ordering a new anticoagulant, to prevent duplication and possible contraindication. Updates also frequently occurred to optimize workflow (22% of all rationales), replicate work as done on paper charts (16%), and support the expansion of eMM use (14%), either to another ward or cohort of patients in the hospital, or to another hospital site in the district. Remaining updates were made to improve documentation (9%), avoid confusion or misinterpretation (6%), improve monitoring (5%), and to improve compliance with policies or guidelines (4%). Of the 31 updates made to optimize workflow, eight updates included additional information on the screen, such as the display of relevant pathology results during prescribing. Other frequent system changes to optimize workflow included the addition of an MPage or tab to support clinical decision making, the addition of a PowerPlan or Care Set, and the addition of options to lists, specifically folders to menu lists (e.g., addition of a nurse-initiated medication folder). For example, an MPage (see definition in [Table 3]) was added to provide clinicians with a consolidated view of their patients' diabetes therapy over the last 30 days, allowing review of the trend in blood glucose and ketone levels over time, and facilitating therapeutic decisions.

Table 3 Definitions of eMM system components

PowerPlan	A set of orders that are grouped together to support a specific condition, procedure, or process. This could describe multiple phases of care and can include additional orders
Care Set	Similar to a PowerPlan, but describes a single phase of care and cannot be modified
Order sentence	A prewritten medication order with prefilled values/components
Order form field	A component of a medication order requiring a value to be inputted
Alert	A “pop-up” window notifying the user that an action or event is about to occur, providing relevant information, providing a recommendation, or warning of a potential risk
MPage/tab	A page in the eMR or web browser that displays specific data from multiple eMR sections (e.g., pathology and medications) based on certain parameters to assist in decision making

Abbreviation: eMR, electronic medical record.

Ninety-six updates reported one change, with the remaining 21 updates reporting between two and five changes. Six broad categories of changes made to the eMM system were identified in the dataset: change to the visual display, change to the options available, change to the CDS, adding a forcing function, improved information transmission, and other. As shown in [Table 2], the most common change to the system was “changes to the options available,” followed by “changes to the content on the visual display.” This former category included options added to lists, which was the most frequent subcategory of changes. The latter category included extra information made available on the screen or the wording or phrasing of text modified. Options added to lists were most frequently to support “work as done,” optimize workflow, and prevent errors. For example, “IV infusion therapy day” was added as a route of administration for antineoplastic medications, as this is regularly prescribed by clinicians. Extra information was made available on the screen primarily to optimize workflow and prevent errors, such as including the date and time of the final scheduled medication dose in the clinical display line to prevent errors resulting from the incorrect continuation of a medication regimen. Modifications to the wording or phrasing of text were most frequently implemented to avoid confusion or misinterpretation and support the expansion of eMM use.

Some updates represented modifications or successive additions to previous updates. [Fig. 2] provides examples of linked updates.

Fig. 2 Examples of updates that reflect modifications to previous updates.

Medications That Were the Focus of Updates

Approximately a third of updates (37%) related to high-risk medications or to medicines known to have an increased risk of causing significant patient harm when misused or used in error.[33] These include antimicrobials, insulin, narcotics, electrolytes, anticoagulants, and chemotherapeutic drugs.[34] For example, an antimicrobial surveillance MPage was implemented to monitor patients with one or more anti-infective drugs at any point during admission. Additionally, PowerPlans or electronic order sets were added and modified for anticoagulants, insulin, and chemotherapy to comply with local protocols. High-risk medications frequently required multiple changes. For example, updates to make hydromorphone safer included the introduction of tallman lettering with red text, the forced selection of brand name or therapeutic substitution when prescribing, and high-risk alerts for both prescribers and administrators. Although the focus of many system updates, each high-risk medication was managed differently and there did not appear to be a standard approach or set of systematic changes for high-risk medications. For example, updates to hydromorphone included those listed above, while updates for insulin included high-risk alerts combined with a diabetic patient care MPage and the forced review of blood glucose results at the point of prescribing.

Rationale for the System Changes Made Across Time

As shown in [Fig. 3], reasons for system changes appeared to vary over time. Updates to support the expansion of eMM use increased from 6% of updates in 2016 to 24% of updates in 2019. In contrast, 29 and 12% of changes were made to optimize workflow and improve documentation in 2016, respectively, but these decreased to 10 and 3% in 2019.

Fig. 3 Rationale for changes made to the eMM across the time-period, as a percentage of total rationales per year. eMM, electronic medication management.

System changes made to improve compliance with policies or guidelines occurred only in 2017 and changes to improve monitoring only in 2019. These latter updates represented the addition of reports to the eMR menu that allowed monitoring of specific elements of eMM use (e.g., medication administration by dose, date, and time).

Discussion

This study used the unique approach of reviewing and classifying eMM system updates, providing concrete examples of system changes introduced to prevent error and improve workflow. We found nearly 150 changes were made to the eMM system over a 4-year period, with most introduced to prevent medication errors and optimize workflow. Options were made available in the eMM to allow continuity of work practices from paper to the eMM. Updates also sought to capitalize on eMM functionality and provide additional support to assist in decision making and guide appropriate user action; these were not possible in a paper-based system. Although a large proportion of updates related to high-risk medications and often multiple changes were introduced in the eMM system to target high-risk medication errors, there did not appear to be a consistent approach taken to optimize high-risk medication use. Over time, with ongoing eMM use, the focus of updates shifted toward monitoring eMM system use and supporting its expansion to other locations both internally and externally.

Updates reviewed in this study most frequently targeted the prevention of medication errors. Although medication error rates have been shown to reduce after eMM implementation,[12] [35] [36] the system has also been associated with new types of errors.[6] [37] Further, the degree of improvement following eMM implementation can vary depending on context, implementation strategy, and system design.[1] [38] Therefore, fulfilling the benefits of eMM requires hospitals to develop error prevention strategies that also minimize the risk of system-related errors, with consideration of clinical and organizational needs. Of note, the introduction of an electronic alert was the most common change aimed at error prevention in our sample. However, an increased number of alerts can lead to alert fatigue, a well-recognized phenomenon,[39] where clinicians become overburdened and their ability to determine which alerts are clinically significant declines, leading to habitual overrides.[40] The importance of optimizing alerts and continually reviewing their effectiveness in preventing errors is now well recognized.[41] In our study, we found that although alerts were added, some were also modified or removed, suggesting that the local eMM team was aware of the risk of alert fatigue and its negative impacts.

We found that options were frequently added to drop-down lists and menus (e.g., adding the frequency of “every 12 hours on therapy day” to antineoplastic orders), to ensure the system supported prescribing and administration practices previously completed on paper. When adding items to lists, we recommend that sites be mindful that incorrect selection from drop-down lists is one of the most frequent system-related errors reported in the literature.[6] [9] [42] [43] Long lists of options can result in excessive scrolling and clicks, increasing the chance of selection errors.[6] [44] Irrelevant or limited options on lists encourage the use of manual entry and free-text ordering, with flow on effects like unclear or inconsistent order information, or medication orders that are unable to trigger CDS.[45] [46] These potential pitfalls highlight the importance of only including relevant list items and good design of lists. Placing frequently used items at the top of a list, rather than alphabetically, can reduce selection errors and the likelihood of picking medication names that look and sound alike.[6] [15]

The use of eMM allows relevant information to be available to users at the point of decision making, but research has shown that some system designs require users to search for pertinent information across screens and pages.[43] For example, a qualitative case study of eMM implementation at two hospitals found a reported increase in workload as a result of the time taken to search for information between systems and computer screens.[43] Good design minimizes navigation between screens and the requirement for users to remember vital information as they move between eMR pages.[47] In our sample, we found that providing extra information on the screen (e.g., displaying the date and time for the final scheduled dose during administration) was a frequently employed strategy to facilitate the streamlining of workflow and to prevent error. Further, some changes involved the consolidation and summary of pertinent clinical information into one location, easily accessible via dedicated MPages to assist in clinical decision making. Although a common approach, noninterruptive CDS may not influence decision making unless actively integrated into workflow.[48] Rather, we suggest anticipating specific patient needs by integrating frequently grouped orders into user workflows to act as a noninterruptive CDS. We found that grouping orders (e.g., PowerPlans and Care Sets) was another strategy for optimizing workflow and guiding appropriate action. By providing timely patient-specific clinical information, improvements can be seen in the quality, efficiency, and safety of medication management.[49]

Our results also demonstrate that particular attention is paid to high-risk medications when preventing errors, as a large proportion of updates related to these. Changes were often implemented simultaneously in the eMM system, and at multiple time points, typically targeting different users (e.g., prescribers and administrators) of the system. This is in line with recommendations from the Institute for Safe Medication Practices,[50] proposing that strategies for risk minimization should be multilayered and target multiple phases in the medication use process. We also found that there did not appear to be a single approach used for these medications; instead careful consideration was given to the appropriate ways to support the use of each high-risk medication. This involved understanding the specific information required for decision making, as well as the interdependencies in clinician workflows, before developing appropriate solutions. For example, the dose and frequency of insulin relies heavily on blood glucose results. In response, a diabetic MPage with a consolidated view of associated patient details, medications, and results was made available to prescribers in the eMM system, while nurses were required to acknowledge previous blood glucose results prior to the administration of insulin. In another example, prescribers were required to select a brand name when ordering hydromorphone, as it has a narrow therapeutic window requiring the correct form to be given (i.e., immediate-release or extended-release). These examples highlight the complexity of medication management and suggest that when implementing updates to reduce the risk of high-risk medication errors, careful consideration should be given to what information is necessary at each point in the medication use process.

Implementation of an eMM system is rarely district-wide, with most implementations in NSW (Australia's largest state), occurring sequentially by piloting at one site first and then expanding to others.[51] In this study, we found that expanding eMM use to other sites necessitated several system changes, particularly to the options available for selection (e.g., removing Care Sets that comply with site-specific policies), and the wording or labeling of existing orders in the form of order sentences, PowerPlans and Care Sets. This coincided with the removal of alerts that were no longer relevant, and the implementation of forcing functions, such as mandatory second signatures. These changes were implemented to minimize the likelihood of users misinterpreting system functionality and to enforce standardization across hospitals, as well as accommodate any site-specific services (e.g., chemotherapy PowerPlans available at a site that offers these services). As clinicians frequently move between sites within a district, and find variability between sites challenging to navigate,[52] we recommend ensuring consistency in wording and workflows to minimize the risk of error and the time required to learn to navigate a new system.

Additionally, monitoring of system use was facilitated by the addition of reports in 2019. Reports import selected data in a meaningful way to monitor areas of interest. These changes are likely to reflect increased vigilance with site expansion and accreditation. Once routine use of the eMM system is reached, attention can be refocused from acute system safety risks to long-term maintenance and improvement. Although knowing what and how to measure system use is difficult,[53] all efforts to improve understanding of the eMM in a specific context are valuable and essential for successful widespread use and interoperability with other information systems.

Limitations

This study is limited by the quality of the data contained in the documents reviewed, which did not include all system changes (e.g., updates to the drug catalogue) and were not always exhaustive, particularly with respect to why system changes were made. To fully understand the “why” of system changes, we plan to complement this study with a qualitative investigation of stakeholder perspectives of system-related errors and updates implemented to improve the eMM system. While our study analyzed system changes, it did not evaluate the impact of these changes on medication error rates or workflows. Despite this, our data provide valuable insights into why changes were made and expected benefits from eMM enhancements. Our analysis was conducted primarily by one researcher, but all difficult cases were reviewed by a group to ensure accurate and consistent coding. Our study was further limited by its qualitative nature and the fact that only one type of eMM system in a single LHD was assessed, and although our findings provide general understanding and lessons for those implementing or optimizing medication systems, caution should be taken when generalizing results to other hospitals or different eMM systems.

Conclusion

Following system implementation, new safety risks can emerge as a result of eMM use, including system-related errors and workflow blocks. To our knowledge, this is the first study to systematically review and categorize system updates that have been made to overcome these risks over time, providing real-life examples that can be considered and applied in other settings. We found that updates or changes to the system sought to guide user actions by refining options available in selection lists, and implementing order sentences and grouped orders. Screen displays were modified to utilize clear language with important information emphasized to reduce misunderstanding and improve decision making. Particular attention was paid to high-risk medications, which require a multilayered approach to limit the chance for error. Overall, interventions like eMM systems are likely to change over time as users become more familiar with the system and use is expanded to more sites. This research has shown that this is an ongoing process in which continual monitoring of the system is necessary to detect areas for improvement and capitalize on the benefits an electronic system can provide.

Clinical Relevance Statement

The transition from paper-based medication charts to eMM has reduced medication errors but also introduced new safety risks. Systems are continuously updated in response to these risks, and this article outlines changes made to a system to mitigate system-related errors and streamline clinician workflow. For institutions planning to implement eMM systems, it is important to recognize that these are not “set-and-forget” systems and therefore require ongoing surveillance and maintenance.

Multiple Choice Questions

• 1. What was the most common reason that changes were made to the system?

  • To support “work as done.”

  • To prevent error.

  • To optimize workflow.

  • To support the expansion of eMM use.

  Correct Answer: The correct answer is option b. Changes were made most frequently to prevent medication errors (24% of all rationales).

• 2. To minimize the risk of errors associated high-risk medications, what types of strategies can be used in electronic systems to align with the Institute for Safe Medication Practices recommendations?

  • Strategies should be standardized across hospitals.

  • Strategies should be multilayered.

  • Strategies should be integrated into workflow.

  • None of the above.

  Correct Answer: The correct answer is option b. The Institute for Safe Medication Practices proposes that strategies for risk minimization should be multilayered, combining various approaches to target specific risks.

• References

• 1 Nuckols TK, Smith-Spangler C, Morton SC. et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 2014; 3: 56
• 2 Gates PJ, Hardie RA, Raban MZ, Li L, Westbrook JI. How effective are electronic medication systems in reducing medication error rates and associated harm among hospital inpatients? A systematic review and meta-analysis. J Am Med Inform Assoc 2021; 28 (01) 167-176
• 3 NSW Therapeutic Advisory Group Inc. and eHealth NSW.. Building Sustainable Governance of Electronic Medication Management: Guiding Principles for Drug and Therapeutic Committees in NSW. Sydney: NSW TAG; 2017
• 4 Baysari MT, Westbrook J, Braithwaite J, Day RO. The role of computerized decision support in reducing errors in selecting medicines for prescription: narrative review. Drug Saf 2011; 34 (04) 289-298
• 5 Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care–an interactive sociotechnical analysis. J Am Med Inform Assoc 2007; 14 (05) 542-549
• 6 Westbrook JI, Baysari MT, Li L, Burke R, Richardson KL, Day RO. The safety of electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated with two commercial systems in hospitals. J Am Med Inform Assoc 2013; 20 (06) 1159-1167
• 7 Sittig DF, Singh H. Defining health information technology-related errors: new developments since to err is human. Arch Intern Med 2011; 171 (14) 1281-1284
• 8 Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006; 13 (05) 547-556
• 9 Armada ER, Villamañán E, López-de-Sá E. et al. Computerized physician order entry in the cardiac intensive care unit: effects on prescription errors and workflow conditions. J Crit Care 2014; 29 (02) 188-193
• 10 Mills PR, Weidmann AE, Stewart D. Hospital staff views of prescribing and discharge communication before and after electronic prescribing system implementation. Int J Clin Pharm 2017; 39 (06) 1320-1330
• 11 Warrick C, Naik H, Avis S, Fletcher P, Franklin BD, Inwald D. A clinical information system reduces medication errors in paediatric intensive care. Intensive Care Med 2011; 37 (04) 691-694
• 12 Westbrook JI, Reckmann M, Li L. et al. Effects of two commercial electronic prescribing systems on prescribing error rates in hospital in-patients: a before and after study. PLoS Med 2012; 9 (01) e1001164
• 13 Pontefract SK, Hodson J, Slee A. et al. Impact of a commercial order entry system on prescribing errors amenable to computerised decision support in the hospital setting: a prospective pre-post study. BMJ Qual Saf 2018; 27 (09) 725-736
• 14 Hernandez F, Majoul E, Montes-Palacios C. et al. An observational study of the impact of a computerized physician order entry system on the rate of medication errors in an orthopaedic surgery unit. PLoS One 2015; 10 (07) e0134101
• 15 Brown CL, Mulcaster HL, Triffitt KL. et al. A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care. J Am Med Inform Assoc 2017; 24 (02) 432-440
• 16 Magrabi F, Ong MS, Runciman W, Coiera E. An analysis of computer-related patient safety incidents to inform the development of a classification. J Am Med Inform Assoc 2010; 17 (06) 663-670
• 17 Debono D, Taylor N, Lipworth W. et al. Applying the Theoretical Domains Framework to identify barriers and targeted interventions to enhance nurses' use of electronic medication management systems in two Australian hospitals. Implement Sci 2017; 12 (01) 42
• 18 Baysari MT, Hardie RA, Lake R. et al. Longitudinal study of user experiences of a CPOE system in a pediatric hospital. Int J Med Inform 2018; 109: 5-14
• 19 Niazkhani Z, Pirnejad H, van der Sijs H, de Bont A, Aarts J. Computerized provider order entry system–does it support the inter-professional medication process? Lessons from a Dutch academic hospital. Methods Inf Med 2010; 49 (01) 20-27
• 20 Singh H, Mani S, Espadas D, Petersen N, Franklin V, Petersen LA. Prescription errors and outcomes related to inconsistent information transmitted through computerized order entry: a prospective study. Arch Intern Med 2009; 169 (10) 982-989
• 21 Slight SP, Eguale T, Amato MG. et al. The vulnerabilities of computerized physician order entry systems: a qualitative study. J Am Med Inform Assoc 2016; 23 (02) 311-316
• 22 Lichtner V, Baysari M, Gates P, Dalla-Pozza L, Westbrook JI. Medication safety incidents in paediatric oncology after electronic medication management system implementation. Eur J Cancer Care (Engl) 2019; 28 (06) e13152
• 23 Kinlay M, Zheng WY, Burke R, Juraskova I, Moles R, Baysari M. Medication errors related to computerized provider order entry systems in hospitals and how they change over time: a narrative review. Res Social Adm Pharm 2021; 17 (09) 1546-1552
• 24 Westbrook JI, Lichtner V. Why is measuring the effects of information technology on medication errors so difficult?. Lancet Digit Health 2019; 1 (08) e378-e379
• 25 Williams J, Bates DW, Sheikh A. Optimising electronic prescribing in hospitals: a scoping review protocol. BMJ Health Care Inform 2020; 27 (01) e100117
• 26 Australian Commission on Safety and Quality in Health Care. A Health IT-Related Classification System. Sydney: ACSQHC; 2019
• 27 Australian Commission on Safety and Quality in Health Care. A Medicine Incident Classification System. Sydney: ACSQHC; 2019
• 28 Amato MG, Salazar A, Hickman TT. et al. Computerized prescriber order entry-related patient safety reports: analysis of 2522 medication errors. J Am Med Inform Assoc 2017; 24 (02) 316-322
• 29 Magrabi F, Baker M, Sinha I. et al. Clinical safety of England's national programme for IT: a retrospective analysis of all reported safety events 2005 to 2011. Int J Med Inform 2015; 84 (03) 198-206
• 30 Magrabi F, Ong MS, Runciman W, Coiera E. Using FDA reports to inform a classification for health information technology safety problems. J Am Med Inform Assoc 2012; 19 (01) 45-53
• 31 Kim MO, Coiera E, Magrabi F. Problems with health information technology and their effects on care delivery and patient outcomes: a systematic review. J Am Med Inform Assoc 2017; 24 (02) 246-250
• 32 Magrabi F, Liaw ST, Arachi D, Runciman W, Coiera E, Kidd MR. Identifying patient safety problems associated with information technology in general practice: an analysis of incident reports. BMJ Qual Saf 2016; 25 (11) 870-880
• 33 Cohen MR. Medication Errors. Washington, DC: American Pharmacist Association; 2007
• 34 Clinical Excellence Commission. High-Risk Medicines Management. Sydney: NSW Health; 2020
• 35 Westbrook JI, Sunderland NS, Woods A, Raban MZ, Gates P, Li L. Changes in medication administration error rates associated with the introduction of electronic medication systems in hospitals: a multisite controlled before and after study. BMJ Health Care Inform 2020; 27 (03) e100170
• 36 Radley DC, Wasserman MR, Olsho LE, Shoemaker SJ, Spranca MD, Bradshaw B. Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems. J Am Med Inform Assoc 2013; 20 (03) 470-476
• 37 Van de Vreede M, McGrath A, de Clifford J. Review of medication errors that are new or likely to occur more frequently with electronic medication management systems. Aust Health Rev 2019; 43 (03) 276-283
• 38 Holmgren AJ, Co Z, Newmark L, Danforth M, Classen D, Bates D. Assessing the safety of electronic health records: a national longitudinal study of medication-related decision support. BMJ Qual Saf 2020; 29 (01) 52-59
• 39 Payne TH. EHR-related alert fatigue: minimal progress to date, but much more can be done. BMJ Qual Saf 2019; 28 (01) 1-2
• 40 Baysari MT, Tariq A, Day RO, Westbrook JI. Alert override as a habitual behavior - a new perspective on a persistent problem. J Am Med Inform Assoc 2017; 24 (02) 409-412
• 41 McGreevey III JD, Mallozzi CP, Perkins RM, Shelov E, Schreiber R. Reducing alert burden in electronic health records: state of the art recommendations from four health systems. Appl Clin Inform 2020; 11 (01) 1-12
• 42 Slight SP, Tolley CL, Bates DW. et al. Medication errors and adverse drug events in a UK hospital during the optimisation of electronic prescriptions: a prospective observational study. Lancet Digit Health 2019; 1 (08) e403-e412
• 43 Cresswell KM, Bates DW, Williams R. et al. Evaluation of medium-term consequences of implementing commercial computerized physician order entry and clinical decision support prescribing systems in two ‘early adopter’ hospitals. J Am Med Inform Assoc 2014; 21 (Suppl. 02) e194-e202
• 44 Schiff GD, Amato MG, Eguale T. et al. Computerised physician order entry-related medication errors: analysis of reported errors and vulnerability testing of current systems. BMJ Qual Saf 2015; 24 (04) 264-271
• 45 Odukoya OK, Chui MA. Relationship between e-prescriptions and community pharmacy workflow. J Am Pharm Assoc (2003) 2012; 52 (06) e168-e174
• 46 Khajouei R, Jaspers MW. The impact of CPOE medication systems' design aspects on usability, workflow and medication orders: a systematic review. Methods Inf Med 2010; 49 (01) 3-19
• 47 Nielsen J. Enhancing the explanatory power of usability heuristics. Paper presented at: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; April 24, 1994; Boston, Massachusetts, United States
• 48 Blecker S, Pandya R, Stork S. et al. Interruptive versus noninterruptive clinical decision support: Usability study. JMIR Human Factors 2019; 6 (02) e12469-e12469
• 49 Teich JM, Osheroff JA, Pifer EA, Sittig DF, Jenders RA, Panel CDSER. CDS Expert Review Panel. Clinical decision support in electronic prescribing: recommendations and an action plan: report of the joint clinical decision support workgroup. J Am Med Inform Assoc 2005; 12 (04) 365-376
• 50 Institute for Safe Medication Practices (ISMP). (US). Your high-alert medication list: relatively useless without associated risk-reduction strategies. 2013. Accessed 29 April 2021 at: https://www.ismp.org/resources/your-high-alert-medication-list-relatively-useless-without-associated-risk-reduction?id=45
• 51 Australian Commission on Safety and Quality in Health Care. Electronic Medication Management Systems: A Guide to Safe Implementation. 3rd edn.. Sydney: ACSQHC; 2019
• 52 Baysari MT, Raban MZ. The safety of computerised prescribing in hospitals. Aust Prescr 2019; 42 (04) 136-138
• 53 Sittig DF, Campbell E, Guappone K, Dykstra R, Ash JS. Recommendations for monitoring and evaluation of in-patient computer-based provider order entry systems: results of a Delphi survey. AMIA Annu Symp Proc 2007; 2007: 671-675