Download PDF
Appl Clin Inform 2019; 10(01): 066-076
DOI: 10.1055/s-0038-1677009
Research Article
Georg Thieme Verlag KG Stuttgart · New York

Reduced Verification of Medication Alerts Increases Prescribing Errors

David Lyell
1   Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
,
Farah Magrabi
1   Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
,
Enrico Coiera
1   Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia
› Author Affiliations Funding This research was supported by a doctoral Scholarship for David Lyell provided by the HCF (Hospitals Contribution Fund of Australia Limited) Research Foundation.
Further Information

Publication History

04 September 2018

29 November 2018

Publication Date:
30 January 2019 (online)

   Permissions and Reprints

Abstract

Objective Clinicians using clinical decision support (CDS) to prescribe medications have an obligation to ensure that prescriptions are safe. One option is to verify the safety of prescriptions if there is uncertainty, for example, by using drug references. Supervisory control experiments in aviation and process control have associated errors, with reduced verification arising from overreliance on decision support. However, it is unknown whether this relationship extends to clinical decision-making. Therefore, we examine whether there is a relationship between verification behaviors and prescribing errors, with and without CDS medication alerts, and whether task complexity mediates this.

Methods A total of 120 students in the final 2 years of a medical degree prescribed medicines for patient scenarios using a simulated electronic prescribing system. CDS (correct, incorrect, and no CDS) and task complexity (low and high) were varied. Outcomes were omission (missed prescribing errors) and commission errors (accepted false-positive alerts). Verification measures were access of drug references and view time percentage of task time.

Results Failure to access references for medicines with prescribing errors increased omission errors with no CDS (high-complexity: χ 2(1) = 12.716; p < 0.001) and incorrect CDS (Fisher's exact; low-complexity: p = 0.002; high-complexity: p = 0.001). Failure to access references for false-positive alerts increased commission errors (low-complexity: χ 2(1) = 16.673, p < 0.001; high-complexity: χ 2(1) = 18.690, p < 0.001). Fewer participants accessed relevant references with incorrect CDS compared with no CDS (McNemar; low-complexity: p < 0.001; high-complexity: p < 0.001). Lower view time percentages increased omission (F(3, 361.914) = 4.498; p = 0.035) and commission errors (F(1, 346.223) = 2.712; p = 0.045). View time percentages were lower in CDS-assisted conditions compared with unassisted conditions (F(2, 335.743) = 10.443; p < 0.001).

Discussion The presence of CDS reduced verification of prescription safety. When CDS was incorrect, reduced verification was associated with increased prescribing errors.

Conclusion CDS can be incorrect, and verification provides one mechanism to detect errors. System designers need to facilitate verification without increasing workload or eliminating the benefits of correct CDS.

Keywords

automation bias - clinical decision support systems - medication alerts - cognitive load - medication errors - human–computer interaction

Authors' Contributions

David Lyell conceived this research and designed and conducted the study with guidance from and under the supervision of Enrico Coiera and Farah Magrabi. David Lyell drafted the manuscript with input from all authors. All authors provided revisions for intellectual content and have approved the final manuscript.


Protection of Human and Animal Subjects

The research was conducted in accordance with protocols approved by the Macquarie University Human Research Ethics Committee (5201401029) and the University of New South Wales Human Research Ethics Advisory Panel (2014–7-32).


Supplementary Material

 

  • References

  • 1 Thomsen LA, Winterstein AG, Søndergaard B, Haugbølle LS, Melander A. Systematic review of the incidence and characteristics of preventable adverse drug events in ambulatory care. Ann Pharmacother 2007; 41 (09) 1411-1426
    CrossrefPubMedGoogle Scholar
  • 2 Tully MP, Ashcroft DM, Dornan T, Lewis PJ, Taylor D, Wass V. The causes of and factors associated with prescribing errors in hospital inpatients: a systematic review. Drug Saf 2009; 32 (10) 819-836
    CrossrefPubMedGoogle Scholar
  • 3 Wolfstadt JI, Gurwitz JH, Field TS. , et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008; 23 (04) 451-458
    CrossrefPubMedGoogle Scholar
  • 4 Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008; 15 (05) 585-600
    CrossrefPubMedGoogle Scholar
  • 5 van Rosse F, Maat B, Rademaker CMA, van Vught AJ, Egberts AC, Bollen CW. The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review. Pediatrics 2009; 123 (04) 1184-1190
    CrossrefPubMedGoogle Scholar
  • 6 Sweidan M, Williamson M, Reeve JF. , et al. Evaluation of features to support safety and quality in general practice clinical software. BMC Med Inform Decis Mak 2011; 11 (01) 1-8
    CrossrefPubMedGoogle Scholar
  • 7 Wright A, Ai A, Ash J. , et al. Clinical decision support alert malfunctions: analysis and empirically derived taxonomy. J Am Med Inform Assoc 2018; 25 (05) 496-506
    CrossrefPubMedGoogle Scholar
  • 8 Wright A, Hickman TT, McEvoy D. , et al. Analysis of clinical decision support system malfunctions: a case series and survey. J Am Med Inform Assoc 2016; 23 (06) 1068-1076
    CrossrefPubMedGoogle Scholar
  • 9 Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8 (03) 910-923
    Article in Thieme ConnectPubMedGoogle Scholar
  • 10 van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13 (02) 138-147
    CrossrefPubMedGoogle Scholar
  • 11 Nanji KC, Slight SP, Seger DL. , et al. Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc 2014; 21 (03) 487-491
    CrossrefPubMedGoogle Scholar
  • 12 Oxford English Dictionary. verification, n. Oxford University Press; 2018
    Google Scholar
  • 13 Manzey D, Reichenbach J, Onnasch L. Human performance consequences of automated decision aids: the impact of degree of automation and system experience. J Cogn Eng Decis Mak 2012; 6 (01) 57-87
    CrossrefPubMedGoogle Scholar
  • 14 Bahner J, Huper AD, Manzey D. Misuse of automated decision aids: complacency, automation bias and the impact of training experience. Int J Hum Comput Stud 2008; 66 (09) 688-699
    CrossrefPubMedGoogle Scholar
  • 15 Bahner J, Elepfandt MF, Manzey D. Misuse of diagnostic aids in process control: the effects of automation misses on complacency and automation bias. Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting, September 22–26, 2008, New York, NY
    PubMedGoogle Scholar
  • 16 Mosier KL, Skitka LJ. Human decision makers and automated decision aids: made for each other. In: Parasuraman R, Mouloua M. , eds. Automation and Human Performance: Theory and Applications. Hillsdale, NJ: Lawrence Erlbaum Associates; 1996: 201-220
    Google Scholar
  • 17 Lyell D, Magrabi F, Raban MZ. , et al. Automation bias in electronic prescribing. BMC Med Inform Decis Mak 2017; 17 (01) 28
    CrossrefPubMedGoogle Scholar
  • 18 Mosier KL, Skitka LJ, Heers S, Burdick M. Automation bias: decision making and performance in high-tech cockpits. Int J Aviat Psychol 1997; 8 (01) 47-63
    PubMedGoogle Scholar
  • 19 Bagheri N, Jamieson GA. The impact of context-related reliability on automation failure detection and scanning behaviour. Paper presented at the IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583), October 10–13, 2004
    PubMedGoogle Scholar
  • 20 Bagheri N, Jamieson GA. Considering subjective trust and monitoring behavior in assessing automation-induced “complacency.”. In: Vincenzi DA, Mouloua M, Hancock PA. , eds. Human Performance, Situation Awareness and Automation: Current Research and Trends. Vol. 2. Mahwah, NJ: Lawrence Erlbaum Associates; 2004: 54-59
    Google Scholar
  • 21 Reichenbach J, Onnasch L, Manzey D. Misuse of automation: the impact of system experience on complacency and automation bias in interaction with automated aids. Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting, September 27 to October 1, 2010, San Francisco, CA
    PubMedGoogle Scholar
  • 22 Reichenbach J, Onnasch L, Manzey D. Human performance consequences of automated decision aids in states of sleep loss. Hum Factors 2011; 53 (06) 717-728
    CrossrefPubMedGoogle Scholar
  • 23 Lyell D, Coiera E. Automation bias and verification complexity: a systematic review. J Am Med Inform Assoc 2017; 24 (02) 423-431
    CrossrefPubMedGoogle Scholar
  • 24 Goddard K, Roudsari A, Wyatt JC. Automation bias: a systematic review of frequency, effect mediators, and mitigators. J Am Med Inform Assoc 2012; 19 (01) 121-127
    CrossrefPubMedGoogle Scholar
  • 25 Bailey NR, Scerbo MW. Automation-induced complacency for monitoring highly reliable systems: the role of task complexity, system experience, and operator trust. Theor Issues Ergon Sci 2007; 8 (04) 321-348
    CrossrefPubMedGoogle Scholar
  • 26 Povyakalo AA, Alberdi E, Strigini L, Ayton P. How to discriminate between computer-aided and computer-hindered decisions: a case study in mammography. Med Decis Making 2013; 33 (01) 98-107
    CrossrefPubMedGoogle Scholar
  • 27 Sweller J. Element interactivity and intrinsic, extraneous, and germane cognitive load. Educ Psychol Rev 2010; 22 (02) 123-138
    CrossrefPubMedGoogle Scholar
  • 28 Lyell D, Magrabi F, Coiera E. The effect of cognitive load and task complexity on automation bias in electronic prescribing. Hum Factors 2018; 60 (07) 1008-1021
    CrossrefPubMedGoogle Scholar
  • 29 De Vries TPGM, Henning RH, Hogerzeil HV, Fresle DA. Guide to Good Prescribing. Geneva: World Health Organization; 1994
    Google Scholar
  • 30 Australian Medicines Handbook Pty Ltd. Australian Medicines Handbook 2015. Available at: http://amhonline.amh.net.au/
    PubMedGoogle Scholar
  • 31 Day RO, Snowden L. Where to find information about drugs. Aust Prescr 2016; 39 (03) 88-95
    PubMedGoogle Scholar
  • 32 Sweller J. Cognitive load theory, learning difficulty, and instructional design. Learn Instr 1994; 4 (04) 295-312
    CrossrefPubMedGoogle Scholar
  • 33 Sweller J, Chandler P. Why some material is difficult to learn. Cogn Instr 1994; 12 (03) 185-233
    CrossrefPubMedGoogle Scholar
  • 34 Hoffman L, Rovine MJ. Multilevel models for the experimental psychologist: foundations and illustrative examples. Behav Res Methods 2007; 39 (01) 101-117
    CrossrefPubMedGoogle Scholar
  • 35 Tabachnick BG. Using Multivariate Statistics. In: Tabachnick BG, Fidel LS. 6th ed. Boston, MA: Pearson; 2013
    Google Scholar
  • 36 Peugh JL. A practical guide to multilevel modeling. J Sch Psychol 2010; 48 (01) 85-112
    CrossrefPubMedGoogle Scholar
  • 37 Bland JM, Altman DG. Multiple significance tests: the Bonferroni method. BMJ 1995; 310 (6973): 170
    CrossrefPubMedGoogle Scholar
  • 38 Twisk J. Applied Multilevel Analysis: A Practical Guide. Cambridge, UK: Cambridge University Press; 2006
    Google Scholar
  • 39 Hayes AF. A primer on multilevel modeling. Hum Commun Res 2006; 32 (04) 385-410
    CrossrefPubMedGoogle Scholar
  • 40 Mack A, Rock I. Inattentional Blindness. Cambridge, MA: MIT Press; 1998
    Google Scholar
  • 41 Fiske ST, Taylor SE. Social cognition. New York, NY: Random House; 1984
    Google Scholar
  • 42 Mosier KL, Skitka LJ, Dunbar M, McDonnell L. Aircrews and automation bias: the advantages of teamwork?. Int J Aviat Psychol 2001; 11 (01) 1-14
    CrossrefPubMedGoogle Scholar