Subscribe to RSS
Usability Flaws in Medication Alerting Systems: Impact on Usage and Work System
30 June 2015
10 March 2018 (online)
Objectives: Previous research has shown that medication alerting systems face usability issues. There has been no previous attempt to systematically explore the consequences of usability flaws in such systems on users (i.e. usage problems) and work systems (i.e. negative outcomes). This paper aims at exploring and synthesizing the consequences of usability flaws in terms of usage problems and negative outcomes on the work system.
Methods: A secondary analysis of 26 papers included in a prior systematic review of the usability flaws in medication alerting was performed. Usage problems and negative outcomes were extracted and sorted. Links between usability flaws, usage problems, and negative outcomes were also analyzed.
Results: Poor usability generates a large variety of consequences. It impacts the user from a cognitive, behavioral, emotional, and attitudinal perspective. Ultimately, usability flaws have negative consequences on the workflow, the effectiveness of the technology, the medication management process, and, more importantly, patient safety. Only few complete pathways leading from usability flaws to negative outcomes were identified.
Conclusion: Usability flaws in medication alerting systems impede users, and ultimately their work system, and negatively impact patient safety. Therefore, the usability dimension may act as a hidden explanatory variable that could explain, at least partly, the (absence of) intended outcomes of new technology.
- 1 Koln L, Corrigan J, Donaldson M. editors. To err is human, building a safer health system. Washington, D.C.,: National Academic Press; 2000
- 2 Kaplan B, Harris-Salamone KD. Health IT success and failure: recommendations from literature and an AMIA workshop. J Am Med Inform Assoc 2009; May 16 (03) 291-9.
- 3 McManus J, Wood-Harper T. Understanding the sources of information systems project failure. Manag Serv 2007; 38-43.
- 4 International Standardization Organization.. Ergonomic requirements for office work with visual display terminals (VDTs) -- Part 11: Guidance on usability (Rep N° 9241-11). Geneva: International Standardization Organization; 1998
- 5 Middleton B, Bloomrosen M, Dente MA, Hash-mat B, Koppel R, Overhage JM. et al. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; Jun 20 e1 e2-e8.
- 6 Magrabi F, Ong MS, Runciman W, Coiera E. An analysis of computer-related patient safety incidents to inform the development of a classification. J Am Med Inform Assoc 2010; Nov 17 (06) 663-70.
- 7 Samaranayake NR, Cheung ST, Chui WC, Cheung BM. Technology-related medication errors in a tertiary hospital: a 5-year analysis of reported medication incidents. Int J Med Inform 2012; Dec 81 (12) 828-33.
- 8 Lavery D, Cockton G. Representing Predicted and Actual Usability Problems. In: Johnson H, Johnson P, O’Neill E. editors. Proceedings of International Workshop on Representations in Interactive Software Development. Queen Mary and Westfield College, University of London; 1997. p. 97-108.
- 9 Carayon P, Alvadaro CJ, Schoofs HA. Work system design in health care. In: Carayon P. editor. Handbook of human factors and ergonomics in health care and patient safety. Laurence Erlbaum Associates; New Jersey: 2007. p. 61-78.
- 10 Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE. et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; Mar 9 293 (10) 1197-203.
- 11 Kushniruk A, Beuscart-Zephir MC, Grzes A, Borycki E, Watbled L, Kannry J. Increasing the safety of healthcare information systems through improved procurement: toward a framework for selection of safe healthcare systems. Healthc Q 2010; Sep 13 Spec No: 53-8.
- 12 Weiner JP, Kfuri T, Chan K, Fowles JB. “e-Iatro-genesis”: the most critical unintended consequence of CPOE and other HIT. J Am Med Inform Assoc 2007; May 14 (03) 387-8.
- 13 Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006; Sep 13 (05) 547-56.
- 14 Kushniruk A, Triola M, Stein B, Borycki E, Kannry J. The relationship of usability to medical error: an evaluation of errors associated with usability problems in the use of a handheld application for prescribing medications. Stud Health Technol Inform 2004; 107 Pt 2 1073-6.
- 15 Nanji KC, Rothschild JM, Boehne JJ, Keohane CA, Ash JS, Poon EG. Unrealized potential and residual consequences of electronic prescribing on pharmacy workflow in the outpatient pharmacy. J Am Med Inform Assoc 2014; May 21 (03) 481-6.
- 16 Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998; Oct 21 280 (15) 1339-46.
- 17 Ranji SR, Rennke S, Wachter RM. Computerised provider order entry combined with clinical decision support systems to improve medication safety: a narrative review. BMJ Qual Saf 2014 Apr 12.
- 18 Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004; Mar 11 (02) 104-12.
- 19 Marcilly R, Ammenwerth E, Vasseur F, Roehrer E, Beuscart-Zephir MC. Usability flaws of medication-related alerting systems: a systematic review. J Biomed Inform 2015; Jun 55: 260-71.
- 20 Scapin DL, Bastien JMC. Ergonomic criteria for evaluating the ergonomic quality of interactive systems. Behaviour and Information Technology 1997; 6 4-5 220-31.
- 21 Sebillotte S. Action representation for home automation. -North-Holland Elsevier Science Publishers B.V.; 1990. p. 985-90.
- 22 Russ AL, Zillich AJ, McManus MS, Doebbeling BN, Saleem JJ. A human factors investigation of medication alerts: barriers to prescriber decision-making and clinical workflow. AMIA Annu Symp Proc 2009; 548-52.
- 23 Chused AE, Kuperman GJ, Stetson PD. Alert override reasons: a failure to communicate. AMIA Annu Symp Proc 2008; 111-5.
- 24 Krall MA, Sittig DF. Clinician’s assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements. Proc AMIA Symp 2002; 400-4.
- 25 Horsky J, Kaufman DR, Patel VL. Computer-based drug ordering: evaluation of interaction with a decision-support system. Stud Health Technol Inform 2004; 107 Pt 2 1063-7.
- 26 Russ AL, Saleem JJ, McManus MS, Zillich AJ, Doebbling BN. Computerized medication alerts and prescriber mental models: observing routine patient care. Proceedings of the Human Factors and Ergonomics Society Annual Meeting; 2009 p. 655-9.
- 27 Khajouei R, Peek N, Wierenga PC, Kersten MJ, Jaspers MW. Effect of predefined order sets and usability problems on efficiency of computerized medication ordering. Int J Med Inform 2010; Oct 79 (10) 690-8.
- 28 Feldstein A, Simon SR, Schneider J, Krall M, Laferriere D, Smith DH. et al. How to design computerized alerts to safe prescribing practices. Jt Comm J Qual Saf 2004; Nov 30 (11) 602-13.
- 29 Russ AL, Zillich AJ, McManus MS, Doebbeling BN, Saleem JJ. Prescribers’ interactions with medication alerts at the point of prescribing: A multi-method, in situ investigation of the human-computer interaction. Int J Med Inform 2012; Apr 81 (04) 232-43.
- 30 Duke JD, Bolchini D. A successful model and visual design for creating context-aware drug-drug interaction alerts. AMIA Annu Symp Proc 2011; 339-48.
- 31 Russ AL, Saleem JJ, McManus MS, Frankel RM, Zillich AJ. The Workflow of Computerized Medication Ordering in Primary Care is Not Prescriptive;. 2010 p. 840-4.
- 32 van der Sijs H, van GT, Vulto A, Berg M, Aarts J. Understanding handling of drug safety alerts: a simulation study. Int J Med Inform 2010; May 79 (05) 361-9.
- 33 Wipfli R, Betrancourt M, Guardia A, Lovis C. A qualitative analysis of prescription activity and alert usage in a computerized physician order entry system. Stud Health Technol Inform 2011; 169: 940-4.
- 34 Weingart SN, Massagli M, Cyrulik A, Isaac T, Morway L, Sands DZ. et al. Assessing the value of electronic prescribing in ambulatory care: a focus group study. Int J Med Inform 2009; Sep 78 (09) 571-8.
- 35 Ash JS, Sittig DF, Dykstra RH, Guappone K, Carpenter JD, Seshadri V. Categorizing the unintended sociotechnical consequences of computerized provider order entry. Int J Med Inform 2007; Jun 76 Suppl 1: S21-S27.
- 36 Trafton J, Martins S, Michel M, Lewis E, Wang D, Combs A. et al. Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy for chronic, non cancer pain by primary care providers. Pain Med 2010; Apr 11 (04) 575-85.
- 37 Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SM. Exploring barriers and facilitators to the use of computerized clinical reminders. J Am Med Inform Assoc 2005; Jul 12 (04) 438-47.
- 38 Patterson ES, Nguyen AD, Halloran JP, Asch SM. Human factors barriers to the effective use of ten HIV clinical reminders. J Am Med Inform Assoc 2004; Jan 11 (01) 50-9.
- 39 Saleem JJ, Patterson ES, Militello L, Anders S, Falciglia M, Wissman JA. et al. Impact of clinical reminder redesign on learnability, efficiency, usability, and workload for ambulatory clinic nurses. J Am Med Inform Assoc 2007; Sep 14 (05) 632-40.
- 40 Baysari MT, Westbrook JI, Richardson KL, Day RO. The influence of computerized decision support on prescribing during ward-rounds: are the decision-makers targeted?. J Am Med Inform Assoc 2011; Nov 18 (06) 754-9.
- 41 Kortteisto T, Komulainen J, Makela M, Kunnamo I, Kaila M. Clinical decision support must be useful, functional is not enough: a qualitative study of computer-based clinical decision support in primary care. BMC Health Serv Res 2012; 12: 349-57.
- 42 Hartmann Hamilton AR, Anhoj J, Hellebek A, Ege-bart J, Bjorn B, Lilja B. Computerised Physician Order Entry (CPOE). Stud Health Technol Inform 2009; 148: 159-62.
- 43 Peute LW, Driest KF, Marcilly R, Bras Da Costa S, Beuscart-Zephir MC, Jaspers MW. A Framework for reporting on Human Factor/Usability studies of Health Information Technologies. Stud Health Technol Inform 2013; 194: 54-60.
- 44 Magrabi F, Ong MS, Runciman W, Coiera E. Using FDA reports to inform a classification for health information technology safety problems. J Am Med Inform Assoc 2012; Jan 19 (01) 45-53.
- 45 Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc 2011; May 1 18 (03) 327-34.
- 46 Oppenheim MI, Vidal C, Velasco FT, Boyer AG, Cooper MR, Hayes JG. et al. Impact of a computerized alert during physician order entry on medication dosing in patients with renal impairment. Proc AMIA Symp 2002; 577-81.
- 47 Thursky K. Use of computerized decision support systems to improve antibiotic prescribing. Expert Rev Anti Infect Ther 2006; Jun 4 (03) 491-507.
- 48 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; Apr 2 330: 765-73.
- 49 Schedlbauer A, Prasad V, Mulvaney C, Phansalkar S, Stanton W, Bates DW. et al. What evidence supports the use of computerized alerts and prompts to improve clinicians’ prescribing behavior?. J Am Med Inform Assoc 2009; Jul 16 (04) 531-8.
- 50 Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008; Sep 15 (05) 585-600.
- 51 Chan J, Shojania KG, Easty AC, Etchells EE. Does user-centred design affect the efficiency, usability and safety of CPOE order sets?. J Am Med Inform Assoc 2011; May 1 18 (03) 276-81.
- 52 Russ AL, Zillich AJ, Melton BL, Russell SA, Chen S, Spina JR. et al. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc 2014 Mar 25.
- 53 Tsopra R, Jais JP, Venot A, Duclos C. Comparison of two kinds of interface, based on guided navigation or usability principles, for improving the adoption of computerized decision support systems: application to the prescription of antibiotics. J Am Med Inform Assoc 2014; Feb 21 e1 e107-e116.