Download PDF
Yearb Med Inform 2015; 24(01): 55-67
DOI: 10.15265/IY-2015-006
Original Article
Georg Thieme Verlag KG Stuttgart

Usability Flaws in Medication Alerting Systems: Impact on Usage and Work System

R. Marcilly
1   INSERM CIC-IT 1403, Lille; Université Lille Nord de France; CHU Lille; UDSL EA 2694, Lille, France
,
E. Ammenwerth
2   Institute of Health Informatics, UMIT – University for Health Sciences, Medical Informatics and Technology, Hall in Tirol, Austria
,
E. Roehrer
3   eHealth Services Research Group, School of Engineering and ICT, University of Tasmania, Hobart, Tasmania, Australia
,
S. Pelayo
1   INSERM CIC-IT 1403, Lille; Université Lille Nord de France; CHU Lille; UDSL EA 2694, Lille, France
,
F. Vasseur
1   INSERM CIC-IT 1403, Lille; Université Lille Nord de France; CHU Lille; UDSL EA 2694, Lille, France
,
M.-C. Beuscart-Zéphir
1   INSERM CIC-IT 1403, Lille; Université Lille Nord de France; CHU Lille; UDSL EA 2694, Lille, France
› Author Affiliations
Further Information

Publication History

30 June 2015

Publication Date:
10 March 2018 (online)

   Permissions and Reprints

Summary

Objectives: Previous research has shown that medication alerting systems face usability issues. There has been no previous attempt to systematically explore the consequences of usability flaws in such systems on users (i.e. usage problems) and work systems (i.e. negative outcomes). This paper aims at exploring and synthesizing the consequences of usability flaws in terms of usage problems and negative outcomes on the work system.

Methods: A secondary analysis of 26 papers included in a prior systematic review of the usability flaws in medication alerting was performed. Usage problems and negative outcomes were extracted and sorted. Links between usability flaws, usage problems, and negative outcomes were also analyzed.

Results: Poor usability generates a large variety of consequences. It impacts the user from a cognitive, behavioral, emotional, and attitudinal perspective. Ultimately, usability flaws have negative consequences on the workflow, the effectiveness of the technology, the medication management process, and, more importantly, patient safety. Only few complete pathways leading from usability flaws to negative outcomes were identified.

Conclusion: Usability flaws in medication alerting systems impede users, and ultimately their work system, and negatively impact patient safety. Therefore, the usability dimension may act as a hidden explanatory variable that could explain, at least partly, the (absence of) intended outcomes of new technology.

Keywords

Human engineering - decision support system - clinical - review - usability - usage - patient safety - alerting functions
 

  • References

  • 1 Koln L, Corrigan J, Donaldson M. editors. To err is human, building a safer health system. Washington, D.C.,: National Academic Press; 2000
    Google Scholar
  • 2 Kaplan B, Harris-Salamone KD. Health IT success and failure: recommendations from literature and an AMIA workshop. J Am Med Inform Assoc 2009; May 16 (03) 291-9.
    CrossrefPubMedGoogle Scholar
  • 3 McManus J, Wood-Harper T. Understanding the sources of information systems project failure. Manag Serv 2007; 38-43.
    PubMedGoogle Scholar
  • 4 International Standardization Organization.. Ergonomic requirements for office work with visual display terminals (VDTs) -- Part 11: Guidance on usability (Rep N° 9241-11). Geneva: International Standardization Organization; 1998
    Google Scholar
  • 5 Middleton B, Bloomrosen M, Dente MA, Hash-mat B, Koppel R, Overhage JM. et al. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; Jun 20 e1 e2-e8.
    CrossrefPubMedGoogle Scholar
  • 6 Magrabi F, Ong MS, Runciman W, Coiera E. An analysis of computer-related patient safety incidents to inform the development of a classification. J Am Med Inform Assoc 2010; Nov 17 (06) 663-70.
    CrossrefPubMedGoogle Scholar
  • 7 Samaranayake NR, Cheung ST, Chui WC, Cheung BM. Technology-related medication errors in a tertiary hospital: a 5-year analysis of reported medication incidents. Int J Med Inform 2012; Dec 81 (12) 828-33.
    CrossrefPubMedGoogle Scholar
  • 8 Lavery D, Cockton G. Representing Predicted and Actual Usability Problems. In: Johnson H, Johnson P, O’Neill E. editors. Proceedings of International Workshop on Representations in Interactive Software Development. Queen Mary and Westfield College, University of London; 1997. p. 97-108.
    Google Scholar
  • 9 Carayon P, Alvadaro CJ, Schoofs HA. Work system design in health care. In: Carayon P. editor. Handbook of human factors and ergonomics in health care and patient safety. Laurence Erlbaum Associates; New Jersey: 2007. p. 61-78.
    Google Scholar
  • 10 Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE. et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; Mar 9 293 (10) 1197-203.
    CrossrefPubMedGoogle Scholar
  • 11 Kushniruk A, Beuscart-Zephir MC, Grzes A, Borycki E, Watbled L, Kannry J. Increasing the safety of healthcare information systems through improved procurement: toward a framework for selection of safe healthcare systems. Healthc Q 2010; Sep 13 Spec No: 53-8.
    CrossrefPubMedGoogle Scholar
  • 12 Weiner JP, Kfuri T, Chan K, Fowles JB. “e-Iatro-genesis”: the most critical unintended consequence of CPOE and other HIT. J Am Med Inform Assoc 2007; May 14 (03) 387-8.
    CrossrefPubMedGoogle Scholar
  • 13 Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006; Sep 13 (05) 547-56.
    CrossrefPubMedGoogle Scholar
  • 14 Kushniruk A, Triola M, Stein B, Borycki E, Kannry J. The relationship of usability to medical error: an evaluation of errors associated with usability problems in the use of a handheld application for prescribing medications. Stud Health Technol Inform 2004; 107 Pt 2 1073-6.
    PubMedGoogle Scholar
  • 15 Nanji KC, Rothschild JM, Boehne JJ, Keohane CA, Ash JS, Poon EG. Unrealized potential and residual consequences of electronic prescribing on pharmacy workflow in the outpatient pharmacy. J Am Med Inform Assoc 2014; May 21 (03) 481-6.
    CrossrefPubMedGoogle Scholar
  • 16 Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998; Oct 21 280 (15) 1339-46.
    CrossrefPubMedGoogle Scholar
  • 17 Ranji SR, Rennke S, Wachter RM. Computerised provider order entry combined with clinical decision support systems to improve medication safety: a narrative review. BMJ Qual Saf 2014 Apr 12.
    PubMedGoogle Scholar
  • 18 Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004; Mar 11 (02) 104-12.
    CrossrefPubMedGoogle Scholar
  • 19 Marcilly R, Ammenwerth E, Vasseur F, Roehrer E, Beuscart-Zephir MC. Usability flaws of medication-related alerting systems: a systematic review. J Biomed Inform 2015; Jun 55: 260-71.
    CrossrefPubMedGoogle Scholar
  • 20 Scapin DL, Bastien JMC. Ergonomic criteria for evaluating the ergonomic quality of interactive systems. Behaviour and Information Technology 1997; 6 4-5 220-31.
    PubMedGoogle Scholar
  • 21 Sebillotte S. Action representation for home automation. -North-Holland Elsevier Science Publishers B.V.; 1990. p. 985-90.
    Google Scholar
  • 22 Russ AL, Zillich AJ, McManus MS, Doebbeling BN, Saleem JJ. A human factors investigation of medication alerts: barriers to prescriber decision-making and clinical workflow. AMIA Annu Symp Proc 2009; 548-52.
    PubMedGoogle Scholar
  • 23 Chused AE, Kuperman GJ, Stetson PD. Alert override reasons: a failure to communicate. AMIA Annu Symp Proc 2008; 111-5.
    PubMedGoogle Scholar
  • 24 Krall MA, Sittig DF. Clinician’s assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements. Proc AMIA Symp 2002; 400-4.
    PubMedGoogle Scholar
  • 25 Horsky J, Kaufman DR, Patel VL. Computer-based drug ordering: evaluation of interaction with a decision-support system. Stud Health Technol Inform 2004; 107 Pt 2 1063-7.
    PubMedGoogle Scholar
  • 26 Russ AL, Saleem JJ, McManus MS, Zillich AJ, Doebbling BN. Computerized medication alerts and prescriber mental models: observing routine patient care. Proceedings of the Human Factors and Ergonomics Society Annual Meeting; 2009 p. 655-9.
    PubMedGoogle Scholar
  • 27 Khajouei R, Peek N, Wierenga PC, Kersten MJ, Jaspers MW. Effect of predefined order sets and usability problems on efficiency of computerized medication ordering. Int J Med Inform 2010; Oct 79 (10) 690-8.
    CrossrefPubMedGoogle Scholar
  • 28 Feldstein A, Simon SR, Schneider J, Krall M, Laferriere D, Smith DH. et al. How to design computerized alerts to safe prescribing practices. Jt Comm J Qual Saf 2004; Nov 30 (11) 602-13.
    PubMedGoogle Scholar
  • 29 Russ AL, Zillich AJ, McManus MS, Doebbeling BN, Saleem JJ. Prescribers’ interactions with medication alerts at the point of prescribing: A multi-method, in situ investigation of the human-computer interaction. Int J Med Inform 2012; Apr 81 (04) 232-43.
    CrossrefPubMedGoogle Scholar
  • 30 Duke JD, Bolchini D. A successful model and visual design for creating context-aware drug-drug interaction alerts. AMIA Annu Symp Proc 2011; 339-48.
    PubMedGoogle Scholar
  • 31 Russ AL, Saleem JJ, McManus MS, Frankel RM, Zillich AJ. The Workflow of Computerized Medication Ordering in Primary Care is Not Prescriptive;. 2010 p. 840-4.
    PubMedGoogle Scholar
  • 32 van der Sijs H, van GT, Vulto A, Berg M, Aarts J. Understanding handling of drug safety alerts: a simulation study. Int J Med Inform 2010; May 79 (05) 361-9.
    CrossrefPubMedGoogle Scholar
  • 33 Wipfli R, Betrancourt M, Guardia A, Lovis C. A qualitative analysis of prescription activity and alert usage in a computerized physician order entry system. Stud Health Technol Inform 2011; 169: 940-4.
    PubMedGoogle Scholar
  • 34 Weingart SN, Massagli M, Cyrulik A, Isaac T, Morway L, Sands DZ. et al. Assessing the value of electronic prescribing in ambulatory care: a focus group study. Int J Med Inform 2009; Sep 78 (09) 571-8.
    CrossrefPubMedGoogle Scholar
  • 35 Ash JS, Sittig DF, Dykstra RH, Guappone K, Carpenter JD, Seshadri V. Categorizing the unintended sociotechnical consequences of computerized provider order entry. Int J Med Inform 2007; Jun 76 Suppl 1: S21-S27.
    CrossrefPubMedGoogle Scholar
  • 36 Trafton J, Martins S, Michel M, Lewis E, Wang D, Combs A. et al. Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy for chronic, non cancer pain by primary care providers. Pain Med 2010; Apr 11 (04) 575-85.
    CrossrefPubMedGoogle Scholar
  • 37 Saleem JJ, Patterson ES, Militello L, Render ML, Orshansky G, Asch SM. Exploring barriers and facilitators to the use of computerized clinical reminders. J Am Med Inform Assoc 2005; Jul 12 (04) 438-47.
    CrossrefPubMedGoogle Scholar
  • 38 Patterson ES, Nguyen AD, Halloran JP, Asch SM. Human factors barriers to the effective use of ten HIV clinical reminders. J Am Med Inform Assoc 2004; Jan 11 (01) 50-9.
    CrossrefPubMedGoogle Scholar
  • 39 Saleem JJ, Patterson ES, Militello L, Anders S, Falciglia M, Wissman JA. et al. Impact of clinical reminder redesign on learnability, efficiency, usability, and workload for ambulatory clinic nurses. J Am Med Inform Assoc 2007; Sep 14 (05) 632-40.
    CrossrefPubMedGoogle Scholar
  • 40 Baysari MT, Westbrook JI, Richardson KL, Day RO. The influence of computerized decision support on prescribing during ward-rounds: are the decision-makers targeted?. J Am Med Inform Assoc 2011; Nov 18 (06) 754-9.
    CrossrefPubMedGoogle Scholar
  • 41 Kortteisto T, Komulainen J, Makela M, Kunnamo I, Kaila M. Clinical decision support must be useful, functional is not enough: a qualitative study of computer-based clinical decision support in primary care. BMC Health Serv Res 2012; 12: 349-57.
    CrossrefPubMedGoogle Scholar
  • 42 Hartmann Hamilton AR, Anhoj J, Hellebek A, Ege-bart J, Bjorn B, Lilja B. Computerised Physician Order Entry (CPOE). Stud Health Technol Inform 2009; 148: 159-62.
    PubMedGoogle Scholar
  • 43 Peute LW, Driest KF, Marcilly R, Bras Da Costa S, Beuscart-Zephir MC, Jaspers MW. A Framework for reporting on Human Factor/Usability studies of Health Information Technologies. Stud Health Technol Inform 2013; 194: 54-60.
    PubMedGoogle Scholar
  • 44 Magrabi F, Ong MS, Runciman W, Coiera E. Using FDA reports to inform a classification for health information technology safety problems. J Am Med Inform Assoc 2012; Jan 19 (01) 45-53.
    CrossrefPubMedGoogle Scholar
  • 45 Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc 2011; May 1 18 (03) 327-34.
    CrossrefPubMedGoogle Scholar
  • 46 Oppenheim MI, Vidal C, Velasco FT, Boyer AG, Cooper MR, Hayes JG. et al. Impact of a computerized alert during physician order entry on medication dosing in patients with renal impairment. Proc AMIA Symp 2002; 577-81.
    PubMedGoogle Scholar
  • 47 Thursky K. Use of computerized decision support systems to improve antibiotic prescribing. Expert Rev Anti Infect Ther 2006; Jun 4 (03) 491-507.
    CrossrefPubMedGoogle Scholar
  • 48 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; Apr 2 330: 765-73.
    CrossrefPubMedGoogle Scholar
  • 49 Schedlbauer A, Prasad V, Mulvaney C, Phansalkar S, Stanton W, Bates DW. et al. What evidence supports the use of computerized alerts and prompts to improve clinicians’ prescribing behavior?. J Am Med Inform Assoc 2009; Jul 16 (04) 531-8.
    CrossrefPubMedGoogle Scholar
  • 50 Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008; Sep 15 (05) 585-600.
    CrossrefPubMedGoogle Scholar
  • 51 Chan J, Shojania KG, Easty AC, Etchells EE. Does user-centred design affect the efficiency, usability and safety of CPOE order sets?. J Am Med Inform Assoc 2011; May 1 18 (03) 276-81.
    CrossrefPubMedGoogle Scholar
  • 52 Russ AL, Zillich AJ, Melton BL, Russell SA, Chen S, Spina JR. et al. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc 2014 Mar 25.
    PubMedGoogle Scholar
  • 53 Tsopra R, Jais JP, Venot A, Duclos C. Comparison of two kinds of interface, based on guided navigation or usability principles, for improving the adoption of computerized decision support systems: application to the prescription of antibiotics. J Am Med Inform Assoc 2014; Feb 21 e1 e107-e116.
    CrossrefPubMedGoogle Scholar