Applied Clinical Informatics, Table of Contents Appl Clin Inform 2016; 07(04): 1069-1087DOI: 10.4338/ACI-2016-06-R-0105 Review Schattauer GmbH Mind the GapA systematic review to identify usability and safety challenges and practices during electronic health record implementationAuthors Author Affiliations Raj Ratwani 1 National Center for Human Factors in Healthcare, MedStar Health, Washington D.C Terry Fairbanks 1 National Center for Human Factors in Healthcare, MedStar Health, Washington D.C Erica Savage 1 National Center for Human Factors in Healthcare, MedStar Health, Washington D.C Katie Adams 1 National Center for Human Factors in Healthcare, MedStar Health, Washington D.C Michael Wittie 2 Office of the National Coordinator of Health Information Technology, Washington D.C. Edna Boone 1 National Center for Human Factors in Healthcare, MedStar Health, Washington D.C Andrew Hayden 1 National Center for Human Factors in Healthcare, MedStar Health, Washington D.C Janey Barnes 3 User-View, Raleigh, North Carolina Zach Hettinger 1 National Center for Human Factors in Healthcare, MedStar Health, Washington D.C Andrew Gettinger 2 Office of the National Coordinator of Health Information Technology, Washington D.C. Recommend Article Abstract Full Text PDF Download(opens in new window) Keywords KeywordsUsability - safety - human factors - electronic health records - implementation References References 1 Goldberg DG, Kuzel AJ, Feng LB, DeShazo JP, Love LE. EHRs in primary care practices: Benefits, challenges, and successful straegies. Am J Manag Care 2012; 18 (02) e48-e54. 2 King J, Patel V, Jamoom EW, Furukawa MF. Clinical benefits of electronic health record use: National findings. Health Serv Res 2014; 49 (1 PART 2): 392-404 PMID: 24359580. 3 Bertman J, Skolnik N, Anderson J. EHRs Get a Failing Grade on Usability. Intern Med News [Internet]. 2010 Available from: http://psych.imng.com/fileadmin/content_pdf/imn/archive_pdf/vol43iss11/70587_main.pdf. 4 Scheck A. Special Report: EHR Success All Comes Down to User-Friendliness. Emerg Med News [Internet] 2011; 33 (05) 26-27 Available from: http://content.wkhealth.com/linkback/openurl?sid=WKPTLP: landingpage&an=00132981-201105000-00002. 5 Koppel R, Kreda DA. Healthcare IT usability and suitability for clinical needs: challenges of design, workflow, and contractual relations. Stud Health Technol Inform. University of Pennsylvania; Philadelphia, PA, USA: rkoppel@sas.upenn.edu; 2010. 157: 7-14 Available from: papers2://publication/uuid/93BD5076-2932-4681-BFCB-BF17C6192753. 6 Middleton B, Bloomrosen M, Dente M a, Hashmat B, Koppel R, Overhage JM, Payne TH, Rosenbloom ST, Weaver C, Zhang J. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Informatics Assoc 2013; Jan 25: 1-7 Available from: http://www.ncbi.nlm.nih.gov/pubmed/23355463 PMID: 23355463. 7 IOM Committee on Patient Safety and Health Information Technology. Transcript. Barrister’s Reporting Service. 2011 8 International Standards Organization. ISO 9241: Ergonomics of Human System Interaction [Internet]. 2010 Available from: https://www.iso.org/obp/ui/#iso:std:iso:9241:-210:ed-1:v1:en. 9 Hettinger AZ, Ratwani R, Fairbanks RJ. New Insights on Safety and Health IT [Internet]. 2015 Available from: https://psnet.ahrq.gov/perspectives/perspective/181/new-insights-on-safety-and-health-it. 10 Bisantz AM, Roth EM, Brickman B, Lin L, McKinney J. Integrating cognitive analyses in a large scale system design process. Int J Hum Comput Stud 2003; 58: 177-206. 11 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (01) 56-76 Available from: http://linkinghub.elsevier.com/re-trieve/pii/S1532046404000206 PMID: 15016386. 12 Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda N. Electronic Health Record Usability: Analysis of the User Centered Design Process of Eleven Electronic Health Record Vendors. J Am Med Informatics Assoc 2015; 22 (06) 1179-1182 DOI: 10.1093/jamia/ocv050. 13 Viitanen J. Usability in Healthcare-An Analysis for the Development of Clinical Information Systems. 2010 Available from: http://www.soberit.hut.fi/kaipio/Viitanen_Research-plan_30112010.pdf. 14 Zhang J, Walji M. TURF: Toward a unified framework of EHR usability. Journal Biomed Informatics 2011; 44 (06) 1056-1067. 15 Department of Health and Human ServicesOffice of the National Coordinator for Health Information Technology. 2015 Edition Health Infomraiton Technology Certification Criteria, 2015 Edition Base Electronic Health Record Definition, and ONC health IT Certificaiton program modification. 2015 16 Meeks DW, Takian A, Sittig DF, Singh H, Barber N. Exploring the sociotechnical intersection of patient safety and electronic health record implementation. J Am Med Inform Assoc 2013; 21 (e1): e28-e34 Available from: http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=pubmed&id=24052536& retmode=ref&cmd=prlinks PMID: 24052536. 17 Middleton B, Bloomrosen M, Dente MA, Hashmat B, Koppel R, Overhage JM, Payne TH, Rosenbloom ST, Weaver C, Zhang J. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; 20 (e1): e2-e8 Available from: http://jamia.bmj.com/cgi/doi/10.1136/amiajnl-2012-001458 PMID: 23355463. 18 Slabodkin G. ONC Reports 4x Spike in Providers Switching EHR Vendors. Health Data Manag. 2015 Available from: http://www.healthdatamanagement.com/news/onc-reports-4x-spike-in-providers-switch ing-ehr-vendors. 19 Fairbanks R, Crittenden CN, O’Gara KG, Wilson MA, Pennington EC, Chin NP, Shah MN. Emergency Medical Services Provider Perceptions of the Nature of Adverse Events and Near-misses in Out-of-hospital Care: An Ethnographic View. Acad Emerg Med 2008; 15 (07) 633-640. 20 Otte-Trojel T, de Bont A, Rundall TG, van de Klundert J. What do we know about developing patient portals? a systematic literature review. J Am Med Informatics Assoc. 2015 (April 2015): ocv114. Available from: http://jamia.oxfordjournals.org/lookup/doi/10.1093/jamia/ocv114 PMID: 26335985. 21 Webiste PS. PRISMA Checklist. Website. http://www.prisma-statement.org/2.1.2%20-%20PRISMA%202009%20Checklist.pdf (accessed 25 April 2016) 22 Corbin J. Grounded theory in practice. Sage; 1997 23 Ratwani RM, Zachary AHettinger, Kosydar A, Fairbanks RJ, Hodgkins ML. A framework for evaluating electronic health record vendor user-centered design and usability testing processes. J Am Med Inform Assoc 2016; 1-5 Available from: http://www.ncbi.nlm.nih.gov/pubmed/27375292 PMID: 27375292. 24 Ajami S, Bagheri-Tadi T. Barriers for adopting electronic health records (EHRs) by physicians. Acta Inform Medica 2013; 21 (02) 129-134 PMID: 24058254. 25 Slight SP, Quinn C, Avery AJ, Bates DW, Sheikh A. A qualitative study identifying the cost categories associated with electronic health record implementation in the UK. J Am Med Inform Assoc 2014; 21 (e2): e226-e231 Available from: http://jamia.oxfordjournals.org/content/21/e2/e226.abstract PMID: 24523391. 26 Wakefield DS, Ward MM, Loes JL, O’Brien J. A network collaboration implementing technology to improve medication dispensing and administration in critical access hospitals. J Am Med Inform Assoc 2010; 17: 584-587 PMID: 20819868. 27 Bobb AM, Boehne J, Ethridge L, Hardy JC, Jacobs RS, Michael A, Lynch TW, Leslie R, Mannino TJ, Przybyla JJ, Reichert BJ, Runnebaum RM, Smestad NR, Troiano DL, Tyndall LL, Wright L, Clark T, Farthing K, Finkelstein BW, Fritz WL, Galt KA, Goldspiel BR, Jordan M, Marino RB, Kevin C, Meisel S, Miller AS, Mitchell S, Patel KR, Perry AB, Poikonen J, Siska MH, Windle M. ASHP guidelines on pharmacy planning for implementation of computerized provider-order-entry systems in hospitals and health systems. Am J Heal Pharm 2011; 68 (06) 537 PMID: 21378303. 28 Lawler EK, Hedge A, Pavlovic-Veselinovic S. Cognitive ergonomics, soci-technical systems, and the impact of healthcare information technologies. Int J Ind Ergon 2011; 41. 29 Takian A, Sheikh A, Barber N. We are bitter, but we are better off: case study of the implementation of an electronic health record system into a mental health hospital in England. BMC Health Serv Res 2012; 12 (01) 484 Available from: http://www.biomedcentral.com/1472-6963/12/484 PMID: 23272770. 30 Cooley TW, May D, Alwan M, Sue C. Implementation of computerized prescriber order entry in four academic medical centers. Am J Heal Pharm 2012; 69 (24) 2166-2173 PMID: 23230041. 31 Ames E, Ciotti V, Mathis B. Meaningful abuse: The rush toward EHR implementation. Healthc Financ Manag 2011; 65 (02) 70-73 PMID: 21428226. 32 Van Der Veen W, De Gier HJJ, Van Der Schaaf T, Taxis K, Van Den Bemt PMLA. Risk analysis and user satisfaction after implementation of computerized physician order entry in Dutch hospitals. Int J Clin Pharm 2013; 35 (02) 195-201 PMID: 23187963. 33 Hundt AS, Adams JA, Schmid JA, Musser LM, Walker JM, Wetterneck TB, Douglas S V, Paris BL, Carayon P. Conducting an efficient proactive risk assessment prior to CPOE implementation in an intensive care unit. Int J Med Inf 2013; 82 (01) 25-38 Available from: http://www.ncbi.nlm.nih.gov/pubmed/22608242 PMID: 22608242. 34 Denham CR, Classen DC, Swenson SJ, Henderson MJ, Zeltner T, Bates DW. Safe use of electronic health records and health information technology systems: trust but verify. J Patient Saf 2013; 09 (04) 177-189 Available from: http://graphics.tx.ovid.com/ovftpdfs/FPDDNCOBNFOANL00/fs047/ovft/live/gv024/01209203/01209203-201312000-00001.pdf PMID: 24257062. 35 Sittig DF, Ash JS, Singh H. The SAFER Guides: Empowering Organizations of Electronic Health Records. Am J Manag Care 2014; 20 (05) 418-423. 36 Hoonakker P, Cartmill RS, Carayon P, Walker JM. Development and psychometric qualities of the SEIPS survey to evaluate CPOE/EHR Implementaiton in ICSU. Int J Helathcare Inf Syst informatics 2011; 06 (01) 51-69. 37 Xiao Y, Montgomery DC. Development of a Tool to Measure User Experience Following Electronic Health Record Implementation. J Nurs Adm 2014; 44 (07) 423-428. 38 Cresswell KM, Lee L, Slee A, Coleman J, Bates DW, Sheikh A. Qualitative analysis of vendor discussions on the procurement of Computerised Physician Order Entry and Clinical Decision Support systems in hospitals: Table 1. BMJ Open 2015; 05 (10) e008313 Available from: http://bmjopen.bmj.com/lookup/doi/10.1136/bmjopen-2015-008313. 39 Spetz J, Burgess JF, Phibbs CS. What determines successful implementation of inpatient information technology systems?. Am J Manag Care 2012; 18 (03) 157-162 PMID: 22435909. 40 Silow-Carroll S, Edwards JN, Rodin D. Using Electronic Health Records to Improve Quality and Efficiency: The Experiences of Leading Hospitals. The Common 2012; (July): 1-40 Available from: http://www.com monwealthfund.org/∼/media/Files/Publications/Issue Brief/2012/Jul/1608_SilowCarroll_using_EHRs_ improve_quality.pdf. 41 Bernd DL, Fine PS. Electronic medical records: a path forward. Front Health Serv Manage 2011; 28 (01) 3-13 PMID: 21961380. 42 Nguyen L, Bellucci E, Nguyen LT. Electronic health records implementation?: An evaluation of information system impact and contingency factors. Int J Med Inform 2014; 83 (11) 779-796 Available from: http://dx.doi.org/10.1016/j.ijmedinf.2014.06.011. 43 Propp DA. Successful introduction of an emergency department electronic health record. West J Emerg Med 2012; 13 (04) 358-361 Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3523896&tool=pmcentrez&rendertype=abstract PMID: 23251715. 44 Muslin SI, Vardaman MJ, Cornell TP. Fostering Acceptance of Computerized Physician Order Entry. Health Care Manag (Frederick) 2014; 33 (02) 165-171 Available from: http://search.ebscohost.com/login.aspx?direct=true&AuthType=ip,shib&db=jlh&AN=2012584120&site=ehost-live&scope=site. 45 Simon SR, Keohane CA, Amato M, Coffey M, Cadet B, Zimlichman E, Bates DW. Lessons learned from implementation of computerized provider order entry in 5 community hospitals: a qualitative study. BMC Med Inform Decis Mak 2013; 13 (01) 67. Available from: http://www.biomedcentral.com/1472-6947/13/67 PMID: 23800211. 46 Haughom JL. Implementation of an electronic health record. BMJ (Clinical Res ed) [Internet] 2011; 343: d5887 Available from: http://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom= pubmed&id=21952445&retmode=ref&cmd=prlinks PMID: 21952445. 47 Shaha JS, El-Othmani MM, Saleh JK, Bozic KJ, Wright J, Tokish JM, Shaha SH, Saleh KJ. The Growing Gap in Electronic Medical Record Satisfaction Between Clinicians and Information Technology Professionals. J Bone Jt Surg 2015; 1979-1984. 48 Huryk LA. Factors influencing nurses’ attitudes towards healthcare information technology. J Nurs Manag 2010; 18 (05) 606-612 PMID: 20636510. 49 Alonso AH, Gonzalez CGR, Saez MS. Information technology and automation in hospitals: Strategies and experience in a tertiary hospital in Spain. EJHP Pract 2011; 17 (04) 26-31 Available from: http://www.eahp.eu/content/download/32026/201159/file/PRI26-31.pdf%5Cnhttp://ovidsp.ovid.com/ovidweb.cgi?T=JS&CSC=Y&NEWS=N&PAGE=fulltext&D=emed10&AN=2011561950%5Cnhttp://sfxhosted.exlibrisgroup.com/cmc?sid=OVID:embase&id=pmid:&id=doi:&issn=1781-9989&isbn=&vol PMID: 2011561950. 50 Chen AR, Lehmann CU. Computerized Provider Order Entry in Pediatric Oncology: Design, Implementation, and Outcomes. J Oncol Pract 2011; 07 (04) 213-218 Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3140440&tool=pmcentrez&rendertype=abstract PMID: 22043182. 51 Wright A, Sittig DF, Ash JS, Bates DW, Feblowitz J, Fraser G, Maviglia SM, McMullen C, Nichol WP, Pang JE, Starmer J, Middleton B. Governance for clinical decision support: case studies and recommended practices from leading institutions. J Am Med Inform Assoc 2011; 18 (02) 187-194 Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3116253&tool=pmcentrez&rendertype=abstract PMID: 21252052. 52 Melnick ER, Nielson JA, Finnell JT, Bullard MJ, Cantrill S V, Cochrane DG, Halamka JD, Handler JA, Holroyd BR, Kamens D, Kho A, McClay J, Shapiro JS, Teich J, Wears RL, Patel SJ, Ward MF, Richardson LD. Delphi consensus on the feasibility of translating the ACEP clinical policies into computerized clinical decision support. Ann Emerg Med 2010; 56 (04) 317-320 Available from: http://dx.doi.org/10.1016/j.anne-mergmed.2010.03.006 PMID: 20363531. 53 Senathirajah Y, Bakken S, Kaufman D. The clinician in the Driver’s Seat: Part 1 - A drag/drop user-composable electronic health record platform. J Biomed Inform 2014; 52: 165-176 Available from: http://dx.doi.org/10.1016/j.jbi.2014.09.002 PMID: 25240253. 54 Horn JR, Hansten PD, Osborn JD, Wareham P, Somani S. Customizing clinical decision support to prevent excessive drug-drug interaction alerts. Am J Heal Pharm 2011; 68: 662-664. 55 Patterson ES, Zhang J, Abbott P, Gibbons MC, Lowry SZ, Quinn MT, Ramaiah M, Brick D. Enhancing Electronic Health Record Usability in Pediatric Patient Care: A Scenario-Based Approach. Jt Comm J Qual Patient Saf. 2013 39. (03): 56 Irizarry T, Barton AJ. A sociotechnical approach to successful electronic health record implementation. Clin Nurse Spec 2013; 27 (06) 283-285 Available from: http://content.wkhealth.com/linkback/open-url?sid=WKPTLP:landingpage&an=00002800-201311000-00003 PMID: 24107749. 57 Ash JS, Sittig DF, McMullen CK, Wright A, Bunce A, Mohan V, Cohen DJ, Middleton B. Multiple perspectives on clinical decision support: a qualitative study of fifteen clinical and vendor organizations. BMC Med Inf Decis Mak 2015; 15 (01) 35 Available from: http://www.ncbi.nlm.nih.gov/pubmed/25903564%5Cnhttp://www.ncbi.nlm.nih.gov/pmc/articles/PMC4447027/pdf/12911_2015_Ar ticle_156.pdf PMID: 25903564. 58 Joukes E, Cornet R, Bruijne DeMC, Keizer DeNF. Eliciting end-user expectations to guide the implementation process of a new electronic health record: A case study using concept mapping. Int J Med Inform 2016; 87: 111-117 Available from: http://dx.doi.org/10.1016/j.ijmedinf.2015.12.014 PMID: 26806718. 59 Peute LW, Aarts J, Bakker PJM, Jaspers MWM. Anatomy of a failure: A sociotechnical evaluation of a laboratory physician order entry system implementation. Int J Med Inform 2010; 79 (04) 58-70 PMID: 19640778. 60 Chaffee BW, Zimmerman CR. Developing and implementing clinical decision support for use in a computerized prescriber-order-entry system. Am J Heal Pharm 2010; 67 (05) 391-400 PMID: 20172991. 61 Novak LL, Anders S, Gadd CS, Lorenzi NM. Mediation of adoption and use: a key strategy for mitigating unintended consequences of health IT implementation. J Am Med Inform Assoc 2012; 19 (06) 1043-1049 Available from: http://www.scopus.com/inward/record.url?eid=2-s2.0-84867678039&partnerID=tZOtx3y1 PMID: 22634157. 62 Pelayo S, Anceaux F, Rogalski J, Elkin P, Beuscart-Zephir MC. A comparison of the impact of CPOE implementation and organizational determinants on doctor-nurse communications and cooperation. Int J Med Inform 2013; 82 (12) 321-330 PMID: 22999779. 63 Friedman A, Crosson JC, Howard J, Clark EC, Pellerano M, Karsh B-T, Crabtree B, Jaen CR, Cohen DJ. A typology of electronic health record workarounds in small-to-medium size primary care practices. J Am Med Inform Assoc 2014; 21 (e1): e78-e83 Available from: http://jamia.bmj.com/cgi/doi/10.1136/amiajnl-2013-001686. 64 March CA, Steiger D, Scholl G, Mohan V, Hersh WR, Gold JA. Use of simulation to assess electronic health record safety in the intensive care unit: a pilot study. BMJ Open 2013; 03 (04) e002549-e002549 Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3641430&tool=pmcentrez&rendertype= abstract%5Cnhttp://bmjopen.bmj.com/cgi/doi/10.1136/bmjopen-2013-002549 PMID: 23578685. 65 Li AC, Kannry JL, Kushniruk A, Chrimes D, McGinn TG, Edonyabo D, Mann DM. Integrating usability testing and think-aloud protocol analysis with “near-live” clinical simulations in evaluating clinical decision support. Int J Med Inform 2012; 81 (11) 761-772 Available from: http://dx.doi.org/10.1016/j.ijme-dinf.2012.02.009 PMID: 22456088. 66 Landman AB, Redden L, Neri P, Poole S, Horsky J, Raja AS, Pozner CN, Schiff G, Poon EG. Using a medical simulation center as an electronic health record usability laboratory. J Am Med Informatics Assoc 2014; 21 (03) 558-563 Available from: http://jamia.oxfordjournals.org/lookup/doi/10.1136/amiajnl-2013-002233 PMID: 24249778 67 Corrao NJ, Robinson AG, Swiernik MA, Naeim A. Importance of testing for usability when selecting and implementing an electronic health or medical record system. J Oncol Pract 2010; 06 (03) 120-124 Available from: http://jop.ascopubs.org/content/6/3/120.full PMID: 20808553. 68 Chao CA, Goldbort J. Lessons Learned from Implementation of a Perinatal Documentation System. JOGNN - J Obstet Gynecol Neonatal Nurs 2012; 41 (05) 599-608 PMID: 22690726. 69 Green LA, Potworowski G, Day A, May-Gentile R, Vibbert D, Maki B, Kiesel L. Sustaining “meaningful use” of health information technology in low-resource practices. Ann Fam Med 2015; 13 (01) 17-22. 70 Holden RJ. What stands in the way of technology-mediated patient safety improvements?: a study of facilitators and barriers to physicians’ use of electronic health records. J Patient Saf 2011; 07 (04) 193-203 Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3220192&tool=pmcentrez&ren dertype=abstract PMID: 22064624. 71 Dennehy P, White MP, Hamilton A, Pohl JM, Tanner C, Onifade TJ, Zheng K. A partnership model for implementing electronic health records in resource-limited primary care settings: experiences from two nurse-managed health centers. J Am Med Informatics Assoc 2011; 18 (06) 820-826 Available from: http://jamia.oxfordjournals.org/content/18/6/820 PMID: 21828225. 72 Patel AA, Ozok AA. Exploring training issues in healthcare: Towards identifying barriers to increase electronic medical records adoption by healthcare professionals. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 2011; 7058 LNCS: 657-671. 73 Miller L, Stimely M, Matheny P, Pope M, McAtee R, Miller K. Novice Nurse Preparedness to Effectively Use Electronic Health Records in Acute Care Settings: Critical Informatics Knowledge and Skill Gaps. Online J Nurs Informatics. 2014 18. (2) (January) 74 Bramble JD, Galt KA, Siracuse M V, Abbott AA, Drincic A, Paschal KA, Fuji KT. The relationship between physician practice characteristics and physician adoption of electronic health records. Health Care Manage Rev 2010; 35 (01) 55-64 PMID: 20010013. 75 Yuan CT, Bradley EH, Nembhard IM. A mixed methods study of how clinician “super users” influence others during the implementation of electronic health records. BMC Med Inform Decis Mak. 2015 15. (1): Available from: http://www.biomedcentral.com/1472-6947/15/26 PMID: 25889076. 76 Kim MS, Clarke MA, Belden JL, Hinton E. Usability Challenges and Barriers in EHR Training of Primary Care Resident Physicians. 2014; 385-391. 77 McAlearney AS, Robbins J, Kowalczyk N, Chisolm DJ, Song PH. The role of cognitive and learning theories in supporting successful EHR system implementation training: a qualitative study. Med Care Res Rev 2012; 69 (03) 294-315 Available from: http://www.ncbi.nlm.nih.gov/pubmed/22451617 PMID: 22451617. 78 RTI International. Health IT Safety Center Roadmap. Available from: http://www.healthitsafety.org/uploads/4/3/6/4/43647387/roadmap.pdf.
