Usability Evaluation of a Community Pharmacy Health Information Exchange Interface Prototype

Abstract

Objective Few community pharmacies have access to health information exchange (HIE) data. We conducted a first-of-its-kind usability evaluation of an HIE interface prototype (referred to throughout as the “HIE-Pioneer mock-up”) developed with pharmacists and pharmacy technicians to aid future implementation in community pharmacies.

Methods Community pharmacists and pharmacy technicians were recruited to complete usability evaluations with the HIE-Pioneer mock-up. Each usability evaluation lasted up to 60 minutes. System usability scale (SUS) scores were collected from each participant following each usability evaluation session and summarized with descriptive statistics. Usability evaluation videos were reviewed for common usability attributes, such as the impact of identified usability problems, learnability, and efficiency. Time on task, task success rates, and prototype utilization were also recorded.

Results Sixteen total participants completed usability testing across three community pharmacies. The average SUS score was 69.7 (scale 0–100, where 100 is the best), with pharmacists on average reporting higher satisfaction than technicians (74.1 vs. 65.3, respectively). Altogether, we identified 23 distinct usability problems. Key problems identified included needed clarification in tool label names and accessibility of HIE links within the existing workflow. Overall, the usability of the HIE-Pioneer mock-up generally fostered pharmacy professionals' ease of learning and efficiency.

Conclusion Our study identified key areas, and potential solutions, to improve the usability of the HIE-Pioneer mock-up. Overall, pharmacy professionals viewed the HIE-Pioneer mock-up positively, with good satisfaction ratings. The HIE-Pioneer mock-up provides a blueprint for future HIE implementation in community pharmacy settings, which would increase community pharmacy teams' access to HIE data nationwide. Community pharmacy access to bi-directional HIE is expected to improve communication among more health care professionals involved in patient care and equip pharmacy professionals with needed information for improved clinical decision-making.

Keywords

system usability scale - health information exchange - pharmacists - pharmacies

• References

• 1 Health Information Exchange. The Office of the National Coordinator for Health Information Technology; . Accessed January 17, 2023 at: https://www.healthit.gov/topic/health-it-and-health-information-exchange-basics/health-information-exchange
• 2 Berenbrok LA, Tang S, Gabriel N. et al. Access to community pharmacies: A nationwide geographic information systems cross-sectional analysis. J Am Pharm Assoc (Wash DC) 2022; 62 (06) 1816-1822.e2
• 3 Hettinger KN, Snyder ME, Russ-Jara AL. A Formative Usability Evaluation of a Health Information Exchange Prototype for use in Community Pharmacies [poster abstract]. Poster presented at: American Medical Informatics Association Symposium Meeting; November 7, 2022; Washington, DC
• 4 Vest MH, Weber T, Savage S. et al. Development of an innovative partnership between a health-system department of pharmacy and external community pharmacies. Am J Health Syst Pharm 2022; 79 (22) 2040-2046
• 5 Craddock DS, Hall RG. Pharmacists without access to the EHR: practicing with one hand tied behind our backs. Innov Pharm 2021; (e-pub ahead of print)
• 6 Hohmeier KC, Spivey CA, Boldin S, Moore TB, Chisholm-Burns M. Implementation of a health information exchange into community pharmacy workflow. J Am Pharm Assoc (Wash DC) 2017; 57 (05) 608-615
• 7 Fanizza FA, Ruisinger JF, Prohaska ES, Melton BL. Integrating a health information exchange into a community pharmacy transitions of care service. J Am Pharm Assoc (Wash DC) 2018; 58 (04) 442-449
• 8 Milosavljevic A, Aspden T, Harrison J. Community pharmacist-led interventions and their impact on patients' medication adherence and other health outcomes: a systematic review. Int J Pharm Pract 2018; 26 (05) 387-397
• 9 Hughes CA, Guirguis LM, Wong T, Ng K, Ing L, Fisher K. Influence of pharmacy practice on community pharmacists' integration of medication and lab value information from electronic health records. J Am Pharm Assoc (Wash DC) 2011; 51 (05) 591-598
• 10 Roberts MF, Reeves K, Divine H. Community pharmacists' lack of access to health records and its impact on targeted MTM interventions. J Am Pharm Assoc (Wash DC) 2019; 59 (4S): S81-S84
• 11 Sethman NC, Hettinger KN, Snyder ME. Planning for health information exchange: Perspectives of community pharmacists in the Community Pharmacy Enhanced Services Network of Indiana. J Am Pharm Assoc (Wash DC) 2022; 62 (05) 1615-1622
• 12 Baker C, Loresto Jr F, Pickett K, Samay SS, Gance-Cleveland B. Facilitating health information exchange to improve health outcomes for school-aged children: school nurse electronic health record access. Appl Clin Inform 2022; 13 (04) 803-810
• 13 Schleyer TKL, Rahurkar S, Baublet AM. et al. Preliminary evaluation of the Chest Pain Dashboard, a FHIR-based approach for integrating health information exchange information directly into the clinical workflow. AMIA Joint Summits on Translational Science Proceedings. AMIA Joint Summits on Translational Science 2019:656–664
• 14 Cresswell KM, Bates DW, Sheikh A. Ten key considerations for the successful implementation and adoption of large-scale health information technology. J Am Med Inform Assoc 2013; 20 (e1): e9-e13
• 15 Bacci JL, Berenbrok LA. Innovative advances in connectivity and community pharmacist patient care services: implications for patient safety. Pharmacotherapy 2018; 38 (08) 867-874
• 16 Webber EC, Peterson RJ, Lory K, Kanis J, Saysana M, Schneider KS. A unique use of regional information exchange by a statewide health system serving refugees: “Operation Allies Welcome”. Appl Clin Inform 2023; 14 (03) 487-493
• 17 HIE Access for Pharmacists-Benefits and Barriers. Accessed January 17, 2023 at: https://www.ethin.org/benefits-and-barriers-hie-access-forpharmacists/#:∼:text=As%20such%2C%20the%20HIEs%20exclude%20pharmacists%20as%20well.,to%20access%20are%20substantial%2C%20the%20benefits%20are%2C%20too
• 18 Vogel LH, Reed WC. Management of Information in Health Care Organizations. In: Shortliffe EH, Cimino JJ, eds., Chiang MF. co-ed. Biomedical Informatics: Computer Applications in Healthcare and Biomedicine. New York: Springer; 2021
• 19 Lowry SZ, Quinn MT, Ramaiah M. et al. A Human Factors Guide to Enhance EHR Usability of Critical User Interactions when Supporting Pediatric Patient Care. NIST Interagency/Internal Report (NISTIR) 2012: 7865
• 20 Bakken S, Dykes PC, Collins Rosetti S. et al. Patient-Centered Care Systems. In: Shortliffe EH, Cimino JJ, eds., Chiang MF. co-ed. Biomedical Informatics: Computer Applications in Healthcare and Biomedicine. New York: Springer; 2021
• 21 Russ AL, Saleem JJ. Ten factors to consider when developing usability scenarios and tasks for health information technology. J Biomed Inform 2018; 78: 123-133
• 22 Brown B, Balatsoukas P, Williams R, Sperrin M, Buchan I. Multi-method laboratory user evaluation of an actionable clinical performance information system: Implications for usability and patient safety. J Biomed Inform 2018; 77: 62-80
• 23 Indiana Health Information Exchange. Accessed January 19, 2023 at: https://ihie.org
• 24 PioneerRx Expands Pharmacy Network to 5,000 Independent Pharmacies. Accessed January 17, 2023 at: https://www.pioneerrx.com/blog/pioneerrx-expands-pharmacy-network-to-5-000-independent-pharmacies#:∼:text=Finding%20its%20start%20in%202008%2C%20PioneerRx%20has%20grown,active%20in%205%2C000%20community%20pharmacies%20across%20the%20country
• 25 Clinical Data Search. Indiana Health Information Exchange. Accessed January 17, 2023 at: https://www.ihie.org/wp-content/uploads/2022/04/CDS-Flyer-2021.pdf
• 26 IHIE HealthDart. Accessed January 17, 2023 at: https://www.ihie.org/products-overview/
• 27 Krauss ZJ, Abraham M, Coby J. Clinical Pharmacy services enhanced by electronic health record (EHR) access: an innovation narrative. Pharmacy (Basel) 2022; 10 (06) 170
• 28 Talmon J, Ammenwerth E, Brender J, de Keizer N, Nykänen P, Rigby M. STARE-HI—statement on reporting of evaluation studies in health informatics. Int J Med Inform 2009; 78 (01) 1-9
• 29 Indiana CPESN. Accessed January 17, 2023 at: https://cpesn.com/networks/cpesn-indiana/
• 30 Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput 2003; 35 (03) 379-383
• 31 Nielson J. Demonstrate Think Aloud by Showing Users a Video. Accessed January 17, 2023 at: https://www.nngroup.com/articles/thinking-aloud-demo-video/
• 32 Fan M, Shi S, Truong KN. Practices and challenges of using think-aloud protocols in industry: an international study. J User Exper 2020; 15 (02) 85-102
• 33 Nielsen J. Estimating the number of subjects needed for a thinking aloud test. Int J Hum Comput Stud 1994; 41: 385-397
• 34 System Usability Scale. Accessed January 17, 2023 at: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
• 35 Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42 (02) 377-381
• 36 Harris PA, Taylor R, Minor BL. et al. The REDCap consortium. Building an international community of software partners. J Biomed Inform 2019; 95: 103208
• 37 Russ AL, Zillich AJ, Melton BL. et al. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc 2014; 21 (e2): e287-e296
• 38 Nielsen J. Ch 2: What is usability?. In: Usability engineering. San Francisco, CA: Morgan Kaufmann; 1993: 23-48
• 39 Russ AL, Baker DA, Fahner WJ. et al. A Rapid Usability Evaluation (RUE) Method for Health Information Technology. AMIA Annual Symposium Proceedings. AMIA Annual Symposium 2010:702–706
• 40 Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 2009; 4 (03) 114-123
• 41 Curran GM, Alle ME, Mukherjee S. et al. QUERI Series: a process for developing an implementation intervention. Implement Sci 2008; 3: 17
• 42 Lefchak B, Bostwick S, Rossetti SC. et al. Assessing usability and ambulatory clinical staff satisfaction with two electronic health records. Appl Clin Inform 2023; 14 (03) 494-502
• 43 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3 (17) 17
• 44 Fragidis LL, Chatzoglou PD. Implementation of a nationwide electronic health record (EHR). Int J Health Care Qual Assur 2018; 31 (02) 116-130
• 45 Thinking Aloud. Accessed January 17, 2023 at: https://www.sciencedirect.com/topics/computer-science/thinking-aloud#:∼:text=An%20obvious%20disadvantage%20of%20thinking%20aloud%20is%20that,used%20to%20get%20an%20impression%20of%20problem%20states
• 46 Jaspers MWM. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform 2009; 78 (05) 340-353
• 47 Russ AL, Jahn MA, Patel H. et al. Usability evaluation of a medication reconciliation tool: embedding safety probes to assess users' detection of medication discrepancies. J Biomed Inform 2018; 82: 178-186
• 48 Nielsen J, Landauer TK. A mathematical model of the finding of usability problems. Proceedings presented at: ACM INTERCHI'93 Conference; April 24–29, 1993; Amsterdam, The Netherlands; 206-213