Download PDF
CC BY-NC-ND 4.0 · Appl Clin Inform 2019; 10(03): 454-470
DOI: 10.1055/s-0039-1692400
Research Article
Georg Thieme Verlag KG Stuttgart · New York

Usability Evaluation of Visual Representation Formats for Emergency Department Records

Nathaniel Brown
1   Department of Learning, Informatics, Management and Ethics, Health Informatics Centre, Karolinska Institutet, Stockholm, Sweden
2   Department of Emergency Medicine, Medical College of Wisconsin, Milwaukee, Wisconsin, United States
,
Aboozar Eghdam∗
1   Department of Learning, Informatics, Management and Ethics, Health Informatics Centre, Karolinska Institutet, Stockholm, Sweden
,
Sabine Koch∗
1   Department of Learning, Informatics, Management and Ethics, Health Informatics Centre, Karolinska Institutet, Stockholm, Sweden
› Author Affiliations
Further Information

Publication History

05 February 2019

29 April 2019

Publication Date:
26 June 2019 (online)

   Permissions and Reprints

Abstract

Background Integration of electronic information is a challenge for multitasking emergency providers, with implications for patient safety. Visual representations can assist sense-making of complex data sets; however, benefit and acceptability in emergency care is unproven.

Objectives This article evaluates visually focused alternatives to lists or tabular formats, to better understand possible usability in Emergency Department Information System (EDIS).

Methods A counterbalanced, repeated-measures experiment, satisfaction surveys, and narrative content analysis was conducted remotely by Web platform. Participants were 37 American emergency physicians; they completed 16 clinical cases comparing 4 visual designs to the control formats from a commercially available EDIS. They then evaluated two additional chart overview representations without controls.

Results Visual designs provided benefit in several areas compared to controls. Task correctness (90% to 76%; p = 0.003) and completion time (median: 49–74 seconds; p < 0.001) were superior for a medication history timeline with class and schedule highlighting. Completion time (median: 45–60 seconds; p = 0.03) was superior for a past medical history design, using pertinent diagnosis codes in highlighting rules. Less mental effort was reported for visual allergy (p = 0.04), past medical history (p < 0.001), and medication timeline (p < 0.001) designs. Most of the participants agreed with statements of likeability, preference, and benefit for visual designs; nonetheless, contrary opinions were seen, and more complex designs were viewed less favorably.

Conclusion Physician performance with visual representations of clinical data can in some cases exceed standard formats, even in absence of training. Highlighting of priority clinical categories was rated easier-to-use on average than unhighlighted controls. Perceived complexity of timeline representations can limit desirability for a subset of users, despite potential benefit.

Keywords

user–computer interface - emergency service - hospital - computer graphics - clinical decision-making

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects. A formal ethics declaration was evaluated in institutional review and approved prior to the project. No real patient data was used. All patient cases are fictional. All physician participants completed a standard written informed consent. Benefits to participants exceeded risks of harm; they are not a vulnerable group and freely chose to participate.


 

  • References

  • 1 Farley HL, Baumlin KM, Hamedani AG. , et al. Quality and safety implications of Emergency Department Information Systems. Ann Emerg Med 2013; 62 (04) 399-407
    CrossrefPubMedGoogle Scholar
  • 2 Laxmisan A, Hakimzada F, Sayan OR, Green RA, Zhang J, Patel VL. The multitasking clinician: decision-making and cognitive demand during and after team handoffs in emergency care. Int J Med Inform 2007; 76 (11-12): 801-811
    CrossrefPubMedGoogle Scholar
  • 3 Chisholm CD, Collison EK, Nelson DR, Cordell WH. Emergency department workplace interruptions: are emergency physicians “interrupt-driven” and “multitasking”?. Acad Emerg Med 2000; 7 (11) 1239-1243
    CrossrefPubMedGoogle Scholar
  • 4 Morrison JB, Rudolph JW. Learning from accident and error: avoiding the hazards of workload, stress, and routine interruptions in the emergency department. Acad Emerg Med 2011; 18 (12) 1246-1254
    CrossrefPubMedGoogle Scholar
  • 5 Källberg A-S, Göransson KE, Florin J, Östergren J, Brixey JJ, Ehrenberg A. Contributing factors to errors in Swedish emergency departments. Int Emerg Nurs 2015; 23 (02) 156-161
    CrossrefPubMedGoogle Scholar
  • 6 Franklin A, Liu Y, Li Z. , et al. Opportunistic decision making and complexity in emergency care. J Biomed Inform 2011; 44 (03) 469-476
    CrossrefPubMedGoogle Scholar
  • 7 Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004; 11 (02) 104-112
    PubMedGoogle Scholar
  • 8 Chaudhry B, Wang J, Wu S. , et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006; 144 (10) 742-752
    PubMedGoogle Scholar
  • 9 Hersh WR. Medical informatics: improving health care through information. JAMA 2002; 288 (16) 1955-1958
    CrossrefPubMedGoogle Scholar
  • 10 Handel DA, Wears RL, Nathanson LA, Pines JM. Using information technology to improve the quality and safety of emergency care. Acad Emerg Med 2011; 18 (06) e45-e51
    CrossrefPubMedGoogle Scholar
  • 11 Zahabi M, Kaber DB, Swangnetr M. Usability and safety in electronic medical records interface design: a review of recent literature and guideline formulation. Hum Factors 2015; 57 (05) 805-834
    CrossrefPubMedGoogle Scholar
  • 12 Ellsworth MA, Dziadzko M, O'Horo JC, Farrell AM, Zhang J, Herasevich V. An appraisal of published usability evaluations of electronic health records via systematic review. J Am Med Inform Assoc 2017; 24 (01) 218-226
    CrossrefPubMedGoogle Scholar
  • 13 Kim MO, Coiera E, Magrabi F. Problems with health information technology and their effects on care delivery and patient outcomes: a systematic review. J Am Med Inform Assoc 2017; 24 (02) 246-250
    CrossrefPubMedGoogle Scholar
  • 14 Caudill-Slosberg M, Weeks WB. Case study: identifying potential problems at the human/technical interface in complex clinical systems. Am J Med Qual 2005; 20 (06) 353-357
    CrossrefPubMedGoogle Scholar
  • 15 Mcdonnell C, Werner K, Wendel L. Electronic Health Record Usability: Vendor Practices and Perspectives. Rockville, MD: Agency for Healthcare Research and Quality; 2010
    Google Scholar
  • 16 Jung H, Kim T, Yang Y. , et al. Aesthetics in data visualization: case studies and design issues. In: Huang M, Huang W. , eds. Innovative Approaches of Data Visualization and Visual Analytics. Hershey, PA: IGI Global: Information Service Reference; 2014: 1-24
    CrossrefGoogle Scholar
  • 17 Ola O, Sedig K. Beyond simple charts: design of visualizations for big health data. Online J Public Health Inform 2016; 8 (03) e195
    PubMedGoogle Scholar
  • 18 Meyer J. Performance with tables and graphs: effects of training and a Visual Search Model. Ergonomics 2000; 43 (11) 1840-1865
    CrossrefPubMedGoogle Scholar
  • 19 Zoss A. Cognitive processes and traits related to graphic comprehension. In: Huang M, Huang W. , eds. Innovative Approaches of Data Visualization and Visual Analytics. Hershey, PA: IGI Global: Information Service Reference; 2014: 94-105
    CrossrefGoogle Scholar
  • 20 Stead WW, Miller RA, Musen MA, Hersh WR. Integration and beyond: linking information from disparate sources and into workflow. J Am Med Inform Assoc 2000; 7 (02) 135-145
    CrossrefPubMedGoogle Scholar
  • 21 Hertzum M, Simonsen J. Effects of electronic emergency-department whiteboards on clinicians' time distribution and mental workload. Health Informatics J 2016; 22 (01) 3-20
    CrossrefPubMedGoogle Scholar
  • 22 Dexheimer JW, Kennebeck S. Modifications and integration of the electronic tracking board in a pediatric emergency department. Pediatr Emerg Care 2013; 29 (07) 852-857
    CrossrefPubMedGoogle Scholar
  • 23 Sopan A, Plaisant C, Powsner S, Shneiderman B. Reducing wrong patient selection errors: exploring the design space of user interface techniques. In: AMIA Annual Symposium Proceedings. American Medical Informatics Association; 2014: 1056-1065 . Available at: http://www.ncbi.nlm.nih.gov/pubmed/25954415 . Accessed October 31, 2017
    PubMedGoogle Scholar
  • 24 Franklin A, Gantela S, Shifarraw S. , et al. Dashboard visualizations: supporting real-time throughput decision-making. J Biomed Inform 2017; 71: 211-221
    CrossrefPubMedGoogle Scholar
  • 25 Rasmussen R. Electronic whiteboards in emergency medicine: a systematic review. In: Luo G, Liu J. , eds. Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium Publication. Vol. 1. New York, NY: Association for Computing Machinery; 2012: 483-492 . Available at: https://rucforsk.ruc.dk/ws/files/37541649/Rasmussen_2012.pdf . Accessed May 18, 2018
    Google Scholar
  • 26 Ozturk S, Kayaalp M, McDonald CJ. Visualization of patient prescription history data in emergency care. In: AMIA Annual Symposium Proceedings. American Medical Informatics Association 2014: 963-968 . Available at: http://www.ncbi.nlm.nih.gov/pubmed/25954404 . Accessed October 31, 2017
    PubMedGoogle Scholar
  • 27 International Organization for Standardization. ISO 9241–11: Ergonomics of Human-System Interaction — Part 11: Usability: Definitions and Concepts; 2018 . Available at: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en . Accessed January 20, 2019
    PubMed
  • 28 Bauer DT, Guerlain S, Brown PJ. The design and evaluation of a graphical display for laboratory data. J Am Med Inform Assoc 2010; 17 (04) 416-424
    CrossrefPubMedGoogle Scholar
  • 29 Tasa UB, Ozcan O, Yantac AE, Unluer A. A case study on better iconographic design in electronic medical records' user interface. Inform Health Soc Care 2008; 33 (02) 125-138
    CrossrefPubMedGoogle Scholar
  • 30 Alonso DL, Rose A, Plaisant C, Norman KL. Viewing personal history records: a comparison of tabular format and graphical presentation using lifelines. Behav Inf Technol 1998; 17 (05) 249-262
    CrossrefPubMedGoogle Scholar
  • 31 Koch SH, Staggers N, Weir C, Agutter J, Liu D, Westenskow DR. Integrated information displays for ICU nurses: field observations, display design, and display evaluation. Proc Hum Factors Ergon Soc Annu Meet 2010; 54 (12) 932-936
    CrossrefPubMedGoogle Scholar
  • 32 Effken JA, Loeb RG, Kang Y, Lin ZC. Clinical information displays to improve ICU outcomes. Int J Med Inform 2008; 77 (11) 765-777
    CrossrefPubMedGoogle Scholar
  • 33 Belden JL, Wegier P, Patel J. , et al. Designing a medication timeline for patients and physicians. J Am Med Inform Assoc 2019; 26 (02) 95-105
    CrossrefPubMedGoogle Scholar
  • 34 Rind A, Wang TD, Aigner W. , et al. Interactive information visualization to explore and query electronic health records. Found Trends Human–Computer Interact 2013; 5 (03) 207-298
    CrossrefPubMedGoogle Scholar
  • 35 Wongsuphasawat K, John Alexis Guerra G, Catherine P, Taowei David W, Ben S, Taieb-Maimon M. LifeFlow: visualizing an overview of event sequences. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Vancouver, BC, Canada 2011: 1747-1756
    PubMedGoogle Scholar
  • 36 Cousins SB, Kahn MG. The visual display of temporal information. Artif Intell Med 1991; 3 (06) 341-357
    CrossrefPubMedGoogle Scholar
  • 37 Berg LM, Källberg A-S, Göransson KE, Östergren J, Florin J, Ehrenberg A. Interruptions in emergency department work: an observational and interview study. BMJ Qual Saf 2013; 22 (08) 656-663
    CrossrefPubMedGoogle Scholar
  • 38 Berlin A, Sorani M, Sim I. A taxonomic description of computer-based clinical decision support systems. J Biomed Inform 2006; 39 (06) 656-667
    CrossrefPubMedGoogle Scholar
  • 39 Starren J, Johnson SB. An object-oriented taxonomy of medical data presentations. J Am Med Inform Assoc 2000; 7 (01) 1-20
    CrossrefPubMedGoogle Scholar
  • 40 Patel VL, Arocha JF, Kaufman DR. A primer on aspects of cognition for medical informatics. J Am Med Inform Assoc 2001; 8 (04) 324-343
    CrossrefPubMedGoogle Scholar
  • 41 Treisman A, Gormican S, Warren HC. , et al. Feature analysis in early vision: evidence from search asymmetries. Psychol Rev 1988; 95 (01) 15-48
    CrossrefPubMedGoogle Scholar
  • 42 Wolfe JM. Guided Search 2.0: a revised model of visual search. Psychon Bull Rev 1994; 1 (02) 202-238
    CrossrefPubMedGoogle Scholar
  • 43 Hesse PN, Schmitt C, Klingenhoefer S, Bremmer F. Preattentive processing of numerical visual information. Front Hum Neurosci 2017; 11: 70
    PubMedGoogle Scholar
  • 44 Grabowecky M, Robertson LC, Treisman A. Preattentive processes guide visual search: evidence from patients with unilateral visual neglect. J Cogn Neurosci 1993; 5 (03) 288-302
    CrossrefPubMedGoogle Scholar
  • 45 Huang ML, Liang J, Huang W. Highlighting in visual data analytics. In: Huang M, Huang W. , eds. Innovative Approaches of Data Visualization and Visual Analytics. Hershey, PA: IGI Global: Information Service Reference; 2014: 178-190
    Google Scholar
  • 46 Curby KM, Glazek K, Gauthier I. A visual short-term memory advantage for objects of expertise. J Exp Psychol Hum Percept Perform 2009; 35 (01) 94-107
    CrossrefPubMedGoogle Scholar
  • 47 Ericsson KA, Kintsch W. Long-term working memory. Psychol Rev 1995; 102 (02) 211-245
    CrossrefPubMedGoogle Scholar
  • 48 Few S. Data presentation: tapping the power of visual perception. Intell Enterp 2004; 13 (07) 33
    PubMedGoogle Scholar
  • 49 Luck SJ, Vogel EK. The capacity of visual working memory for features and conjunctions. Nature 1997; 390 (6657): 279-281
    CrossrefPubMedGoogle Scholar
  • 50 Kirsh D. Interaction, external representation and sense making. AI Soc 2010; 25: 441-454
    CrossrefPubMedGoogle Scholar
  • 51 Zhang J, Norman DA. Representations in distributed cognitive tasks. Cogn Sci 1994; 18 (01) 87-122
    CrossrefPubMedGoogle Scholar
  • 52 Russell DM, Stefik MJ, Pirolli P, Card SK. The cost structure of sensemaking. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems; 1993: 269-276
    PubMedGoogle Scholar
  • 53 Etikan I, Abubakar Musa S, Sunusi Alkassim R. Comparison of convenience sampling and purposive sampling. Am J Theor Appl Stat 2016; 5 (01) 1-4
    PubMedGoogle Scholar
  • 54 Pollatsek A, Well AD. On the use of counterbalanced designs in cognitive research: a suggestion for a better and more powerful analysis. J Exp Psychol Learn Mem Cogn 1995; 21 (03) 785-794
    CrossrefPubMedGoogle Scholar
  • 55 Counselman FL, Babu K, Edens MA. , et al; 2016 EM Model Review Task Force; American Board of Emergency Medicine. The 2016 model of the clinical practice of emergency medicine. J Emerg Med 2017; 52 (06) 846-849
    CrossrefPubMedGoogle Scholar
  • 56 Caldwell R, Salem L, Smith WB. Health Design Challenge: Studio TACK. Health Design Challenge. 2013 . Available at: http://healthdesignchallenge.com/showcase/studiotack/studiotack.pdf . Accessed March 17, 2018
    PubMedGoogle Scholar
  • 57 Simes-Marques M. L. I. Usability of interfaces. In: Nunes IL. , ed. Ergonomics - A Systems Approach. Rijeka, Croatia: InTech; 2012: 155-170
    Google Scholar
  • 58 Johnson R, Onwuegbuzie A. Mixed methods research: a research paradigm whose time has come. Educ Res 2004; 33 (07) 14-26
    PubMedGoogle Scholar
  • 59 Sauro J, Dumas JS. Comparison of three one-question, post-task usability questionnaires. In: Proceedings of the 27th International Conference on Human Factors in Computing Systems - CHI '09. Boston, MA; 2009: 1599-1608
    PubMedGoogle Scholar
  • 60 U.S. Department of Health & Human Services Washington DC. System Usability Scale [SUS]. Usability.gov - Improving the User Experience. Published September 6, 2013 . Available at: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html . Accessed January 17, 2018
    PubMedGoogle Scholar
  • 61 J de Smith M. Statistical Analysis Handbook. 2018 . Available at: http://www.statsref.com/HTML/index.html . Accessed April 21, 2017
    PubMed
  • 62 Altman DG, Bland JM. Time to event (survival) data. BMJ 1998; 317 (7156): 468-469
    PubMedGoogle Scholar
  • 63 Abadi A, Saadat S, Yavari P, Bajdik C, Jalili P. Comparison of Aalen's additive and Cox proportional hazards models for breast cancer survival: analysis of population- based data from British Columbia, Canada. Asian Pac J Cancer Prev 2011; 12 (11) 3113-3116
    PubMedGoogle Scholar
  • 64 Machin D, Cheung YB, Parmar MKB. Survival Analysis: A Practical Approach. 2nd ed. West Sussex, England: John Wiley & Sons, Ltd; 2006
    Google Scholar
  • 65 Rummel B. Beyond average: Weibull analysis of task completion times. J Usability Stud 2017; 12 (02) 56-72
    PubMedGoogle Scholar
  • 66 Liu C, White RW, Dumais S. Understanding web browsing behaviors through Weibull analysis of dwell time. In: Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR '10; 2010: 379-387
    PubMedGoogle Scholar
  • 67 Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today 2004; 24 (02) 105-112
    CrossrefPubMedGoogle Scholar
  • 68 Fisher DL, Coury BG, Tengs TO, Duffy SA. Minimizing the time to search visual displays: the role of highlighting. Hum Factors 1989; 31 (02) 167-182
    CrossrefPubMedGoogle Scholar
  • 69 Pirolli P, Card S. Information foraging. Psychol Rev 1999; 106 (04) 643-675
    CrossrefPubMedGoogle Scholar
  • 70 Croskerry P. From mindless to mindful practice--cognitive bias and clinical decision making. N Engl J Med 2013; 368 (26) 2445-2448
    CrossrefPubMedGoogle Scholar
  • 71 Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med 2003; 78 (08) 775-780
    CrossrefPubMedGoogle Scholar
  • 72 Aigner W, Miksch S, Müller W, Schumann H, Tominski C. Visualizing time-oriented data—a systematic view. Comput Graph 2007; 31 (03) 401-409
    CrossrefPubMedGoogle Scholar
  • 73 Sanchez A. Understanding collections and their implicit structures through information visualization. In: Huang ML, Huang W. , eds. Innovative Approaches of Data Visualization and Visual Analytics. Hershey, PA: IGI Global: Information Service Reference; 2014: 151-171
    CrossrefGoogle Scholar
  • 74 Zhang S, Wu K. Feature-based uncertainty visualization. In: Huang M, Huang W. , eds. Innovative Approaches of Data Visualization and Visual Analytics. Hershey, PA: IGI Global: Information Service Reference; 2014: 68-93
    CrossrefGoogle Scholar
  • 75 Macefield R. How to specify the participant group size for usability studies: a practitioner's guide. J Usability Stud 2009; 5 (01) 34-45
    PubMedGoogle Scholar