Improving Clinical Trial Participant Tracking Tools Using Knowledge-anchored Design Methodologies

P.R.O. Payne; P.J. Embi; S.B. Johnson; E. Mendonca; J. Starren

doi:10.4338/ACI-2010-02-RA-0012

Applied Clinical Informatics, Table of Contents

Appl Clin Inform 2010; 01(02): 177-196
DOI: 10.4338/ACI-2010-02-RA-0012

Research Article

Schattauer GmbH

Improving Clinical Trial Participant Tracking Tools Using Knowledge-anchored Design Methodologies

P.R.O. Payne

¹The Ohio State University, Department of Biomedical Informatics and Center for Clinical and Translational Science, Columbus, OH

,

P.J. Embi

²University of Cincinnati, Center for Health Informatics and Department of Medicine, Cincinnati, OH

,

S.B. Johnson

³Columbia University, Department of Biomedical Informatics, New York, NY

,

E. Mendonca

⁴The University of Chicago, Department of Pediatrics, Chicago, IL

,

J. Starren

⁵Marshfield Clinic Research Foundation, Marshfield, WI

› Author Affiliations

Abstract

Summary

Objective: Rigorous human-computer interaction (HCI) design methodologies have not traditionally been applied to the development of clinical trial participant tracking (CTPT) tools. Given the frequent use of iconic HCI models in CTPTs, and prior evidence of usability problems associated with the use of ambiguous icons in complex interfaces, such approaches may be problematic. Presentation Discovery (PD), a knowledge-anchored HCI design method, has been previously demonstrated to improve the design of iconic HCI models. In this study, we compare the usability of a CTPT HCI model designed using PD and an intuitively designed CTPT HCI model.

Methods: An iconic CPTP HCI model was created using PD. The PD-generated and an existing iconic CTPT HCI model were subjected to usability testing, with an emphasis on task accuracy and completion times. Study participants also completed a qualitative survey instrument to evaluate subjective satisfaction with the two models.

Results: CTPT end-users reliably and reproducibly agreed on the visual manifestation and semantics of prototype graphics generated using PD. The performance of the PD-generated iconic HCI model was equivalent to an existing HCI model for tasks at multiple levels of complexity, and in some cases superior. This difference was particularly notable when tasks required an understanding of the semantic meanings of multiple icons.

Conclusion: The use of PD to design an iconic CTPT HCI model generated beneficial results and improved end-user subjective satisfaction, while reducing task completion time. Such results are desirable in information and time intensive domains, such as clinical trials management.

Citation: Payne PRO, Embi PJ, Johnson SB, Mendonca E, Starren J. Improving clinical trial participant tracking tools using knowledge-anchored design methodologies. Appl Clin Inf 2010; 1: 177–196 http://dx.doi.org/10.4338/ACI-2010-02-RA-0012

Keywords

User-computer interface - visual perception - computer graphics - medical informatics - clinical trial - biomedical research

Full Text

References

References
1 Chung TK, Kukafka R, Johnson SB. Reengineering clinical research with informatics. J Investig Med 2006 Sep 1 54 (Suppl. 06) 327-333.
2 Payne PR, Johnson SB, Starren JB, Tilson HH, Dowdy D. Breaking the translational barriers: the value of integrating biomedical informatics and translational research. J Investig Med 2005; May 53 (Suppl. 04) 192-200.
3 Sung NS, Crowley Jr WF, Genel M, Salber P, Sandy L, Sherwood LM. et al. Central challenges facing the national clinical research enterprise. JAMA 2003 Mar 12 289 (Suppl. 10) 1278-1287.
4 Khan SA, Kukafka R, Payne PR, Bigger JT, Johnson SB. A day in the life of a clinical research coordinator: observations from community practice settings. Medinfo 2007; 12 Pt 1 247-251.
5 Khan SA, Payne PR, Johnson SB, Bigger JT, Kukafka R. Modeling clinical trials workflow in community practice settings. Proc AMIA Symp 2006; Jan 1 419-423.
6 Kukafka R, Johnson SB, Linfante A, Allegrante JP. Grounding a new information technology implementation framework in behavioral science: a systematic analysis of the literature on IT use. J Biomed Inform 2003; Jun 36 (Suppl. 03) 218-227.
7 Chow SC, Liu JP. Design and Analysis of Clinical Trials: Concepts and Methodologies. 2 ed. Hoboken, NJ: John Wiley & Sons, Inc.; 2003
8 Brandt CA, Argraves S, Money R, Ananth G, Trocky NM, Nadkarni PM. Informatics tools to improve clinical research study implementation. Contemporary clinical trials. 2006 Apr 1 27 (Suppl. 02) 112-122.
9 Payne PRO, Greaves AW, Kipps TJ. editors. CRC Clinical Trials Management System (CTMS): An Integrated Information Management Solution for Collaborative Clinical Research. AMIA 2003 Symposium; 2003; Washington, DC, USA.:
10 Payne PR, Greaves AW, Kipps TJ. CRC Clinical Trials Management System (CTMS): an integrated information management solution for collaborative clinical research. AMIA Annu Symp Proc 2003; 967.
11 Starren JB, Payne PR, Kaufman DR. Human computer interaction issues in Clinical Trials Management Systems. 2006
12 NIGMS.. National Institute of General Medical Sciences. Bethesda, MD: National Institutes of Health; 2010. Available from: http://www.nigms.nih.gov
13 Shneiderman B. Designing the user interface : strategies for effective human-computer-interaction. 3rd ed. Reading, Mass: Addison Wesley Longman; 1998
14 Payne PR, Starren JB. Quantifying visual similarity in clinical iconic graphics. J Am Med Inform Assoc 2005; May-Jun 12 (Suppl. 03) 338-345.
15 Payne PR, Starren JB. Presentation discovery: building a better icon. CHI ‘06 extended abstracts on Human factors in computing systems; Motreal, Quebec, Canada: ACM; 2006
16 Starren JB, Payne PR. Quantifying visual similarity in clinical iconic graphics. AMIA Annu Symp Proc 2003; 1016.
17 Gosbee JW. Applying CHI in Health Care: domain issues, resources and requirements. CHI 98 conference summary on Human factors in computing systems; Los Angeles, California, United States: ACM Press; 1998
18 Cole WG. editor. Metaphor Graphics & Visual Analogy For Medical Data. Annu Symp Comput Appl Med Care. 1987
19 Cole WG, Stewart JG. Human performance evaluation of a metaphor graphic display for respiratory data. Methods Inf Med 1994; Oct 33 (Suppl. 04) 390-396.
20 Kelly GA. The psychology of personal constructs. [1st ] ed. New York: Norton; 1955
21 Marcus A. Icons, symbols, and signs: visible languages to facilitate communication interactions. 2003; 10 (Suppl. 03) 37-43.
22 Patel VL, Arocha JF, Kaufman DR. A primer on aspects of cognition for medical informatics. J Am Med Inform Assoc 2001; Jul-Aug 8 (Suppl. 04) 324-343.
23 Chandler D. Semiotics: The Basics. 1 ed. New York: Routledge; 2003
24 Leite JC. editor. A Semiotic-based framework to user interface design. NordiCHI; 2002. Arhus, Denmark.:
25 Marcus A. Human communications issues in advanced UIs. Commun ACM 1993; 36 (Suppl. 04) 100-109.
26 Elting LS, Bodey GP. Is a picture worth a thousand medical words? A randomized trial of reporting formats for medical research data. Meth Info Med 1991; 30 (Suppl. 02) 145-150.
27 Litt HI, Schmidt DF. editors. Application of the Visual Chart in an Ambulatory OB-GYN Clinic. Annu Symp Comput Appl Med Care. 1995
28 Abad-Mota S, Kulikowski C, Gong L, Stevenson S, Mezrich R, Tria A. et al., editors. Iconic reporting: a new way of communicating radiologic findings. Annu Symp Comput Appl Med Care. 1995
29 Barr P, Noble J, Biddle R. Icons R icons. Adelaide, Australia: Australian Computer Society, Inc.; 2003
30 Horton WK. The icon book : visual symbols for computer systems and documentation. New York: J. Wiley; 1994
31 Starren J. From Multimodal Sublanguages to Medical Data Presentations. New York: Columbia University; 1997
32 Payne PR, Starren JB. Modeling categorical sorting behavior. Medinfo 2004; 2004 CD 1805.
33 Deitzer JR, Payne PR, Starren JB. Coverage of clinical trials tasks in existing ontologies. Proc AMIA Symp. 2006 Jan 1: 903.
34 Payne PR, Deitzer JR, Mendonca EA, Starren JB. Consensus-based construction of a taxonomy of clinical trial tasks. Proc AMIA Symp. 2006 Jan 1: 1059.
35 Rugg G, McGeorge P. The sorting techniques: a tutorial paper on card sorts, picture sorts and tiem sorts. Expert Systems 1997; May 14 (Suppl. 02) 80-93.
36 JMP 7ed: SAS.
37 Romesburg HC. Cluster Analysis for Researchers. 2 ed. Raleigh, North Carolina: Lulu Press; 2004
38 Morae.. TechSmith. 2010
39 Chin JP, Diehl VA, Normal KL. editors. Development of an instrument measuring user satisfaction of the human-computer interface. CHI Conference on Human Factors in Computing Systems 1988. Washington D C.: ACM Press.;
40 Likert R. A Technique for the Measurement of Attitudes. Archives of Psychology 1932; 140 (June).
41 Perneger TV. What’s Wrong with Bonferroni Adjustments. BMJ 1998; 316 April 1236-1238.
42 Sankoh AJ, Huque MF, Dusbey SD. Some comments on frequently used multipled endpoint adjustment methods in clinical trials. Stat Med 1997; 16 (Suppl. 22) 2529-2543.
43 Telea AC. Data visualization : principles and practice. Wellesley, Mass.: A K Peters; 2008
44 Harris Z, Mattick P. Science Sublanguages and the Prospects for a Global Language of Science. Annals of the American Academy of Political and Social Science. 1988 Jan 495: 73-83.
45 Harris Z. Theory of Language Structure. American Philosophical Quarterly 1976; 13 (Suppl. 04) 237-255.
46 Hripcsak G, Wilcox A. Reference standards, judges, and comparison subjects: roles for experts in evaluating system performance. J Am Med Inform Assoc 2002; Jan-Feb 9 (Suppl. 01) 1-15.
47 Dicks RS. Mis-usability: on the uses and misuses of usability testing. Proceedings of the 20th annual international conference on Computer documentation Toronto, Ontario, Canada: 584960 ACM; 2002: 26-30.
48 Glaser B, Strauss A. The Discovery of Grounded Theory: Strategies for Qualitative Research. Piscataway, NJ: Aldine Transaction; 1967