Usability and Suitability of the Omics-Integrating Analysis Platform tranSMART for Translational Research and Education

Abstract

Background Platforms like tranSMART assist researchers in analyzing clinical and corresponding omics data. Usability is an important, yet often overlooked, factor affecting the adoption and meaningful use. Analyses on the specific needs of translational researchers and considerations about the application of such platforms for education are rare.

Objectives The aim of this study was to test whether tranSMART can be used in education and how well medical students and professional researchers can handle it; to identify which kind of translational researchers—in terms of skills, experienced limitations, and available data—can take advantage of tranSMART; and to evaluate the usability and to generate recommendations for improvements.

Methods An online-based test has been done by medical students (N = 109) and researchers (N = 26). The test comprised 13 tasks in the context of four typical research scenarios based on experimental and clinical data. A web questionnaire was provided to identify both the needs and the conditions of research as well as to evaluate the system's usability based on the “System Usability Scale” (SUS).

Results Students and researchers were able to handle tranSMART well and coped with most scenarios: cohort identification, data exploration, hypothesis generation, and hypothesis validation were answered with a rate of correctness between 82 and 100%. Of the total, 72.2% of the teaching researchers considered tranSMART suitable for their lessons and 84.6% of the researchers considered the platform useful for their daily work; 65.4% of the researchers named the nonavailability of a platform like tranSMART as a restriction on their research. The usability was rated “acceptable” with a SUS of 70.8.

Conclusion tranSMART is potentially suitable for education purposes and fits most of the needs of translational researchers. Improvements are needed on the presentation of analysis results and on the guidance of users through the analysis, especially to ensure the compliance of the analysis with the requirements of statistical testing.

Keywords

translational research - biomedical research - surveys and questionnaires - education - adoption - medical education - computer systems evaluation

Protection of Human and Animal Subjects

Our research plan was reviewed by the Ethics Committee of the Friedrich-Alexander-Universität Erlangen-Nürnberg which considered this type of research to be exempted from its approval process and issued a written confirmation thereof. In addition, we declare that the study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects.

• References

• 1 Joyce AR, Palsson BØ. The model organism as a system: integrating ‘omics’ data sets. Nat Rev Mol Cell Biol 2006; 7 (03) 198-210
• 2 Jonnagaddala J, Croucher JL, Jue TR. , et al. Integration and analysis of heterogeneous colorectal cancer data for translational research. Stud Health Technol Inform 2016; 225: 387-391
• 3 Rance B, Canuel V, Countouris H, Laurent-Puig P, Burgun A. Integrating heterogeneous biomedical data for cancer research: the CARPEM infrastructure. Appl Clin Inform 2016; 7 (02) 260-274
• 4 Gao J, Aksoy BA, Dogrusoz U. , et al. Integrative analysis of complex cancer genomics and clinical profiles using the cBioPortal. Sci Signal 2013; 6 (269) pl1
• 5 Ohno-Machado L, Bafna V, Boxwala AA. , et al; iDASH team. iDASH: integrating data for analysis, anonymization, and sharing. J Am Med Inform Assoc 2012; 19 (02) 196-201
• 6 Athey BD, Braxenthaler M, Haas M, Guo Y. tranSMART: an open source and community-driven informatics and data sharing platform for clinical and translational research. AMIA Jt Summits Transl Sci Proc 2013; 2013: 6-8
• 7 Canuel V, Rance B, Avillach P, Degoulet P, Burgun A. Translational research platforms integrating clinical and omics data: a review of publicly available solutions. Brief Bioinform 2015; 16 (02) 280-290
• 8 Wozney L, McGrath PJ, Newton A. , et al. Usability, learnability and performance evaluation of Intelligent Research and Intervention Software: a delivery platform for eHealth interventions. Health Informatics J 2016; 22 (03) 730-743
• 9 Soto-Rey I, N'Dja A, Cunningham J. , et al. User satisfaction evaluation of the EHR4CR Query Builder: a multisite patient count cohort system. BioMed Res Int 2015; 2015: 801436
• 10 Mathew D, McKibbon KA, Lokker C, Colquhoun H. Engaging with a Wiki related to knowledge translation: a survey of WhatisKT Wiki users. J Med Internet Res 2014; 16 (01) e21
• 11 Dunn Jr W, Burgun A, Krebs M-O, Rance B. Exploring and visualizing multidimensional data in translational research platforms. Brief Bioinform 2016; bbw080
• 12 Wheelock CE, Goss VM, Balgoma D. , et al; U-BIOPRED Study Group. Application of 'omics technologies to biomarker discovery in inflammatory lung diseases. Eur Respir J 2013; 42 (03) 802-825
• 13 Bierkens M, van der Linden W, Weistra W. , et al. Abstract 3166: querying, viewing and analyzing colorectal cancer translational research studies in tranSMART. Cancer Res 2016; 76 (14, Suppl): 3166
• 14 Bauer CR, Knecht C, Fretter C. , et al. Interdisciplinary approach towards a systems medicine toolbox using the example of inflammatory diseases. Brief Bioinform 2017; 18 (03) 479-487
• 15 Visser PJ, Streffer J. A European medical information framework for Alzheimer's disease (EMIF-AD). Alzheimers Dement 2015; 11 (07) 120-121
• 16 Camacho Rodriguez JC, Stäubert S, Löbe M. Automated import of clinical data from HL7 messages into OpenClinica and tranSMART using Mirth Connect. Stud Health Technol Inform 2016; 228: 317-321
• 17 Firnkorn D, Merker S, Ganzinger M, Muley T, Knaup P. Unlocking data for statistical analyses and data mining: generic case extraction of clinical items from i2b2 and tranSMART. Stud Health Technol Inform 2016; 228: 567-571
• 18 Wang S, Pandis I, Wu C. , et al. High dimensional biological data retrieval optimization with NoSQL technology. BMC Genomics 2014; 15 (Suppl. 08) S3
• 19 Huang K, Liu Y, Huang Y. , et al. Intelligent biology and medicine in 2015: advancing interdisciplinary education, collaboration, and data science. BMC Genomics 2016; 17 (Suppl. 07) 524
• 20 Gomez-Cabrero D, Abugessaisa I, Maier D. , et al. Data integration in the era of omics: current and future challenges. BMC Syst Biol 2014; 8 (Suppl. 02) I1
• 21 Salari K. The dawning era of personalized medicine exposes a gap in medical education. PLoS Med 2009; 6 (08) e1000138
• 22 Kohane IS, Churchill SE, Murphy SN. A translational engine at the national scale: informatics for integrating biology and the bedside. J Am Med Inform Assoc 2012; 19 (02) 181-185
• 23 i2b2 tranSMART Foundation. The i2b2 tranSMART Foundation announce completion of Merger [cited 2017 Aug 18]. Available from: http://transmartfoundation.org/the-i2b2-transmart-foundation-announce-completion-of-merger/
• 24 Cancer Genome Atlas Network. Comprehensive molecular characterization of human colon and rectal cancer. Nature 2012; 487 (7407): 330-337
• 25 Alexander Bondarev. tMDataLoader [cited 2017 Mar 15]. Available from: https://github.com/ThomsonReuters-LSPS/tMDataLoader
• 26 Christoph J, Knell C, Naschberger E. , et al. Two years of tranSMART in a university hospital for translational research and education. Stud Health Technol Inform 2017; 236: 70-79
• 27 Brooke J. SUS-A quick and dirty usability scale. Usability Eval Industry 1996; 189 (194) 4-7
• 28 Leiner DJ. SoSci Survey-Online Questionnaire [cited 2017 Mar 15]. Available at: https://www.soscisurvey.de/
• 29 Bush WS, Moore JH. Chapter 11: genome-wide association studies. PLOS Comput Biol 2012; 8 (12) e1002822
• 30 Subramanian A, Tamayo P, Mootha VK. , et al. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci U S A 2005; 102 (43) 15545-15550
• 31 Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput Interact 2008; 24 (06) 574-594
• 32 Herzinger S, Gu W, Satagopam V. , et al; eTRIKS Consortium. SmartR: an open-source platform for interactive visual analytics for translational research data. Bioinformatics 2017; 33 (14) 2229-2231
• 33 Knell C, Sedlmayr M, Christoph J. Developing interactive plug-ins for tranSMART using the SmartR framework: the case of survival analysis. Stud Health Technol Inform 2017; 236: 375-382
• 34 Sinha A, Markatou M. A platform for processing expression of short time series (PESTS). BMC Bioinformatics 2011; 12: 13
• 35 Cerami E, Gao J, Dogrusoz U. , et al. The cBio cancer genomics portal: an open platform for exploring multidimensional cancer genomics data. Cancer Discov 2012; 2 (05) 401-404
• 36 Barrett T, Troup DB, Wilhite SE. , et al. NCBI GEO: archive for high-throughput functional genomic data. Nucleic Acids Res 2009; 37 (Database issue): D885-D890
• 37 Tomczak K, Czerwińska P, Wiznerowicz M. The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge. Contemp Oncol (Pozn) 2015; 19 (1A): A68-A77