Download PDF
Appl Clin Inform 2018; 09(03): 565-575
DOI: 10.1055/s-0038-1667041
Research Article
Georg Thieme Verlag KG Stuttgart · New York

Leveraging Patient-Reported Outcomes Using Data Visualization

Lisa V. Grossman
1   Department of Biomedical Informatics, Columbia University, New York, New York, United States
2   College of Physicians and Surgeons, Columbia University, New York, New York, United States
,
Steven K. Feiner
3   Department of Computer Science, Columbia University, New York, New York, United States
,
Elliot G. Mitchell
1   Department of Biomedical Informatics, Columbia University, New York, New York, United States
,
Ruth M. Masterson Creber
4   School of Nursing, Columbia University, New York, New York, United States
› Author Affiliations Funding This study is supported by the National Institute of Nursing Research (K99NR016275, PI: Masterson Creber).
Further Information

Publication History

21 March 2018

04 June 2018

Publication Date:
01 August 2018 (online)

   Permissions and Reprints

Abstract

Background Health care organizations increasingly use patient-reported outcomes (PROs) to capture patients' health status. Although federal policy mandates PRO collection, the challenge remains to better engage patients in PRO surveys, and ensure patients comprehend the surveys and their results.

Objective This article identifies the design requirements for an interface that assists patients with PRO survey completion and interpretation, and then builds and evaluates the interface.

Methods We employed a user-centered design process that consisted of three stages. First, we conducted qualitative interviews and surveys with 13 patients and 11 health care providers to understand their perceptions of the value and challenges associated with the use of PRO measures. Second, we used the results to identify design requirements for an interface that collects PROs, and designed the interface. Third, we conducted usability testing with 12 additional patients in a hospital setting.

Results In interviews, patients and providers reported that PRO surveys help patients to reflect on their symptoms, potentially identifying new opportunities for improved care. However, 6 out of 13 patients reported significant difficultly in understanding PRO survey questions, answer choices and results. Therefore, we identified aiding comprehension as a key design requirement, and incorporated visualizations into our interface design to aid comprehension. In usability testing, patients found the interface highly usable.

Conclusion Future interfaces designed to collect PROs may benefit from employing strategies such as visualization to aid comprehension and engage patients with surveys.

Keywords

data visualization - consumer health informatics - human–computer interaction - patient-reported outcomes - patient engagement - interfaces and usability

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects, and was reviewed by the Columbia University Medical Center Institutional Review Board.


 

  • References

  • 1 Ali J, Basch E, Baumhauer J. , et al. Users' Guide to Integrating Patient-Reported Outcomes in Electronic Health Records. Report. Baltimore, MD: Johns Hopkins University; 2017
    Google Scholar
  • 2 Wu A, Jensen R, Salzberg C, Snyder C. Advances in the use of patient reported outcome measures in electronic health records. Pcori Natl Work to Adv use PRO Meas Electron Heal Rec]; 2013 :1–82. Available at: http://www.pcori.org/assets/2013/11/PCORI-PRO-Workshop-EHR-Landscape-Review-111913.pdf . Accessed June 21, 2018
    PubMedGoogle Scholar
  • 3 Wu AW, Kharrazi H, Boulware LE, Snyder CF. Measure once, cut twice–adding patient-reported outcome measures to the electronic health record for comparative effectiveness research. J Clin Epidemiol 2013; 66 (8, Suppl): S12-S20
    CrossrefPubMedGoogle Scholar
  • 4 Wagle N. Implementing Patient-Reported Outcome Measures. NEJM Catal [Internet]; 2016 . Available at: http://catalyst.nejm.org/implementing-proms-patient-reported-outcome-measures/ . Accessed June 21, 2018
    PubMedGoogle Scholar
  • 5 Kotronoulas G, Kearney N, Maguire R. , et al. What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol 2014; 32 (14) 1480-1501
    CrossrefPubMedGoogle Scholar
  • 6 Basch E, Deal AM, Kris MG. , et al. Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 2016; 34 (06) 557-565
    PubMedGoogle Scholar
  • 7 Valderas JM, Kotzeva A, Espallargues M. , et al. The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Qual Life Res 2008; 17 (02) 179-193
    PubMedGoogle Scholar
  • 8 Marshall S, Haywood K, Fitzpatrick R. Impact of patient-reported outcome measures on routine practice: a structured review. J Eval Clin Pract 2006; 12 (05) 559-568
    CrossrefPubMedGoogle Scholar
  • 9 Greenhalgh J, Meadows K. The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. J Eval Clin Pract 1999; 5 (04) 401-416
    CrossrefPubMedGoogle Scholar
  • 10 Flynn KE, Dew MA, Lin L. , et al. Reliability and construct validity of PROMIS® measures for patients with heart failure who undergo heart transplant. Qual Life Res 2015; 24 (11) 2591-2599
    CrossrefPubMedGoogle Scholar
  • 11 Cella D, Riley W, Stone A. , et al. Initial Adult Health Item Banks and First Wave Testing of the Patient-Reported Outcomes Measurement Information System (PROMIS) Network: 2005–2008. J Clin Epidemiol 2011; 63 (11) 1179-1194
    PubMedGoogle Scholar
  • 12 Bevans M, Ross A, Cella D. Patient-Reported Outcomes Measurement Information System (PROMIS): efficient, standardized tools to measure self-reported health and quality of life. Nurs Outlook 2009; 62 (05) 339-345
    PubMedGoogle Scholar
  • 13 Cella D, Yount S, Rothrock N. , et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care 2007; 45 (05) (Suppl. 01) S3-S11
    CrossrefPubMedGoogle Scholar
  • 14 Paul M. Northwestern leads consortium to integrate patient-reported outcomes into electronic health records]; 2016. Available at: http://www.healthmeasures.net/images/applications/EASIPRO_PressRelease_102016.pdf . Accessed June 21, 2018
    PubMedGoogle Scholar
  • 15 Van Cranenburgh OD, Ter Stege JA, De Korte J, De Rie MA, Sprangers MAG, Smets EMA. Patient-reported outcome measurement in clinical dermatological practice: relevance and feasibility of a web-based portal. Dermatology 2016; 232 (01) 64-70
    CrossrefPubMedGoogle Scholar
  • 16 Jensen RE, Snyder CF, Abernethy AP. , et al. Review of electronic patient-reported outcomes systems used in cancer clinical care. J Oncol Pract 2014; 10 (04) e215-e222
    CrossrefPubMedGoogle Scholar
  • 17 Bennett AV, Jensen RE, Basch E. Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin 2015; 62 (05) 337-347
    PubMedGoogle Scholar
  • 18 Wynia MK, Osborn CY. Health literacy and communication quality in health care organizations. J Health Commun 2010; 15 (Suppl. 02) 102-115
    PubMedGoogle Scholar
  • 19 Huba N, Zhang Y. Designing patient-centered personal health records (PHRs): health care professionals' perspective on patient-generated data. J Med Syst 2012; 36 (06) 3893-3905
    CrossrefPubMedGoogle Scholar
  • 20 Snyder CF, Smith KC, Bantug ET. , et al. What do these scores mean? Presenting patient-reported outcomes data to patients and clinicians to improve interpretability. Cancer 2017; 123 (10) 1848-1859
    CrossrefPubMedGoogle Scholar
  • 21 Brundage MD, Smith KC, Little EA, Bantug ET, Snyder CF. , PRO Data Presentation Stakeholder Advisory Board. Communicating patient-reported outcome scores using graphic formats: results from a mixed-methods evaluation. Qual Life Res 2015; 24 (10) 2457-2472
    CrossrefPubMedGoogle Scholar
  • 22 Smith KC, Brundage MD, Tolbert E. , et al. Engaging stakeholders to improve presentation of patient-reported outcomes data in clinical practice. Support Care Cancer. Support Care Cancer 2016; 24 (10) 4149-4157
    CrossrefPubMedGoogle Scholar
  • 23 Arcia A, Woollen J, Bakken S. A systematic method for exploring data attributes in preparation for designing tailored infographics of patient reported outcomes. EGEMS (Wash DC) 2018; 6 (01) 1-9
    CrossrefPubMedGoogle Scholar
  • 24 Ambrosy AP, Fonarow GC, Butler J. , et al. The global health and economic burden of hospitalizations for heart failure: lessons learned from hospitalized heart failure registries. J Am Coll Cardiol 2014; 63 (12) 1123-1133
    CrossrefPubMedGoogle Scholar
  • 25 Benjamin EJ, Blaha MJ, Chiuve SE. , et al; American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Heart Disease and Stroke Statistics–2017 Update: a report from the American Heart Association. Circulation 2017; 135 (10) e146-e603
    CrossrefPubMedGoogle Scholar
  • 26 Goldberg RJ, Spencer FA, Szklo-Coxe M. , et al. Symptom presentation in patients hospitalized with acute heart failure. Clin Cardiol 2010; 33 (06) E73-E80
    PubMedGoogle Scholar
  • 27 Yen P-Y, Wantland D, Bakken S. Development of a customizable Health IT Usability Evaluation Scale. AMIA Annu Symp Proc 2010; 201: 917-921
    PubMedGoogle Scholar
  • 28 Masterson Creber R, Prey J, Ryan B. , et al. Engaging hospitalized patients in clinical care: Study protocol for a pragmatic randomized controlled trial. Contemp Clin Trials 2016; 47: 165-171
    CrossrefPubMedGoogle Scholar
  • 29 Chew L, Bradley KA, Boyko EJ. Brief questions to identify patients with inadequate health literacy. Health (London) 2004; 11: 12
    PubMedGoogle Scholar
  • 30 Jurgens CY, Fain JA, Riegel B. Psychometric testing of the heart failure somatic awareness scale. J Cardiovasc Nurs 2006; 21 (02) 95-102
    CrossrefPubMedGoogle Scholar
  • 31 Jurgens CY, Lee CS, Reitano JM, Riegel B. Heart failure symptom monitoring and response training. Hear Lung J Acute Crit Care. 2013; 42 (04) 273-280
    PubMedGoogle Scholar
  • 32 Jurgens CY, Lee CS, Riegel B. Psychometric analysis of the heart failure somatic perception scale as a measure of patient symptom perception. J Cardiovasc Nurs 2017; 32 (02) 140-147
    CrossrefPubMedGoogle Scholar
  • 33 Shenton AK. Strategies for ensuring trustworthiness in qualitative research projects. Educ Inf 2004; 22 (February): 63-75
    PubMedGoogle Scholar
  • 34 Lincoln YS. Emerging criteria for quality in qualitative and interpretive research. Qual Inq 1995; 1 (03) 275-289
    CrossrefPubMedGoogle Scholar
  • 35 Tobin GA, Begley CM. Methodological rigour within a qualitative framework. J Adv Nurs 2004; 48 (04) 388-396
    CrossrefPubMedGoogle Scholar
  • 36 Grossman LV, Mitchel EG. Visualizing the patient-reported outcomes measurement information system (PROMIS) measures for clinicians and patients. AMIA Annu Symp Proc 2017; 2289-2293
    PubMedGoogle Scholar
  • 37 Sauro J. SUPR-Q: a comprehensive measure of the quality of the website user experience. J Usability Stud 2015; 10 (02) 68-86
    PubMedGoogle Scholar
  • 38 Arcia A, Suero-Tejeda N, Bales ME. , et al. Sometimes more is more: iterative participatory design of infographics for engagement of community members with varying levels of health literacy. J Am Med Inform Assoc 2016; 23 (01) 174-183
    CrossrefPubMedGoogle Scholar
  • 39 Arcia A, Velez M, Bakken S. Style guide: an interdisciplinary communication tool to support the process of generating tailored infographics from electronic health data using EnTICE3. EGEMS (Wash DC) 2015; 3 (01) 1120
    PubMedGoogle Scholar
  • 40 Arcia A, Bales ME, Brown W. , et al. Method for the development of data visualizations for community members with varying levels of health literacy. AMIA 2013; 2013: 51-60
    PubMedGoogle Scholar
  • 41 Riegel B, Dickson VV. A situation-specific theory of heart failure self-care. J Cardiovasc Nurs 2008; 23 (03) 190-196
    CrossrefPubMedGoogle Scholar
  • 42 Riegel B, Carlson B, Moser DK, Sebern M, Hicks FD, Roland V. Psychometric testing of the self-care of heart failure index. J Card Fail 2004; 10 (04) 350-360
    CrossrefPubMedGoogle Scholar
  • 43 Riegel B, Lee CS, Dickson VV. Self-care in patients with chronic heart failure. Nat Rev Cardiol 2011; 8 (11) 644-654
    CrossrefPubMedGoogle Scholar
  • 44 Galesic M, Garcia-Retamero R. Graph literacy: a cross-cultural comparison. Med Decis Making 2011; 31 (03) 444-457
    CrossrefPubMedGoogle Scholar
  • 45 Nayak JG, Hartzler AL, Macleod LC, Izard JP, Dalkin BM, Gore JL. Relevance of graph literacy in the development of patient-centered communication tools. Patient Educ Couns 2016; 99 (03) 448-454
    CrossrefPubMedGoogle Scholar
  • 46 Sharit J, Lisigurski M, Andrade AD, Karanam C, Nazi KM, Lewis JR, Ruiz JG. The roles of health literacy, numeracy, and graph literacy on the usability of the VA's personal health record by veterans. J Usability Stud 2014; 9 (04) 173-193
    PubMedGoogle Scholar
  • 47 Ruiz JG, Andrade AD, Hogue C. , et al. The association of graph literacy with use of and skills using an online personal health record in outpatient veterans. J Health Commun 2016; 21 (sup2): 83-90
    CrossrefPubMedGoogle Scholar
  • 48 Fiscella K, Franks P, Clancy CM. Skepticism toward Medical Care and Health Care Utilization. Lippincott Williams & Wilkins Stable; 2017;36(2):180–189. Available at: http://www.jstor.org/stable/3767180 . Accessed June 21, 2018
    PubMedGoogle Scholar
  • 49 Sittig DF, Singh H. A socio-technical approach to preventing, mitigating, and recovering from ransomware attacks. Appl Clin Inform 2016; 7 (02) 624-632
    Article in Thieme ConnectPubMedGoogle Scholar
  • 50 Harkins M, Freed AM. The ransomware assault on the healthcare sector. J Law Cyber Warf 2018; 6 (02) 148-164
    PubMedGoogle Scholar
  • 51 Dalal AK, Dykes P, Mcnally K. , et al. Engaging Patients, Providers, and Institutional Stakeholders in Developing a Patient-centered Microblog. AMIA 2014 Annual Symposium. Washington, DC; 2014
    PubMedGoogle Scholar
  • 52 Kaelber D, Jha A, Johnston D, Middleton B, Bates D. A research agenda for personal health records (PHRs). J Am Med Inform Assoc 2008; 15 (06) 729-736
    CrossrefPubMedGoogle Scholar