Appl Clin Inform 2022; 13(05): 1015-1023
DOI: 10.1055/a-1942-6889
Research Article

Effect of Notes' Access and Complexity on OpenNotes' Utility

1   Industrial Engineering Department, School of Engineering, Mercer University, Macon, Georgia, United States
,
Ian Kratzke
2   Department of Surgery, School of Medicine, University of North Carolina-Chapel Hill, Chapel Hill, North Carolina, United States
,
Karthik Adapa
3   Department of Radiation Oncology, School of Medicine, University of North Carolina-Chapel Hill, Chapel Hill, North Carolina, United States
,
Lawrence Marks
3   Department of Radiation Oncology, School of Medicine, University of North Carolina-Chapel Hill, Chapel Hill, North Carolina, United States
,
Lukasz Mazur
3   Department of Radiation Oncology, School of Medicine, University of North Carolina-Chapel Hill, Chapel Hill, North Carolina, United States
› Author Affiliations

Abstract

Background Health care providers are now required to provide their patients access to their consultation and progress notes. Early research of this concept, known as “OpenNotes,” showed promising results in terms of provider acceptability and patient adoption, yet objective evaluations relating to patients' interactions with the notes are limited.

Objectives To assess the effect of the complexity level of notes and number of accesses (initial read vs. continuous access) on the user's performance, perceived usability, cognitive workload, and satisfaction with the notes.

Methods We used a 2*2 mixed subjects experimental design with two independent variables: (1) note's complexity at two levels (simple vs. complex) and (2) number of accesses to notes at two levels (initial vs. continuous). Fifty-three participants were randomly assigned to receive a simple versus complex radiation oncology clinical note and were tested on their performance for understanding the note content after an initial read, and then with continuous access to the note. Performance was quantified by comparing each participant's answers to the ones developed by the research team and assigning a score of 0 to 100 based on participants' understanding of the notes. Usability, cognitive workload, and satisfaction scores of the notes were quantified using validated tools.

Results Performance for understanding was significantly better in simple versus complex notes with continuous access (p = 0.001). Continuous access to the notes was also positively associated with satisfaction scores (p = 0.03). The overall perceived usability, cognitive workload, and satisfaction scores were considered low for both simple and complex notes.

Conclusion Simplifying notes can improve understanding of notes for patients/families. However, perceived usability, cognitive workload, and satisfaction with even the simplified notes were still low. To make notes more useful for patients and their families, there is a need for dramatic improvements to the overall usability and content of the notes.

Protection of Human and Animal Subjects

Participation was voluntary and does not pose undue risk. All human participants read and signed the informed consent form and all needed information was given to them when deciding whether to participate in the study. This was a remote unmoderated study and was fully implemented in Qualtrics.[42] The study protocol was reviewed and approved by the University of North Carolina at Chapel Hill Institutional Review Board (IRB) under reference ID: 338124.


Supplementary Material



Publication History

Received: 09 June 2022

Accepted: 11 September 2022

Accepted Manuscript online:
14 September 2022

Article published online:
26 October 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 
  • References

  • 1 HealthIT.gov. Office-based physician electronic health record adoption. Accessed May 18, 2021 at: https://dashboard.healthit.gov/quickstats/pages/physician-ehr-adoption-trends.php
  • 2 Roehrs A, da Costa CA, Righi RD, de Oliveira KSF. Personal health records: a systematic literature review. J Med Internet Res 2017; 19 (01) e13
  • 3 Delbanco T, Walker J, Darer JD. et al. Open notes: doctors and patients signing on. Ann Intern Med 2010; 153 (02) 121-125
  • 4 Kayastha N, Pollak KI, LeBlanc TW. Open notes: a qualitative study of oncology patients' experiences reading their cancer care notes. J Clin Oncol 2017; 35 (31, suppl): 33-33
  • 5 Murugan A, Gooding H, Greenbaum J. et al. Lessons learned from OpenNotes learning mode and subsequent implementation across a pediatric health system. Appl Clin Inform 2022; 13 (01) 113-122
  • 6 Salmi L, Dong ZJ, Yuh B, Walker J, DesRoches CM. Open notes in oncology: patient versus oncology clinician views. Cancer Cell 2020; 38 (06) 767-768
  • 7 Shaverdian N, Wang X, Hegde JV. et al. The patient's perspective on breast radiotherapy: initial fears and expectations versus reality. Cancer 2018; 124 (08) 1673-1681
  • 8 Shaverdian N, Chang EM, Chu F-I. et al. Impact of open access to physician notes on radiation oncology patients: results from an exploratory survey. Pract Radiat Oncol 2019; 9 (02) 102-107
  • 9 Turer RW, DesRoches CM, Salmi L, Helmer T, Rosenbloom ST. Patient perceptions of receiving COVID-19 test results via an online patient portal: an open results survey. Appl Clin Inform 2021; 12 (04) 954-959
  • 10 Sarabu C, Lee T, Hogan A, Pageler N. The value of OpenNotes for pediatric patients, their families and impact on the patient-physician relationship. Appl Clin Inform 2021; 12 (01) 76-81
  • 11 Ponathil AP, Khasawneh A, Byrne K, Madathil KC. Factors affecting the choice of a dental care provider by older adults based on online consumer reviews. IISE Trans Healthc Syst Eng 2021; 11 (01) 51-69
  • 12 Blease C, Salmi L, DesRoches CM. Open notes in cancer care: coming soon to patients. Lancet Oncol 2020; 21 (09) 1136-1138
  • 13 NIH. Cancer patients say clinical notes access valuable - National Cancer Institute. Accessed May 18, 2021 at: https://www.cancer.gov/news-events/cancer-currents-blog/2020/open-clinical-notes-access-by-cancer-patients
  • 14 Cohen J. The effect size. In: Statistical Power Analysis for the Behavioral Sciences. Mahwah: NJLawrence Erlbaum Associates; 1988: 8-13
  • 15 Khasawneh A, Chalil Madathil K, Dixon E, Wisniewski P, Zinzow H, Roth R. An investigation on the portrayal of blue whale challenge on youtube and twitter. Proc Hum Factors Ergon Soc Annu Meet 2019; 63 (01) 887-888
  • 16 Research for Me - Home. Accessed May 10, 2022 at https://researchforme.unc.edu/index.php/en/
  • 17 Harwood TG, Garry T. An overview of content analysis. Marketing Rev 2003; 3 (04) 479-498
  • 18 Richards KAR, Hemphill MA. A practical guide to collaborative qualitative data analysis. J Teach Phys Educ 2018; 37 (02) 225-231
  • 19 Khasawneh A, Chalil Madathil K, Dixon E, Wiśniewski P, Zinzow H, Roth R. Examining the self-harm and suicide contagion effects of the blue whale challenge on YouTube and Twitter: qualitative study. JMIR Ment Health 2020; 7 (06) e15973
  • 20 Khasawneh A, Madathil KC, Zinzow H. et al. An investigation of the portrayal of social media challenges on youtube and twitter. Trans Soc Comput 2021; 4 (01) 1-23
  • 21 Peres SC, Pham T, Phillips R. Validation of the system usability scale (SUS). Proc Hum Factors Ergon Soc Annu Meet 2013; 57 (01) 192-196
  • 22 Wilson MK, Khasawneh A, Ponathil A. et al. A preliminary study investigating patients' perceptions of research consenting methods. Proc Hum Factors Ergon Soc Annu Meet 2019; 63 (01) 1931-1935
  • 23 Khasawneh A, Rogers H, Bertrand J, Madathil KC, Gramopadhye A. Human adaptation to latency in teleoperated multi-robot human-agent search and rescue teams. Autom Construct 2019; 99: 265-277
  • 24 Gomes KM, Ratwani RM. Evaluating improvements and shortcomings in clinician satisfaction with electronic health record usability. JAMA Netw Open 2019; 2 (12) e1916651
  • 25 Cole AC, Adapa K, Khasawneh A, Richardson DR, Mazur L. Codesign approaches involving older adults in the development of electronic healthcare tools: a systematic review. BMJ Open 2022; 12 (07) e058390
  • 26 Lewis JR, Sauro J. Item benchmarks for the system usability scale. Journal of Usability Studies 2018; 13 (03) 158-167
  • 27 Sauro J, Lewis JR. Quantifying the User Experience: Practical Statistics for User Research. Amsterdam: Elsevier; 2004
  • 28 Hart SG. Nasa-Task Load Index (NASA-TLX); 20 years later. Proc Hum Factors Ergon Soc Annu Meet 2006; 50 (09) 904-908
  • 29 Hamad J, Fox A, Kammire MS, Hollis AN, Khairat S. Evaluating the experiences of new and existing teledermatology patients during the COVID-19 pandemic: cross-sectional survey study. JMIR Dermatol 2021; 4 (01) e25999
  • 30 Cowan N. What are the differences between long-term, short-term, and working memory?. Prog Brain Res 2008; 169: 323-338
  • 31 Mishra VK, Hoyt RE, Wolver SE, Yoshihashi A, Banas C. Qualitative and quantitative analysis of patients' perceptions of the patient portal experience with OpenNotes. Appl Clin Inform 2019; 10 (01) 10-18
  • 32 Fossa AJ, Bell SK, DesRoches C. OpenNotes and shared decision making: a growing practice in clinical transparency and how it can support patient-centered care. J Am Med Inform Assoc 2018; 25 (09) 1153-1159
  • 33 Leveille SG, Walker J, Ralston JD, Ross SE, Elmore JG, Delbanco T. Evaluating the impact of patients' online access to doctors' visit notes: designing and executing the OpenNotes project. BMC Med Inform Decis Mak 2012; 12: 32
  • 34 Bialostozky M, Huang JS, Kuelbs CL. Are you in or are you out? provider note sharing in pediatrics. Appl Clin Inform 2020; 11 (01) 166-171
  • 35 Adobe XD Ideas. The System Usability Scale & How it's Used in UX. Accessed February 7, 2022 at: https://xd.adobe.com/ideas/process/user-testing/sus-system-usability-scale-ux/
  • 36 Usability.gov. System Usability Scale (SUS). Accessed February 8, 2022 at: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
  • 37 Prabaswari AD, Basumerda C, Utomo BW. The mental workload analysis of staff in study program of private educational organization. IOP Conf Ser: Mater Sci Eng 2019; 528: 012018
  • 38 Walker J, Leveille S, Bell S. et al. Opennotes after 7 years: patient experiences with ongoing access to their clinicians' outpatient visit notes. J Med Internet Res 2019; 21 (05) e13876
  • 39 Panattoni L, Stone A, Chung S, Tai-Seale M. Patients report better satisfaction with part-time primary care physicians, despite less continuity of care and access. J Gen Intern Med 2015; 30 (03) 327-333
  • 40 Alpert JM, Morris BB, Thomson MD, Matin K, Geyer CE, Brown RF. OpenNotes in oncology: oncologists' perceptions and a baseline of the content and style of their clinician notes. Transl Behav Med 2019; 9 (02) 347-356
  • 41 Joseph A, Chalil Madathil K, Jafarifiroozabadi R. et al. Communication and teamwork during telemedicine-enabled stroke care in an ambulance. Hum Factors 2022; 64 (01) 21-41
  • 42 QualtricsXM . The leading experience management software. Accessed November 2, 2021 at: https://www.qualtrics.com/