Best Papers in Human Factors and Sociotechnical Development

Summary

Objectives: To select the best papers that made original and high impact contributions in human factors and organizational issues in biomedical informatics in 2021.

Methods: A rigorous extraction process based on queries from Web of Science® and PubMed/Medline was conducted to identify the scientific contributions published in 2021 that address human factors and organizational issues in biomedical informatics. The screening of papers on titles and abstracts independently by the two section editors led to a total of 3,206 papers. These papers were discussed for a selection of 12 finalist papers, which were then reviewed by the two section editors, two chief editors, and by three external reviewers from internationally renowned research teams.

Results: The query process resulted in 12 papers that reveal interesting and rigorous methods and important studies in human factors that move the field forward, particularly in clinical informatics and emerging technologies such as brain-computer interfaces and mobile health. This year three papers were clearly outstanding and help advance in the field. They provide examples of examining novel and important topics such as the nature of human-machine interaction behavior and norms, use of social-media based design for an electronic health record, and emerging topics such as brain-computer interfaces. thematic development of electronic health records and usability techniques, and condition-focused patient facing tools. Those concerning the Corona Virus Disease 2019 (COVID-19) were included as part of that section.

Conclusion: The selected papers make important contributions to human factors and organizational issues, expanding and deepening our knowledge of how to apply theory and applications of new technologies in health.

Keywords

Human factors - human computer interaction - organizational issues - electronic health records - mobile health - sociotechnical

1 Introduction

Papers published in 2021 on human factors and organizational issues (HFOI) focused on user healthcare technology and data needs, for both patients and physicians. In addition, several papers focused on specific conditions, methodological innovations, and thematic enhancements to electronic health records. Finally, there were other papers that focused on design factors for healthcare systems. Together these papers provide a small cross section of the best HFOI research in biomedical informatics in 2021.

2 Methods

We collected papers from 2021 by a standard search in PubMed and Web of Science®. Searches were performed in January 2022 to identify peer-reviewed journal articles published in the English language in 2021 and related to HFOI research in biomedical informatics. In addition to the search of electronic databases, manual searches of key themes were performed in major biomedical journals (e.g., Journal of the American Medical Informatics Association, Methods of Information in Medicine, Journal of Medical Internet Research, etc.). This resulted in an initial set of 3,206 papers. Independent reading of the titles and abstracts resulted in each co-editor creating a set of 12 candidates, which were then discussed. From this search, we selected 12 best papers; these were sent for additional peer review by a set of external reviewers and two chief editors, with each paper being reviewed by at least four people. We categorized the best papers according to four themes: 1) data presentation for accuracy and understanding; 2) thematic enhancements of electronic health records; 3) methodological innovation; and 4) addressing specific conditions. We then selected the best four papers using these reviews, while also ensuring the papers represented a variety of themes and had strong methods and results.

3 Results

The four best papers are listed in [Table 1]. All reviewers placed these in the top scores. They cover a range of topics with interesting and important methods and conclusions.

Table 1 Best paper selection of HFOI articles for the IMIA Yearbook of Medical Informatics 2022.The articles are listed in alphabetical order of the first author’s surname. *Denotes special consideration.

de Melo et al., [[1]] conducted a fascinating study of how human interaction with machines needs to be studied, given the advent of intelligent systems in everyday life (such as autonomous vehicles) and how COVID-19 experiences shape human altruistic responses to machines. Given the advent of autonomous intelligent systems in daily life, the authors [[1]] correctly claim that more study of how humans can collaborate, and their attitudes and behavior toward machines differs from social norms with humans. For example, normally humans are less likely to apply fairness and altruism norms to machines. However recent events also may have resulted in increased empathy toward others, increased faith in technology, and increased heuristic thinking, given the pandemic, our reliance on technology to handle it, and the overall self-questioning it may have imposed. They make use of the ‘Computers as Social Actors’ theory of Reeves and Nass (1996), which was influential in human computer and robot interaction research. It argues that people heuristically treat machines like people, and that encouraging intuitive thinking, in contrast to deliberation, led to increased cooperation in non-strategic settings. The authors are the first to apply and test this with concrete cognitive studies. The dictator game is used to measure altruism; the user has options to give tokens to another user (in this case the computer or a ‘human’ (both delivered by computer message to obscure the source). 186 participants were used as senders, across 40 US states, and provided a diverse sample. They were administered the abbreviated Post-Traumatic Stress Disorder (PTSD) checklist (to measure COVID-19 impact), and three subjective scales to gain insight on mechanisms. These were the Cognitive Reflection test to measure if those impacted engage in reduced reflection, i.e., more intuitive thinking, the Faith in Technology scale, and the Moral Foundations Questionnaire. Results showed a reduction in the usual bias against fairness toward machines the more the user had been impacted by COVID-19. There were also sharp increases in intuitive (and incorrect) thinking and faith in technology among the most highly affected group. The authors through multiple mediation analysis showed that faith in technology and heuristic thinking mediate the offer bias. They also caution that in times of stress the disproportional impact of Covid19 on vulnerable groups leads to the need for ethical guidelines and regulations to ensure altruism/cooperation shown to machines is well deserved. They also point out the factors such as individual stress propensity, education level, and socioeconomic status could make individuals susceptible to heuristic thinking, and other social norms such as reciprocity, trust and fairness may also shape collaboration with machines. The paper is a fascinating and rigorous look at the issue and given increased development of intelligent systems for the public, including in healthcare, a study of these factors is timely and well done.

It is a commonplace that loss of one sense is often compensated by gain in the acuity of the other senses: the ability of many blind people to “read” Braille through their fingertips is but one outstanding example. Osinski et al., [[2]] observe that while many sensory deficits have been addressed, little has been done to mitigate the colorless world of the blind and embarked on the design of a system that gives sound to color and creates a stereophonic soundscape to convey the symphony of color that sighted people take for granted. Sonification originates from attempts to transform data to sound, much as visualization methods turn data into images.

The paper of Thayer et al., [[3]] is an exemplary exposition of a project to integrate and visualize urgently needed patient information for a pediatric emergency department. The emergency room at the Children’s Hospital of Philadelphia, a highly regarded tertiary care health system, treats more than 7,000 acute asthma cases each year, with an admission rate of 30%. The emergency clinical team’s work is intense and high stress, with distressed young patients needing urgent attention. Many of these children may be on the second or third visit to the Emergency Department (ED) in a year. The project reported here began with an observation by the lead informatics physician: the complexity of screen navigation involved in collecting all the vital information needed by a clinician to treat an acute asthma exacerbation made the process very cumbersome and time consuming. Notwithstanding the immediate team’s understanding of the problem, they did the right thing and stepped back to assess whether this was—and precisely what was—a problem for the users of the system. They framed this as a problem for human-centered design, i.e., prioritization of the users’ needs over other considerations. Clearly, good design, development, and implementation were paid full attention at the right time. The approach entailed observation of the emergency clinicians’ workflow and inclusive semi-structured interviews with every role and grade represented. This aimed to ensure that real needs could be translated into system requirements that would fit in with, rather than force changes to, the natural process of information gathering and patient assessment. It was established that the focus had to be high- and rising-risk patients and that encapsulating the patient’s history visually would be the most effective way to deal with the implied information burden. A staged development process through cognitive task analysis, design, prototype evaluation, intervention development, and post-implementation continuing feedback, ensured a smooth transition from concept to reality. Within each of these stages additional work was identified and built into the program. For example, in intervention development, the team had to take account of features yet to be delivered by the electronic health record (EHR) vendor and, concurrently, the evolving specification of Fast Healthcare Interoperability Resources (FHIR, an emerging interoperability standard), leading finally to a contribution adopted by the SMART framework. Evidence from post-implementation surveys of emergency clinicians indicates broad acceptance and adoption of the new visualization app.

The fourth best paper by Sridharan et al., [[4]] is an outstanding description of a “microblogging” system designed for rapid summary communication between clinicians to reduce EHR burden. Great Ormond Street Hospital for Children in London is the premier tertiary pediatric hospital in the UK. Recognizing that comprehensive documentation of patient information in the EHR is bought at the price of complexity in structure and navigation, and taking a hint from the success of platforms such as Twitter, a senior team at the hospital promoted a parallel development to the EHR in the form of the “Essence” microblogging application. Based on a maximum length of 156 characters, the platform constrains the user and encourages brevity and succinct expression. These brief summaries are, perhaps surprisingly, highly effective in conveying the essence of the patient’s condition. Indeed, it is observed that a “stack’ of microblogs over time or across disciplines provides an excellent picture of the patient’s journey. Intriguingly, development of the microblogging app took place at the same time as the hospital’s major EHR rollout—a counterintuitive concept in most settings. The development and meshing process are not described in detail, but there is ample evidence of the effectiveness of the solution. In a year, over 90,000 Essence notes were filed across 49 specialties, with cardiology and neurosurgery ahead of pediatric intensive care in volume. Nursing staff entered more and longer notes (56% of the total, with median length of 14 words vs. 12 for physicians). Physicians tend to mention diagnoses while nurses mainly note procedures—a glimpse of hospital anthropology, perhaps. The authors do not attempt to compare the content of the microblogs vs. the full EHR record. This is not the only attempt to overcome documentation complexity and the concomitant phenomenon of physician burnout. The American Medical Informatics Association’s program “25 x 5” has been instituted to reduce documentation burden to 25% of the current level over the next 5 years. Others are attempting to voice-record and store “one minute” notes. Whichever method, or methods, succeed, what is beyond doubt is the undeniable fact of excessive documentation burden and its toll on physicians—and all these initiatives deserve to succeed.

4 Additional Themes and Emerging Trends

The finalists included papers which were all excellent, with strong methods and notable findings. Although not selected as best papers, the remaining candidate best papers were just as excellent and influential. We summarize them below, organized by the three major themes emerging: 1) thematic enhancements of electronic health records, 2) methodological innovation; and 3) addressing specific conditions.

4.1. Enhancements of Electronic Health Records

Genomics and EHRs

The use of genomic information in clinical decision support (CDS) is still relatively new. Its integration in clinical workflows necessitates analysis that is best accomplished in the context of phenotypic, i.e. EHR-based, information. The project reported here [[5]] utilized an existing artificial intelligence (AI)-based chart review tool to bring the two sets of information together in an experiment to assess the validity of such an approach in prototypical use by clinical domain experts in Mendelian genetic disorders. The acceptability of such solutions is critical to the success of genomics-based CDS.

Patient Values and Reported Outcomes in EHRs

What is my patient like as a person? What are her values, goals, and preferences? In times of crisis, this information can make an enormous difference in how, and so how well, the patient is treated. The ghastly isolation of patients dying of COVID-19 is a telltale reminder. More recently, the clever inference of the first monkeypox case in a US traveler from Canada, through careful attention to the details of his story, should be another. In their paper [[6]], the team at Memorial Sloan Kettering Cancer Center (MSK) reports on a framework for oncology nurses to work with newly diagnosed patients to determine their wishes for care at the end of life, but also broader personal values, such as sources of strength and hope, their concerns, and their idea of an acceptable quality of life. These providers use a structured template to document their discussions of patient goals of care which are then available in an EHR Patient Values Tab.

4.2 Methodological Innovation

Mator et al., [[7]] dressed an interesting overview of how usability testing has been adopted, measured, and used in different subdomains, such as telehealth, EHRs, special patient populations, e.g., Autism Spectrum disorder. It describes the usage of testing, usability concepts, how these have been employed in the different subdomains, and some of the notable results. It provides some recommendations for future work.

“Design Thinking” has proved successful in domains, such as healthcare, where problems may arise from conflicting requirements among legitimate stakeholder participants. The selected paper [[8]] by Novak et al., represents an in-depth assessment of the value of Project Health Design, a Robert Wood Johnson Foundation-sponsored study of design thinking for personal health records (PHR).

Non-pharmacological management of pain has attracted its share of attention in the wake of problems associated with the alternative, especially the risk of opioid overprescription, dependence and abuse. The efficacy of VR-based methods has been established in well-resourced settings. The implementation science-inspired study by Sarkar et al., [[9]] explored the method in less advantaged and safety-net settings. Contextual factors, such as the appropriateness of content to diverse populations, played an important part in its success.

4.3 Addressing Specific Conditions

Creation of apps for teaching the autistic is a specialized challenge. Ntalindwa et al., [[10]] describe a successful App development and testing for autistic children in Rwanda, many of whom do not have access to regular schooling due to stigma. The paper describes use of the App depicting local coins and drag/drop tasks to motivate children to learn arithmetic by dropping the correct denominations in combination. It provides a useful example of how to meet and test highly specialized needs with due attention to both unusual aspects of the task and culture-specific requirements.

The study by Norris et al., [[11]] describes successful adaptation of an app originally directed at Latina teens, to promote healthy behaviors and role playing around sexual behavior and social media. It nicely describes the issues in expanding scope of an app to a larger multicultural audience, and with gaining parental approval for topics which are often controversial or disapproved by parents. It adds to our knowledge about how to expand successful pilots to a far broader audience.

A large (4,159 subject) study of adoption of wearables and data sharing with providers contradicts the findings of previous such studies [[12]]. Those with diabetes, hypertension and multiple chronic conditions were significantly more likely to use wearables, and as one might expect, those with a consistent usual source of care were more likely to share their data with providers. 29% of respondents used wearables and of those, half shared data with a provider. Social determinants of health including income affected data sharing in a similar manner as with other consumer health technologies. This is notable as it indicates wearable adoption and data sharing are more common than prior studies found, despite digital disparities. The authors call for further studies of how to expand their use in care delivery.

The use and usefulness of health- and wellbeing-related apps depend on a variety of criteria. These include the trust providers are willing to place on data from a “wearable” device—noting that this is itself a somewhat slippery term—as well as the wearer’s sense of privacy threat from sharing of data that is often location-sensitive, and so possibly identifying, even after “de-identification” has been applied. The first of these two studies (i.e., the app addressed to wearables) suggests that adoption is increasing with evidence of trust in the sharing of data with providers. This provides early evidence of increasing reliance on remote monitoring, especially of patients with specific conditions, such as hypertension, type II diabetes, heart failure, and metabolic syndrome.

In the second study [[13]] a number of established frameworks are combined to explore the persuasive and trust factors that impinge on sustained use of Boohee, the most popular diet management app in China. Methods used include the unified theory of acceptance and use of technology, the health belief model, the theory of self-control (dual planner/doer model), and risk perception of information technology. Motivation, behavioral intention, and trust (perception of security) each plays its part, with implications for health app promotion.

How may an app aimed at an aging demographic adapt to the motivations, needs and functional capabilities of its potential users? A broad methodological review protocol [[14]] proposes to explore studies in English, Spanish and Portuguese with a view to assessing their support of physical activity.

Trust in scientific studies depends to a considerable extent on the ability of the reader to make sense of methods and findings. Wang et al., [[15]] take inspiration from the success of comics to adapt the approach to the presentation of data. It exemplifies its method through a review of controlled studies in Human Computer Interaction, making it almost into a reflexive study of methodology. The possibility to extend the approach to controlled studies in other domains presents an interesting prospect.

The importance of social determinants of health (SDoH) is well established, notwithstanding their patchy acquisition and storage in electronic health records. There are many instruments for the collection of SDoH, but the process remains delicate, not least because of the personal and potentially embarrassing information they elicit. The question arises, on resource grounds, if for no other reason, whether the elicitation of SDoH from all or only from some patients is the most appropriate option. The study by Grus I et al., [[16]] is a well-planned study that formed part of a larger project and analyzed the presence of external pressure, perceptions, motivations, workflow adaptations, and advocacy for SDoH collection of 43 health care staff and professionals from eight community health centers in five different US states. It provides a good organizational foundation for future undertakings in the collection of SDoH.

5 Adaptation of Technology to Human Use

We conclude with two papers [[16], [17]] that approach technical issues, respectively, from a fundamental point of view and from a human design perspective.

Hramov et al., [[16]] outlines the principles, challenges, and suggested solutions to problems arising in attempts to interface human thought to physical action. The problem is of eminent applicability, with the needs of individuals with motor disabilities uppermost for consideration, although the prospect of brain to brain direct interaction is also examined.

How are spinal surgeons best served by a tool to help them align a screw at exactly the right angle? It is natural to consider a visual display that does not force the surgeon to divert their gaze from the site of intervention. A small display fitted on the tool itself is designed and tested with a number of surgeons to demonstrate its efficacy in the study of Schütz et al., [[18]].

6 Conclusion

The selected papers make important contributions to human factors and organizational issues, expanding and deepening our knowledge of how to apply theory and applications of new technologies in health.

No conflict of interest has been declared by the author(s).

Acknowledgements

We would like to acknowledge the support of Adrien Ugon, Martina Hutter, Kate Fultz Hollis, Lina Soualmia, and the whole Yearbook Editorial Committee as well as the reviewers for their contribution to the selection process of the Human Factor and Organizational Issues section for the IMIA Yearbook 2022.

Section Editors for the IMIA Yearbook Section on Human Factors and Organizational Issues

• References

• 1 de Melo CM, Gratch J, Krueger F. Heuristic thinking and altruism toward machines in people impacted by COVID-19. iScience 2021 Mar 19;24(3):102228.
• 2 Osinski D, Lukowska M, Hjelme DR, Wierzchon M. Colorophone 2.0: A Wearable Color Sonification Device Generating Live Stereo-Soundscapes-Design, Implementation, and Usability Audit. Sensors (Basel) 2021 Nov 5;21(21):7351.
• 3 Thayer JG, Ferro DF, Miller JM, Karavite D, Grundmeier RW, Utidjian L, et al. Human-centered development of an electronic health record-embedded, interactive information visualization in the emergency department using fast healthcare interoperability resources. J Am Med Inform Assoc 2021 Jul 14;28(7):1401-10.
• 4 Sridharan S, Peters C, Newcombe S, Jephson C, Robinson R, Mulder B, et al. The essence of healthcare records: embedded electronic health record system microblogging functionality for patient care narrative. Future Healthc J 2021 Nov;8(3):e709-e713.
• 5 Kulchak Rahm A, Walton NA, Feldman LK, Jenkins C, Jenkins T, Person TN, et al. User testing of a diagnostic decision support system with machine-assisted chart review to facilitate clinical genomic diagnosis. BMJ Health Care Inform 2021 May;28(1):e100331.
• 6 Desai AV, Michael CL, Kuperman GJ, Jordan G, Mittelstaedt H, Epstein AS, et al. A Novel Patient Values Tab for the Electronic Health Record: A User-Centered Design Approach. J Med Internet Res 2021 Feb 17;23(2):e21615.
• 7 Mator JD, Lehman WE, McManus W, Powers S, Tiller L, Unverricht JR, et al. Usability: Adoption, Measurement, Value. Hum Factors 2021 Sep;63(6):956-73.
• 8 Novak LL, Harris JW, Koonce TY, Johnson KB. Design thinking in applied informatics: what can we learn from Project HealthDesign? J Am Med Inform Assoc. 2021 Aug 13;28(9):1858-65.
• 9 Sarkar U, Lee JE, Nguyen KH, Lisker S, Lyles CR. Barriers and Facilitators to the Implementation of Virtual Reality as a Pain Management Modality in Academic, Community, and Safety-Net Settings: Qualitative Analysis. J Med Internet Res 2021 Sep 22;23(9):e26623.
• 10 Ntalindwa T, Nduwingoma M, Karangwa E, Rashid Soron T, Uworwabayeho A, Uwineza A. Development of a Mobile App to Improve Numeracy Skills of Children With Autism Spectrum Disorder: Participatory Design and Usability Study. JMIR Pediatr Parent 2021 Aug 31;4(3):e21471.
• 11 Norris AE, Thalasinos RD, Hecht ML. Multicultural Adaptation of Mighty Girls for Widespread Dissemination: Pilot Study, App Development and Usability Testing, and Gauging Parent Support With Focus Groups. JMIR Form Res 2021 Jun 2;5(6):e24937.
• 12 Turner K, Jo A, Wei G, Tabriz AA, Clary A, Jim HSL. Sharing patient-generated data with healthcare providers: findings from a 2019 national survey. J Am Med Inform Assoc 2021 Feb 15;28(2):371-6.
• 13 Wei J, Vinnikova A, Lu L, Xu J. Understanding and Predicting the Adoption of Fitness Mobile Apps: Evidence from China. Health Commun 2021 Jul;36(8):950-61.
• 14 Lorenzi LJ, Belo LF, Frohlich DM, Dourado VZ, Castro PC, Gomes GAO. Factors related to the adoption and adherence of physical activity mobile applications by older people: a scoping review protocol. BMJ Open 2021 Oct 8;11(10):e052414.
• 15 Wang Z, Ritchie J, Zhou J, Chevalier F, Bach B. Data Comics for Reporting Controlled User Studies in Human-Computer Interaction. IEEE Trans Vis Comput Graph 2021 Feb;27(2):967-77.
• 16 Gruß I, Bunce A, Davis J, Dambrun K, Cottrell E, Gold R. Initiating and Implementing Social Determinants of Health Data Collection in Community Health Centers. Popul Health Manag 2021 Feb;24(1):52-8.
• 17 Hramov AE, Maksimenko VA, Pisarchik AN. Physical principles of brain–computer interfaces and their applications for rehabilitation, robotics and control of human brain states. Phys Rep 2021(918):1-133. Available from: https://doi.org/10.1016/j.physrep.2021.03.002
• 18 Schütz L, Brendle C, Esteban J, Krieg SM, Eck U, Navab N. Usability of Graphical Visualizations on a Tool-Mounted Interface for Spine Surgery. J Imaging 2021 Aug 21;7(8):159.

Publication History

Article published online:
04 December 2022