PDF icon Download PDF
Open Access
CC BY-NC-ND 4.0 · Yearb Med Inform 2018; 27(01): 079-082
DOI: 10.1055/s-0038-1667074
Section 2: Human Factors and Organizational Issues
Synopsis
Georg Thieme Verlag KG Stuttgart

Findings from the 2018 Yearbook Section on Human Factors and Organizational Issues

Sylvia Pelayo
1   Univ. Lille, INSERM, CHU Lille, CIC-IT /Evalab 1403 - Centre d'Investigation clinique, EA 2694, Lille, France
,
Johanna Kaipio
2   Department of Computer Science, Aalto University, Espoo, Finland
,
Section Editors for the IMIA Yearbook Section on Human Factors and Organizational Issues› Author Affiliations
Further Information

Correspondence to

S. Pelayo
INSERM CIC-IT 1403 Evalab, CHU Lille, UDSL EA 2694, Lille University
F-59000, Lille
France   

Publication History

Publication Date:
29 August 2018 (online)

Also available ateRef
 
 PDF Download Permissions and Reprints

Summary

Objective: To summarize significant research contributions published in 2017 on Human Factors and Organizational Issues (HFOI) in medical informatics.

Methods: An extensive search using PubMed/Medline and Web of Science® was conducted to identify the scientific contributions published in 2017 that HFOI issues in medical informatics. The selection process comprised three steps: (i) 15 candidate best papers out of 695 references were first selected by the two section editors, (ii) external reviewers from internationally renowned research teams reviewed each candidate best paper, and (iii) the final selection of five best papers was conducted by the editorial board of the Yearbook.

Results: The five best papers offer a glimpse of the quality and breadth of the work being conducted in the HFOI community.

Conclusion: The selection of the HFOI section of the 2018 IMIA Yearbook highlights a growing number of high quality studies. There are especially more studies interested in testing Human Factors and Ergonomics methods and demonstrating the benefits.


 

Keywords

Human factors - ergonomics - health information technology - usability - organizational issues

Introduction

In 2018, we reused the optimized query developed for the Human Factors and Organizational Issues (HFOI) section in the 2017 edition of the International Medical Informatics Association (IMIA) Yearbook. The number of retrieved papers decreased but the selection highlighted a growing number of high quality studies, based on solid well-described methodologies. The selection process of the final 15 candidate best papers has been much more arduous with more studies meeting the IMIA Yearbook scientific quality requirements. Many studies are concerned by grasping the concepts used, refining them compared to previous studies, or using methods to make more direct links between Health Information Technology (HIT) dysfunctions/problems and consequences for patients. A group of studies emerged which aim was to evaluate with established methods, even including randomized trials, the impact of a human factors interventions.

The five selected papers offer a glimpse of the quality and breadth of the work being conducted in the HFOI community.


 

About the Paper Selection

Two electronic databases were searched, PubMed/Medline and Web of Science®. Searches were performed in October and December 2017 to identify peer-reviewed journal articles published in 2017, in the English language, related to HFOI research in medical informatics. In addition to the search of electronic databases, manual searches of key themes were performed in seven major biomedical journals (Journal of the American Medical Informatics Association, Methods of information in medicine, Journal of Medical Internet Research, International Journal of Medical Informatics, Journal of Biomedical Informatics, BMC Medical Informatics and Decision Making, Informatics for health and social care).

Keywords used in the searches included both free-text and coded keywords. Consequently, two queries were built: one based on MeSH terms used as major topic in the Pubmed/Medline database, the second one based on free-text keywords searched in titles or abstracts through Pubmed/ Medline and Web of Science® databases. Pubmed was queried to test keywords in an iterative process.

One of the two section editors performed the literature search. Database searches yielded a total of 695 references. The two section editors independently undertook the initial screening of titles and abstracts to identify papers relevant to the field of interest. Papers were considered according to their originality, innovativeness, scientific and/or practical impact, and scientific quality. Both section editors classified the papers into three categories: accepted, rejected, or pending. They then reviewed in detail the accepted and pending full-text articles to reach two lists of 18 and 20 candidate papers, respectively. The two lists were then pooled. Eight articles were identical and were kept for further evaluation. The remaining 22 papers were reviewed in detail to get to a consensual list of seven candidate papers, and then a final list of 15 candidate best papers.

In accordance with the IMIA Yearbook selection process, the 15 candidate best papers were evaluated by the two section editors and by additional external reviewers (at least four reviewers per paper). Five papers were finally selected as best papers ([Table 1]). A content summary of the selected best papers can be found in the appendix of this synopsis.

Table 1

Best paper selection of articles for the IMIA Yearbook of Medical Informatics 2018 in the section ‘Human Factors and Organizational Issues'. The articles are listed in alphabetical order of the first author's surname.

Section

Human Factors and Organizational Issues

▪ Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R, with the HITEC Investigators. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak 2017 Apr 10;17(1):36.

▪ Blijleven V, Koelemeijer K, Wetzels M, Jaspers M. Workarounds emerging from electronic health record system usage: consequences for patient safety, effectiveness of care, and efficiency of care. JMIR Hum Factors 2017 Oct 5;4(4):e27.

▪ Cresswell KM, Mozaffar H, Lee L, Williams R, Sheikh A. Safety risks associated with the lack of integration and interfacing of hospital health information technologies: a qualitative study of hospital electronic prescribing systems in England. BMJ Qual Saf 2017 Jul;26(7):530-41.

▪ Dufendach KR, Koch S, Unertl KM, Lehmann CU. A randomized trial comparing classical participatory design to VandAID, an interactive crowdsourcing platform to facilitate user-centered design. Methods Inf Med 2017 Oct 26;56(5):344-9.

▪ Luna DR, Rizzato Lede DA, Otero CM, Risk MR, González Bernaldo de Quirós F. User-centered design improves the usability of drug-drug interaction alerts: experimental comparison of interfaces. J Biomed Inform 2017 Feb;66:204-13.

 

Conclusions and Outlook

A number of high quality, interesting, and well-written articles were identified. Out of them, five papers highlighting aspects of particular importance for the HFOI research were finally selected as best papers.

Three papers address concepts central to HFOI, and widely studied in biomedical informatics. These papers significantly contribute in bringing new insights to the HFOI field.

Blijleven et al. [1] wrote an excellent paper on an important topic, the workarounds emerging from Electronic Health Record (EHR) Systems. Based on a solid method that produced meaningful and useful results, insights gained from knowing a workaround's degree of influence as well as its impact on patient safety, effectiveness of care, and efficiency of care can today inform the design and redesign of electronic health record (EHR) systems to further align EHR design with work contexts, subsequently leading to better organization and provision of care.

Ancker et al. [2] conducted a well-done hypothesis-driven study. They contributed to alert fatigue phenomenon by testing two possible mechanisms: (i) the cognitive overload associated with the amount of work, complexity of work, and effort distinguishing informative and uninformative alerts, and (ii) the desensitization from repeated exposure to the same alert over time. The results are promising for reducing alert overrides and alert fatigue.

Cresswell et al. [3] performed an impressive longitudinal qualitative study addressing the thorny question for hospitals of either pursuing a strategy of purchasing standalone systems and then interfacing them, or of buying hospital-wide multi-modular systems. The results provided good insights to support hospitals decisions. While multi-modular systems offered somewhat better usability, standalone systems provided greater flexibility and opportunity for innovation, particularly in relation with interoperability with external systems and in relation with customizability to the needs of different users’ groups.

The two other papers are focused on the evaluation of specific and innovative methods of User-Centered Design (UCD).

Dufendach et al. [4] provided a very interesting solution to facilitate end users involvement in the design of HIT solutions. The authors developed a web-based crowdsourcing platform that allows responsive remote customization and feedback of a visual user interface, the VandAID platform. They evaluated the platform with the robust design of a randomized study: their platform was compared to classical participatory design. The use of the VandAID tool was significantly faster and less onerous to providers and researchers while yielding similar results.

Luna et al. [5] performed an experimental comparison of two interfaces: a standard one developed under traditional techniques and another one, generated using participatory UCD methods. The results indicated that the system developed following UCD was more efficient, more effective, and more satisfying.

Although not selected as best papers, the remaining 10 candidate best papers were equally interesting.

Brox et al. [6] shared the lessons learned of three years of experience with seniors involved in design and provided a UCD protocol tailored to senior needs. Petersen et al. [7] proposed a paper based on a well-described Design Thinking methodology to support the development and testing of a mobile app supporting diabetes self-management. Horsky et al. [8] investigated the accuracy of different medication reconciliation tools using a cognitively demanding scenario and complex medication history. They showed that an accurate assessment of safety and effectiveness of electronic reconciliation tools requires rigorous testing and should prioritize complex rather than simpler tasks that are currently used for EHR certification and product demonstration. Lyell et al. [9] found evidence of the automation bias in e-prescribing. They tested the impact of task complexity and interruptions on the automation bias. Although the study group was not representative of typical practitioners but of trainees, the authors provided a very nice methodology that may be reused for those who are interested in analyzing the automation bias. Minotra et al. [10] supported a better understanding of the effects of standardization on medical device usability. They proposed a protocol to experimentally examine the impact of standardized nomenclature on performance with the use of an unfamiliar ventilator product. Russ et al.[11] presented a feasibility study to demonstrate the effectiveness of a method to evaluate the prescribers’ information processing of medication alert display. The method supplements traditional usability evaluation methods and may be useful for evaluating information processing of other healthcare technologies.

Two studies addressed the benefits of usability methods and provided interesting results. Gonser et al. [12] examined the costs and possible benefits of usability testing for hospitals before buying new medical devices for theatre. They succeeded to demonstrate that not only hospitals could economically benefit from investing in a usability testing before deciding to buy a medical device, but patients would also profit from a higher usability that reduce possible operator errors and increase safety and performance of use. Richardson et al. [13] aimed to understand the facilitators of usability and to evaluate the types of additional information gained from proceeding “Near Live” testing after completing a “Think Aloud” protocol. “Think Aloud” and “Near Live” usability testing provide designers with complementary insights. Combining these types of usability testing allows the tools to be rigorously evaluated and adapted to the needs of users.

Finally, Murphy et al.[14] addressed issues related to patient-related information problems (PIP), well-known to lead to workflow challenges, delayed patient-care decisions, and negative impacts on the patient. They employed qualitative data collection methods to deeply analyze the PIP and then discussed socio-technical recommendations for organizational policies and training, as well as EHR design improvements. Snowden et al.[15] performed a mixed-method approach to collect data on the usability of an EHR, staff engagement, and staff experience at four time points spanning 30 months from the inception of the system. The authors stressed that managing properly an EHR implementation process requires a deep understanding of the implementation process itself. Whilst the technical challenges are reasonably well understood, both the nature of the complexity of the implementation process and the time taken for the organization to begin recovery from the challenges are misunderstood.


 
Summary of Best Papers Selected for the 2018 Edition of the IMIA Yearbook, Section Human Factors and Organizational Issues

Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R, with the HITEC Investigators

Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system

BMC Med Inform Decis Mak 2017 Apr 10;17(1):36

Although alert fatigue is blamed for high override rates in contemporary clinical decision support systems, the concept of alert fatigue is poorly defined. This study tests two possible alert fatigue mechanisms: (i) cognitive overload associated with a high amount of work, complexity of work, and efforts distinguishing informative from uninformative alerts, and (ii) desensitization from repeated exposure to the same alert over time. The results show that clinicians became less likely to accept alerts as they received a lot of them, particularly in the case of repeated alerts. There was no evidence of an effect of workload per se, or of desensitization over time for a newly deployed alert.

Blijleven V, Koelemeijer K, Wetzels M, Jaspers M

Workarounds emerging from electronic health record system usage: consequences for patient safety, effectiveness of care, and efficiency of care

JMIR Hum Factors 2017 Oct 5;4(4):e27

Health care providers resort to informal temporary practices known as workarounds for handling exceptions to normal workflow unintendedly imposed by electronic health record systems (EHRs). Research on the scope and impact of EHR workarounds on patent care processes is scarce. Based on a large case study, the authors present an overview of 15 bottom-up identified rationales for EHR workarounds and give a definition for each rationale. In addition, for the most prominent workarounds, their scope and impact on patient safety, effectiveness of care, and efficiency of care are discussed from a sociotechnical perspective using the Systems Engineering Initiative for Patient Safety (SEIPS) framework.

Cresswell KM, Mozaffar H, Lee L, Williams R, Sheikh A

Safety risks associated with the lack of integration and interfacing of hospital health information technologies: a qualitative study of hospital electronic prescribing systems in England

BMJ Qual Saf 2017 Jul;26(7):530-41

Mitigating safety risks in health information technology is highly dependent on the effective integration of information within systems and/or interoperability to allow information exchange across systems. The paper explores the social and technical challenges relating to integration and interfacing experienced by early adopter hospitals of standalone and hospital-wide multi-modular integrated electronic prescribing systems. Based on a longitudinal qualitative study, the results highlight that while multi-modular systems offer somewhat better usability, standalone systems provide greater flexibility and opportunity for innovation, particularly in relation to interoperability with external systems and to customizability to the needs of different user groups.

Dufendach KR, Koch S, Unertl KM, Lehmann CU

A randomized trial comparing classical participatory design to VandAID, an interactive crowdsourcing platform to facilitate user-centered design

Methods Inf Med 2017 Oct 26;56(5):344-9

Early involvement of stakeholders in the design of medical software is particularly important due to the need to incorporate complex knowledge and actions associated with clinical work. Standard user-centered design (UCD) methods may limit user involvement to a small number of individuals due to the significant time investment from designers and end users. VandAID, a new web-based crowdsourcing platform, was tested in a randomized trial. The results show that VandAID can simultaneously involve multiple users in UCD and provides means of obtaining design feedback remotely.

Luna DR, Rizzato Lede DA, Otero CM, Risk MR, González Bernaldo de Quirós F

User-centered design improves the usability of drug-drug interaction alerts: experimental comparison of interfaces

J Biomed Inform 2017 Feb;66:204-13

Clinical Decision Support Systems can alert health professionals about drug-drug interactions when they prescribe medications. But the alert override rate of this kind of system is very high. This paper describes the methodology of a User-Centered Design (UCD) that goes beyond UCD and cooperative design approaches to include end users as active participants in the design and decision-making. The authors tested a crossover method for scientifically compare the usability of an interface designed with standard method with an interface designed with a participatory UCD in terms of efficiency, effectiveness, and user satisfaction.


 

Acknowledgement

We would like to acknowledge the support of Adrien Ugon, Martina Hutter, John H. Holmes, Brigitte Séroussi and the whole Yearbook editorial team as well as the reviewers for their contribution to the selection process of the Human Factor and Organizational Issues section of the 2018 IMIA Yearbook.


Correspondence to

S. Pelayo
INSERM CIC-IT 1403 Evalab, CHU Lille, UDSL EA 2694, Lille University
F-59000, Lille
France   


  Permissions and Reprints