Human Factors and Organizational Issues in Health Informatics: Innovations and Opportunities

Keywords

Human factors and ergonomics - human centered design - user computer interface - health informatics - health information technology

1 Introduction

The goal of Human Factors and Ergonomics (HF/E) is to “improve people's lives by making technology work well for them”, rather than making individuals adapt their behaviors to accommodate how designers have created technologies [[1]]. There is now broad recognition that health informatics tools must account for the diverse needs, characteristics, and abilities of end users, as well as their context of use. As described in this synthesis, HF/E frameworks and methods are becoming integrated into the health informatics research and practice community.

Health informatics researchers and practitioners continue to broaden their application of HF/E methods beyond the evaluation and critique of commercial electronic health records (EHRs). Because recent manuscripts have synthesized current knowledge related to the implications of poor EHR design [[2], [3]], this manuscript focuses largely on the novel and exciting work being carried out in other application areas. The pervasiveness of mobile health technologies and the ability to customize EHR functions and displays significantly expands the opportunity to apply HF/E methods during the design of new health informatics technologies. We see evidence of the significant growth in research within these domains based on the numerous recent manuscripts published on these topics.

This manuscript is divided into three sections to help identify the scope of recent HF/E-focused health informatics research and allow us to define important HF/E gaps that the health informatics community needs to address. We describe where current health informatics publications fit into the HF/E design process, and the focus of HF/E-focused health informatics efforts with respect to types of technologies being developed and outcomes being measured. We then synthesize these findings and describe opportunities for future HF/E-focused health informatics research.

2 Methods

To understand the current nature and scope of integration of the HF/E and health informatics research communities, we manually reviewed all manuscripts published in several primary HF/E and health informatics journals and conference proceedings during 2020, including 2021 publications with available preprints. For the HF/E community, this review included the journals Human Factors, Ergonomics, Applied Ergonomics, and Ergonomics in Design, and the proceedings from the Human Factors and Ergonomics Association Annual Meeting and International Symposium on Human Factors and Ergonomics in Health Care. For the health informatics community, this review included the journals the Journal of the American Medical Informatics Association (JAMIA), JAMIA Open, Applied Clinical Informatics (ACI), ACI Open, International Journal of Medical Informatics (IJMI), the Journal of Biomedical Informatics (JBI), and the proceedings of the American Medical Informatics Association (AMIA) Annual Symposium. Each publication was coded according to: 1) the HF/E design cycle phase addressed (e.g., understand, create, evaluate), 2) the type of technology studied (e.g., EHR, CDS customization, mobile app, …etc.); and 3) the HF/E outcome measured (e.g., performance, safety, satisfaction).

3 Results

3.1 Diversity in Design Phase

While the design approaches used in the reviewed studies are diverse, they all fit into the simplified HF/E design cycle model shown in [Figure 1] [[1]]. We therefore use this design cycle model to describe the diverse body of recent HF/E-focused health informatics research included in this synthesis. The studies included in this synthesis all addressed some subset of the understanding phase of the design cycle, some description of the creation phase, and/or various evaluation efforts they employed in their work. In doing so, HF/E-focused health informatics research seeks to understanding user needs (both clinician and patient), and/or the creation and evaluation of novel health informatics tools to address those needs.

Fig. 1 HF/E Design Cycle [[1]].

Numerous HF/E design frameworks and models used by researchers and practitioners within the health informatics community align with the HF/E design cycle shown in [Figure 1], including human-centered design (HCD), user-centered design (UCD), and the system development life cycle. HCD, defined by ISO, is both process- and outcome-focused, defined as an “approach to systems design and development that aims to make interactive systems more usable” [[4]]. HCD typically includes three general phases that are aligned with the HF/E design cycle: Inspiration (Understand), Ideation (Create), and Implementation (Create + Evaluate) [[5]]. UCD, while related to HCD, focuses more tightly on the needs of the specific end users of a design, and is more commonly used in health informatics research [[4]]. UCD consists of iterative design cycles that involve understanding of end user needs (Understand) and designing and iteratively refining prototypes with close involvement of end users (Create) before deploying final designs (Create + Evaluate). The Office of the National Coordinator of Health Information Technology (ONC) now requires a “user-centered design processes to be applied to EHR technology that includes certain capabilities” [[6]]. The exact UCD process is not prescribed by ONC, but resources exist from organizations such as NIST to provide guidance on UCD process with respect to EHRs [[7]]. Yet, one recent comprehensive study showed that vendors vary significantly in the quality of their UCD practices, ranging from well-developed UCD processes to fundamental misconceptions of the UCD process [[8]]. Several studies included in this synthesis addressed the value of following a user- or human-centered design approach broadly, either conducting UCD or HCD in their own work or noting the importance of these approaches in health informatics design [[2], [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17]]. However, for the reasons above, readers of HF/E-focused health informatics manuscripts should be careful to interpret the authors’ use of the (often confused) terms HCD and UCD, and not assume their design process is of high quality. Other manuscripts synthesize how user perspectives and needs can be integrated into various stages of a health informatics-focused versions of the systems development life cycle (SDLC), including project planning, analysis (Understand), design of the system (Create), implementation (Create + Evaluate), and system support/maintenance (Evaluate) [[18], [19]].

Shown in [Table 1], the bulk of studies included in this synthesis focused on understanding user needs or on system evaluation efforts, but there was also a significant amount of work aimed at and creating technologies to meet user needs. These technology creation efforts may be driven by a recent shift in design capability to a broader community beyond EHR vendors, as the technology-focus portion of this synthesis describes the large volume of recent work focused on EHR customizations and mobile app development.

Table 1 HF/E design cycle phase addressed, and methods used in studies.

HF/E Design Cycle Phase	Understand	[9,17,20-51]
	Create	[9,11,15,30,41,46,50-70]
	Evaluate	[9,11,14,15,17,30,41,46-49,52-59,66-126]
	Heuristic analysis	[16,23,41,62,69,70,121]
	Observation	[20,22,28,32,33,36,45,72,74,93,120,124]
	Focus groups	[7,9, 12,15,19,22,26,31,34,41,51,55,91,100,107,108,111, 118,122,127-133]
	Cognitive walkthroughs	[70,121]
Study Methods	Interviews	[11,18,20,24-26,29,31-34,36,37,47,48,51,52,55,57,58, 70,74,78,79,82-84,88,93,99,112,114,117,119,120,122, 126,134]
	Structured usability testing	[7,16,18,66,67,69,120,132]
	User performance during simulated tasks	[54,69,93,105,113,118,123,125,135]
	Surveys	[14,30,31,35,48,52,55,57,64,66,73,76,83,87,88,91,93,99, 101,103,107-109,111,114,115,120]
	Questionnaires	[14,18,48,89,91,116-118,120,126]

The scope of analysis within which this HF/E design cycle occurs is diverse. For simplicity in communication, the HF/E community often partitions their scope of work into the three separate domains of cognitive, physical, and organizational ergonomics. Not surprisingly, HF/E-focused health informatics research often focuses on one of these three domains. Because health informatics technologies are often designed to support information-intensive work, many HF/E focused health informatics technologies aim to support clinician and/or patient cognition. Commonly used cognitive ergonomics-focused approaches such as cognitive task analysis, focus on supporting and improving a range of processes including clinician and patient comprehension, decision-making, distributed or team cognition, and errors [[18], [136]]. Some HF/E-focused health informatics research attempts to capture all three domains (cognitive, physical, organizational), focusing on the complex sociotechnical systems within which health informatics technologies are used. For example, SEIPS 2.0, a “sociotechnical work system → process outcomes” model based on the Systems Engineering Initiative for Patient Safety model (SEIPS), helps HF/E researchers and practitioners capture and evaluate the system elements impacting and being impacted by health informatics technologies [[137], [138]]. Another sociotechnical model developed by Sittig and Singh is “designed to address the socio-technical challenges involved in design, development, implementation, use, and evaluation of HIT within complex adaptive healthcare systems” [[139]].

Studies included in this synthesis used diverse methods to conduct the “understand → create → evaluate HF/E design” cycle phases, shown in [Table 1]. Many studies used or reviewed the use of one or more qualitative HF/E approaches, such as heuristic analysis, observations, focus groups, cognitive walkthroughs, and interviews. Others used or reviewed the use of one or more quantitative HF/E methods such as structured usability testing, user performance measurements during simulated tasks, surveys, and questionnaires. Many studies combined qualitative and quantitative methods, such as quantitative measurements during simulated tasks followed by debriefing interviews.

This large body of health informatics work using a diverse set of HF/E methods to carry out the “understand → create → evaluate” cycle is exciting. The volume of studies focused on understanding user needs, creating health informatics technologies to meet those needs, and evaluating technologies shows that HF/E frameworks as methods are becoming embedded in the activities of the health informatics community. The health informatics community would benefit from HF/E-focused health informatics researchers and practitioners focusing significant effort on consolidating methodological “understand → create → evaluate” best practices and ensuring that those best practices are accessible to the broad health informatics community. For example, evaluations of users’ perceptions of system usability ranged widely. While methods and instruments measure different constructs, it would be helpful to come to agreement on best practices for qualitative and quantitative HF/E methods and making those recommendations accessible to the broad health informatics community. Doing so would help the HF/E-focused health informatics community better synthesize findings from these labor and resource-intensive studies.

3.2 Diversity in Technology Types

There are numerous ways we could categorize the technologies in the studies included in this synthesis, such as the mode of delivery (e.g., handheld, desktop, …etc.), interaction-type (e.g., voice, touch, gesture, etc.), or underlying algorithmic approach (e.g., practice guideline-based, AI, machine learning, etc.). In this synthesis, we focused on categorizing studies based on the expansion of health informatics design opportunities supported by the (relatively) recent ability to customize EHR interfaces, open APIs that allow developers to directly create new software tools leveraging EHR data, and increasingly accessible platforms for mobile app development.

Shown in [Table 2], while some studies included in this synthesis were focused on physicians interacting with existing EHR computerized provider order entry (CPOE) interfaces, most studies in clinical settings focused on customizations to EHRs. Many technologies focused on the creation and evaluation of customized clinical decision support tools. The application of these customized clinical decision support tools varied widely in context, including diagnostic support [[17], [132]], antibiotic stewardship [[36], [70]], screening for and management of chronic conditions [[53], [91], [119]], identifying individuals at risk for varied clinical outcomes [[50], [69], [87], [118]].

Table 2 Technology types addressed in studies.

Technology types	CPOE interfaces	[105,113]
	Customized clinical decision support tools	[17,29,69,70,81,87,91,99,118,119,126,129,33,130,132,142, 34,36,37,49,50,53,67]
	Customized data displays	[10,11,92,96,98,123-125,50-52,55,61,66,72,84]
	Mobile apps	[13,15,47,54,56-58,63,75,85,108,110,23,115-117,121,122, 127,131,25-27,30,35,40,41]

Many studies focused on the creation and evaluation of customized data displays. These customized data displays also focused on a diverse set of application areas, including integrated dashboards [[10], [61], [124]], critical care displays [[96]], opioid management [[123]], plan of care tools [[125]], and patient-focused communication [[11], [55], [98]]. A smaller number of studies addressed non-EHR integrated information systems [[9], [65], [68], [83], [112], [120]], and EHR training design [[46], [59], [135]].

Many studies focused on creating and evaluating mobile apps, typically aimed at addressing the needs of patients and consumers. These mobile apps focused on a variety of chronic diseases such as diabetes and hypertension [[63], [122]], cardiovascular health [[30], [58], [121]], cancer care [[58], [127]], mental health [[13], [23], [25], [57], [110]], seizure management [[57]], bladder monitoring [[56]], tuberculosis treatment [[15]], and parental education [[40], [41]]. A small number of studies focused on telehealth [[10], [14], [140]] and personal health records or patient portals [[38], [109], [141]].

While evaluations of EHR usability are still critical [[2], [3]], the ability of HF/E-focused health informatics researchers and practitioners to be designers of new technologies – rather than purely evaluators – is critical for health informatics technologies of the future to be useful to and useable by a variety of end users. HF/E-focused health informatics researchers and practitioners must therefore advocate for continued development of accessible design resources and platforms that allow them to be innovators.

3.3 Outcomes of Interest

HF/E methods aim to improve how individuals interact with complex systems. Qualitative and quantitative HF/E methods can help to understand and affect positive change on a range of human-system interactions, including supporting health information technology user cognition and understanding how health informatics technologies affect and are affected by complex sociotechnical systems within which they are embedded. These positive impacts of HF/E-focused health informatics research include improving the safety (reducing risk of injury or death), performance (increasing productivity, quality, and efficiency) and satisfaction (acceptance, comfort, and wellbeing) of health informatics technologies [[1]]. While HF/E-focused health informatics research ideally improves all three of these outcomes, the relative weighting of safety, performance, and satisfaction typically depends on the context of application, shown in [Figure 2]. The length of each leg on the triangle represents the relative importance of that outcome, with a longer leg meaning that outcome is typically weighted more heavily in that domain.

Fig. 2 Relative weighting of HF/E outcomes by domain [[1]].

Part of the challenge in designing and evaluating health informatics technologies is the lack of clarity in which domain(s) a particular technology is deployed within. For example, EHRs and customized CDS and visual displays are typically deployed in high-risk workplaces. Yet, as we show below, the bulk of the evaluation measures from the HF/E-focused health informatics work included in this synthesis focused on measuring (often only) end user satisfaction. There are a multitude of likely reasons for this, including a known lack of system usability, an assumption about the interrelatedness of these measures (e.g., the impact of performance on satisfaction), and the relative ease of measuring perceived satisfaction. This tension becomes even greater as we consider how mobile health data might be integrated into the EHR – where the implementation of these technologies involves a consumer product and related technology deployed in a high-risk workplace.

While patient safety is often a motivator for the development of health informatics technologies, it is difficult to measure, and is therefore infrequently directly assessed – especially considering the HF/E concept of safety focuses on reducing risk of injury or death. Shown in [Table 3], some studies in this synthesis instead focused on how EHR designs might negatively influence patient safety, including CPOE ordering accuracy [[105]], unexpected use of free text data entry [[71]], discrepancies in documentation during patient transfers [[86]], lack of patient identification during CPOE [[113]], and appropriate responses to alerts [[134], [143]]. Other studies use proxy measures that may be correlated with, or lead to, safety issues. For example, a recent analysis of an inpatient safety dashboard in the context of opioid management focused on measuring user performance (i.e., time on task, mouse clicks, mouse movement, cognitive load, and task inaccuracy) [[123]]. Another analysis of a patient safety dashboard measured system usage and perceived satisfaction [[124]]. A recent study focused on developing and evaluating a dashboard targeting acute kidney injury (AKI) to improve patient safety measured system usage and performance with respect to six quality indicators, but the quality measures were developed by end users – not validated safety measures [[126]]. The measures in these studies are important and likely related to safety outcomes, but the relative rarity of patient safety events and lack of empirically validated safety markers makes directly measuring the impact of health informatics design on safety quite difficult.

Table 3 Outcomes addressed in studies

Outcome measures	Safety	Potential impact of design on safety	[71,86,105,113,134,143]
		Proxy measures	[123,124,126]
	Performance	Accuracy and completeness Efficiency	[9,16,17,52,57,93,96,98,102,117,118,125] [16,17,52,54,65,76,80,83,93,95,96,99,101,104, 111,123,125]
	Satisfaction	Interviews and focus groups	[9,10,15,26,30,37,41,46,51,55,64,74,79,82,91,93,114,119,120,122,134]
		SUS	[9,16,46,54,57,72,117,120]
		TAM	[46,83,99]
		UTAUT	[54,82,116]
		Health-ITUES	[84,124]
		PSSUQ	[69]
		Computer system usability questionnaire	[125]
		TTF	[144,145]
		SUMI	[89]
		Object-action interface questionnaire	[126]
		NuHISS	[103]
		Self-developed questionnaires	[14,48,73,75,77,87,91,114,115]

A larger set of studies focus on measuring aspects of performance (e.g., productivity, quality, and efficiency) via measures related to time to complete tasks, markers of task completion, and errors while completing those tasks. Many studies assessing performance measured how accurate or complete the user interaction with the system was, with those interactions varying from a layperson interpretating a visualization to a clinician documenting a patient encounter. Shown in [Table 3], many studies focused on improving the efficiency of health informatics technologies, by measuring current inefficiencies due to issues such as dispersion of information across areas in the interface or interrupted workflows, and in some cases designing tools that improve (or at least maintain) user time to complete tasks of interest. The level of granularity of these time measures, however, ranged from milliseconds (e.g., eye movements when searching for information) to hours or days (e.g., time to complete results review).

Numerous studies assessed users’ satisfaction with a given technology. Shown in [Table 3], many studies used interviews and focus groups to glean users’ perceptions. Others used a variety of questionnaire and survey instruments based on the System Usability Scale (SUS), Technology Acceptance Model, Unified Theory of Acceptance and Usage of Technology (UTAUT), Health-ITUES, Post-Study System Usability Questionnaire (PSSUQ), Computer System Usability Questionnaire, Task-Technology Fit (TTF), Software Usability Measurement Inventory (SUMI), Object-Action Interface (OAI), National Usability-focused HIS Scale (NuHISS), and self-developed questionnaires.

Health informatics researchers and practitioners clearly see the need for their work to jointly address the HF/E goals of improving safety, performance, and satisfaction. Recent research demonstrates the breadth of methods and measures being used by the HF/E-focused health informatics community to assess these three outcomes. Because injuries or safety events of a specific type are relatively infrequent or difficult to detect, safety is often measured via the prevalence of potentially unsafe actions or via proxy measure that may be correlated with safety. While performance and satisfaction are frequently assessed, there is relatively little cohesion around a standard set of methods or measures to use when evaluating health informatics technologies. By coming to agreement on methods to measure these outcomes, the health informatics community can better synthesize findings across studies.

4 Conclusion

The health informatics community is at an exciting time with respect to developing innovative technologies and interventions that truly improve system safety and performance and increase end user satisfaction. HF/E methods are becoming embedded in the culture of the health informatics community, with broad acknowledgement that the needs of end users should be accounted for the health informatics design. There is a significant gap, however, between the health informatics community acknowledging the importance of HF/E methods and the scattered nature in which these methods are being deployed. HF/E-focused health informatics would be well-served to coalesce around common design methods and outcome measures that can be easily communicated to the broader health informatics community. By not doing so, we are bypassing an opportunity to synthesize the important, time-consuming work being conducted across the health informatics community.

HF/E-focused health informatics researchers and practitioners can also be strong advocates for technology development platforms and design training programs that reduce barriers to entry for health informatics design innovators, supporting the democratization of health informatics design. HF/E-focused health informatics can also be strong advocates for building a community of health informatics designers that is as diverse as the end users of their designs.

• References

• 1 Lee JD, Wickens CD, Liu Y, Boyle LN. Designing for people: An introduction to human factors engineering. CreateSpace. 2017
• 2 Carayon P, Hoonakker P. Human Factors and Usability for Health Information Technology: Old and New Challenges. Yearb Med Inform 2019; 28 (01) 71-7
• 3 Abbott PA, Weinger MB. Health information technology: Fallacies and Sober realities - Redux A homage to Bentzi Karsh and Robert Wears. Appl Ergon [Internet] 2020 Jan. 82. (October 2019):102973. Available from: https://doi.org/10.1016/j.apergo.2019.102973
• 4 DIS ISO, 9241-210: 2010. Ergonomics of human system interaction-Part 210: Human-centred design for interactive systems (formerly known as 13407) [Internet]. International Standardization Organization (ISO); Switzerland: 2010. . Available from: https://www.iso.org/obp/ui/#iso:std:iso:9241:-210:ed-1:v1:en
• 5 IDEO.org. The Field Guide to Human-Centered Design. 1st ed.. Determann's Field Guide To Data Privacy Law. IDEO.org; 2015
• 6 Office of the Federal Register. Federal Register. Vol. 77. 2012: 13869
• 7 Schumacher. Robert M, Lowry SZ. NISTIR 7741: NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records. NISTIR Tech [Internet] 2010; 7741: 1-62 . Available from: http://www.nist.gov/itl/hit/upload/Guide_Final_Publication_Version.pdf
• 8 Ratwani RM, Fairbanks RJ, Zachary Hettinger A, Benda NC. Electronic health record usability: Analysis of the user-centered design processes of eleven electronic health record vendors. J Am Med Inform Assoc 2015; 22 (06) 1179-82
• 9 Fareed N, Swoboda CM, Jonnalagadda P, Griesenbrock T, Gureddygari HR, Aldrich A. Visualizing Opportunity Index Data Using a Dashboard Application: A Tool to Communicate Infant Mortality-Based Area Deprivation Index Information. Appl Clin Inform 2020; 11 (04) 515-27
• 10 Dixit RA, Hurst S, Adams KT, Boxley C, Lysen-Hendershot K, Bennett SS. et al. Rapid development of visualization dashboards to enhance situation awareness of COVID-19 telehealth initiatives at a multihospital healthcare system. J Am Med Inform Assoc 2020; 27 (09) 1456-61
• 11 Kim S, Trinidad B, Mikesell L, Aakhus M. Improving prognosis communication for patients facing complex medical treatment: A user-centered design approach. Int J Med Inform [Internet]. 2020 141. (2019 December):104147. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104147
• 12 Holden RJ, Cornet VP, Valdez RS. Patient ergonomics: 10-year mapping review of patient-centered human factors. Appl Ergon [Internet]. 2020 82. (October 2019):102972. Available from: https://doi.org/10.1016/j.apergo.2019.102972
• 13 Fausett C, Lee J, Godwin E, Palmer E, Fouquet S. Informing the Design of Mental Health Smartphone Apps for Adolescents. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 218-22
• 14 Su YY, Huang ST, Wu YH, Chen CM. Factors Affecting Patients’ Acceptance of and Satisfaction with Cloud-Based Telehealth for Chronic Disease Management: A Case Study in the Workplace. Appl Clin Inform 2020; 11 (02) 286-94
• 15 Iribarren SJ, Rodriguez Y, Lin L, Chirico C, Discacciati V, Schnall R. et al. Converting and expanding a mobile support intervention: Focus group and field-testing findings from individuals in active tuberculosis treatment. Int J Med Inform [Internet]. 2020 136. (December 2019):104057. Available from: https://doi.org/10.1016/j.ijmedinf.2019.104057
• 16 Pierce RP, Eskridge BR, Rehard LA, Ross B, Day MA, Belden JL. The Effect of Electronic Health Record Usability Redesign on Annual Screening Rates in an Ambulatory Setting. Appl Clin Inform 2020; 11 (04) 580-8
• 17 Salwei ME, Carayon P, Hoonakker P, Hundt AS, Novak C, Wang Y. et al. Assessing workflow of emergency physicians in the use of clinical decision support. Proc Hum Factors Ergon Soc Annu Meet 2019; 63 (01) 772-6
• 18 Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004; 37 (01) 56-76
• 19 Yen PY, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012; 19 (03) 413-22
• 20 Cohen DJ, Wyte-Lake T, Dorr DA, Gold R, Holden RJ, Koopman RJ. et al. Unmet information needs of clinical teams delivering care to complex patients and design strategies to address those needs. J Am Med Inform Assoc 2020; 27 (05) 690-9
• 21 Esmaeilzadeh P, Mirzaei T, Dharanikota S. The impact of data entry structures on perceptions of individuals with chronic mental disorders and physical diseases towards health information sharing. Int J Med Inform [Internet]. 2020 141. 104157 . Available from: https://doi.org/10.1016/j.ijmedinf.2020.104157
• 22 Sockolow PS, Bowles KH, Wojciechowicz C, Bass EJ. Incorporating home healthcare nurses’ admission information needs to inform data standards. J Am Med Inform Assoc 2020; 27 (08) 1278-86
• 23 Johnson AJ, Bisantz AM, Reynolds AL, Meier ST. Assessing Decision Support Needs of those in Mental Health Crisis Utilizing Cognitive Task Analysis. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 46-46
• 24 Jahn Holbrook M, Caldwell BS. Development of a Multi-Layer Systems Engineering Visualization for Diabetes Team Coordination. Proc Hum Factors Ergon Soc Annu Meet 2019; 63 (01) 643-7
• 25 Ishak T, Torres A, Asan O. Developing Technology-Enabled Solutions for College Students to Cope with Factors Leading to Health Problems. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 56-60
• 26 Marcu G, Aizen R, Roth AM, Lankenau S, Schwartz DG. Acceptability of smartphone applications for facilitating layperson naloxone administration during opioid overdoses. JAMIA Open 2020; 3 (01) 44-52
• 27 Lorca-Cabrera J, Martí-Arques R, Albacar-Riobóo N, Raigal-Aran L, Roldan-Merino J, FerréGrau C. Mobile Applications for Caregivers of Individuals with Chronic Conditions and/or Diseases: Quantitative Content Analysis. Int J Med Inform 2020; 145 (July 2020): 104310
• 28 Burgermaster M, Son JH, Davidson PG, Smaldone AM, Kuperman G, Feller DJ. et al. A new approach to integrating patient-generated data with expert knowledge for personalized goal setting: A pilot study. Int J Med Inform [Internet]. 2020 139. (October 2019):104158. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104158
• 29 Benda NC, Das LT, Abramson EL, Blackburn K, Thoman A, Kaushal R. et al. “how did you get to this number?” Stakeholder needs for implementing predictive analytics: A pre-implementation qualitative study. J Am Med Inform Assoc 2020; 27 (05) 709-16
• 30 Zachary WW, Wilkin T, Puglisi L, Butler M, Ferdousi I, Smart J. et al. Designing a Cardiovascular Health App for Citizens Returning from Incarceration: Where You Start May Not Be Where You End Up. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 34-42
• 31 Kabukye JK, de Keizer N, Cornet R. Elicitation and prioritization of requirements for electronic health records for oncology in low resource settings: A concept mapping study. Int J Med Inform [Internet]. 2020 135. (December 2019):104055. Available from: https://doi.org/10.1016/j.ijmedinf.2019.104055
• 32 Schubel L, Stein L, Barrientos RR, Valdiviezo C, Townsend M, Basch P. et al. Bridging the gap: Workflow analysis evaluating “work-as-imagined” versus “work-as-done” for cardiac risk calculation. Proc Hum Factors Ergon Soc Annu Meet 2019; 63 (01) 767-71
• 33 Ash JS, Chase D, Baron S, Filios MS, Shiffman RN, Marovich S. et al. Clinical Decision Support for Worker Health: A Five-Site Qualitative Needs Assessment in Primary Care Settings. Appl Clin Inform 2020; 11 (04) 635-43
• 34 Ostropolets A, Chen RJ, Zhang L, Hripcsak G. Characterizing physicians’ information needs related to a gap in knowledge unmet by current evidence. JAMIA Open 2020; 3 (02) 281-9
• 35 Austin E, Lee JR, Amtmann D, Bloch R, Lawrence SO, McCall D. et al. Use of patient-generated health data across healthcare settings: implications for health systems. JAMIA Open 2020; 3 (01) 70-6
• 36 Ozkaynak M, Metcalf N, Cohen DM, May LS, Dayan PS, Mistry RD. Considerations for Designing EHR-Embedded Clinical Decision Support Systems for Antimicrobial Stewardship in Pediatric Emergency Departments. Appl Clin Inform 2020; 11 (04) 589-97
• 37 Bhat S, Derington CG, Derington CG, Trinkley KE, Trinkley KE. Clinicians’ Values and Preferences for Medication Adherence and Cost Clinical Decision Support in Primary Care: A Qualitative Study. Appl Clin Inform 2020; 11 (03) 405-14
• 38 Knisely BM, Vaughn-Cooke M. A Comprehensive Survey of Preference for Personal Health Record Features in Patients with Diabetes. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 171-5
• 39 Daei A, Soleymani MR, Ashrafi-rizi H, Zargham-Boroujeni A, Kelishadi R. Clinical information seeking behavior of physicians: A systematic review. Int J Med Inform 2020. 139. (April):
• 40 Paricio-Talayero JM, Mena-Tudela D, Cervera-Gasch Á, González-Chordá VM, Paricio-Burtin Y, Sánchez-Palomares M. et al. Is it compatible with breastfeeding? www.e-lactancia.org: Analysis of visits, user profile and most visited products. Int J Med Inform [Internet]. 2020 141. (November 2019):104199. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104199
• 41 Rau NM, Hasan K, Ahamed SI, Asan O, Flynn KE, Basir MA. Designing a tablet-based prematurity education app for parents hospitalized for preterm birth. Int J Med Inform [Internet]. 2020 141. (December 2019):104200. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104200
• 42 Esmaeilzadeh P, Mirzaei T, Maddah M. The effects of data entry structure on patients’ perceptions of information quality in Health Information Exchange (HIE). Int J Med Inform [Internet]. 2020 135. (December 2019):104058. Available from: https://doi.org/10.1016/j.ijmedinf.2019.104058
• 43 Kannampallil T, Abraham J. Listening and question-asking behaviors in resident and nurse handoff conversations: a prospective observational study. JAMIA Open 2020; 3 (01) 87-93
• 44 Stone JA, Watterson T, Xiong KZ, Ramly E, Kleinschmidt P, Seminak M. et al. Identifying Vulnerabilities in HealthIT: A Case Study of CancelRx Implementation in Clinic and Pharmacy Sociotechnical Systems. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 64-6
• 45 Cho I, Lee M, Kim Y. What are the main patient safety concerns of healthcare stakeholders: a mixed-method study of Web-based text. Int J Med Inform [Internet] 2020; 140: 104162 . Available from: https://doi.org/10.1016/j.ijmedinf.2020.104162
• 46 Tennant R, Leggett A, Smith C, Tan L, Mercer K, Burns CM. Addressing Human Factor Challenges in Paediatric Home Care: Development and Evaluation of a Mobile Home Care Communication App. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 37-9
• 47 Abdolkhani R, Gray K, Borda A, DeSouza R. Patient-generated health data management and quality challenges in remote patient monitoring. JAMIA Open 2019; 2 (04) 471-8
• 48 Taber P, Ghani P, Schiffman JD, Kohlmann W, Hess R, Chidambaram V. et al. Physicians’ strategies for using family history data: having the data is not the same as using the data. JAMIA Open 2020; 3 (October): 378-85
• 49 Ozkaynak M, Bisantz A, Militello L, Miller K, Rayo M, Slagle J. What Makes a Difference in Patient Safety when Designing, Implementing and Evaluating Clinical Decision Support Systems?. Proc Hum Factors Ergon Soc Annu Meet 2019; 63 (01) 693-7
• 50 Gilbank P, Johnson-Cover K, Truong T. Designing for Physician Trust: Toward a Machine Learning Decision Aid for Radiation Toxicity Risk. Ergon Des 2020; 28 (03) 27-35
• 51 Calzoni L, Clermont G, Cooper GF, Visweswaran S, Hochheiser H. Graphical Presentations of Clinical Data in a Learning Electronic Medical Record. Appl Clin Inform 2020; 11 (04) 680-91
• 52 Curran RL, Kukhareva PV, Taft T, Weir CR, Reese TJ, Nanjo C. et al. Integrated displays to improve chronic disease management in ambulatory care: A SMART on FHIR application informed by mixed-methods user testing. J Am Med Inform Assoc 2020; 27 (08) 1225-34
• 53 Schubel L, Stein L, Romero R, Miller K. Mitigating Cardiovascular Risk Through User Informed Clinical Decision Support. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 67-9
• 54 Boillat T, Siebert JN, Alduaij N, Ehrler F. GOFlow: Smartwatch app to deliver laboratory results in emergency departments - A feasibility study. Int J Med Inform [Internet]. 2020 134. (July 2019):104034. Available from: https://doi.org/10.1016/j.ijmedinf.2019.104034
• 55 Stonbraker S, Porras T, Schnall R. Patient preferences for visualization of longitudinal patient-reported outcomes data. J Am Med Inform Assoc 2020; 27 (02) 212-24
• 56 Park S, Oh H, Choi J, Jang W. A Design of Mobile Application for Ultrasound Bladder Monitoring System Based on Usability Engineering. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 177-80
• 57 Yoo S, Lim K, Baek H, Jang SK, Hwang Gyoung, Kim H. et al. Developing a mobile epilepsy management application integrated with an electronic health record for effective seizure management. Int J Med Inform [Internet]. 2020 134. (December 2019):104051. Available from: https://doi.org/10.1016/j.ijmedinf.2019.104051
• 58 Kelley M, Foraker R, Lin E-JD, Kulkarni M, Lustberg M, Weaver KE. Oncologists’ Perceptions of a Digital Tool to Improve Cancer Survivors’ Cardiovascular Health. ACI Open 2019; 03 (02) e78-87
• 59 Scott K, Hathaway E, Sharp K, Smailes P. The Development and Evaluation of an Electronic Health Record Efficiency Workshop for Providers. Appl Clin Inform 2020; 11 (02) 336-41
• 60 Wosik J, Fudim M, Cameron B, Gellad ZF, Cho A, Phinney D. et al. Telehealth transformation: COVID-19 and the rise of virtual care. J Am Med Inform Assoc 2020; 27 (06) 957-62
• 61 Foster M, Albanese C, Chen Q, Sethares KA, Evans S, Lehmann LS. et al. Heart Failure Dashboard Design and Validation to Improve Care of Veterans. Appl Clin Inform 2020; 11 (01) 153-9
• 62 Chaparro JD, Hussain C, Lee JA, Hehmeyer J, Nguyen M, Hoffman J. Reducing Interruptive Alert Burden Using Quality Improvement Methodology. Appl Clin Inform 2020; 11 (01) 46-58
• 63 Choi W, Wang S, Lee Y, Oh H, Zheng Z. A systematic review of mobile health technologies to support self-management of concurrent diabetes and hypertension. J Am Med Inform Assoc 2020; 27 (06) 939-45
• 64 Martikainen S, Kaipio J, Lääveri T. End-user participation in health information systems (HIS) development: Physicians’ and nurses’ experiences. Int J Med Inform [Internet]. 2020 137. (December 2019):104117. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104117
• 65 Challen R, Tsaneva-Atanasova K, Edwards T, Gompels L, Dayer M, Pitt M. et al. Factors influencing digital review of pathology test results in an inpatient setting: a cross-sectional study. JAMIA Open 2020; 3 (02) 290-8
• 66 Yu SC, Lai AM, Smyer J, Flaherty J, Mangino JE, McAlearney AS. et al. Novel Visualization of Clostridium difficile Infections in Intensive Care Units. ACI Open 2019; 03 (02) e71-7
• 67 Schubel L, Littlejohn R, Barrientos RR, Arnold R, Capan M, Miller K. Everything but the Kitchen Sink: A Mixed Methods Approach to Health IT Usability Testing. Proc Hum Factors Ergon Soc Annu Meet 2019; 63 (01) 718-22
• 68 Alfano M, Lenzitti B, Lo Bosco G, Muriana C, Piazza T, Vizzini G. Design, development and validation of a system for automatic help to medical text understanding. Int J Med Inform [Internet]. 2020 138. (August 2018):104109. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104109
• 69 Abdel-Rahman SM, Gill H, Carpenter SL, Gueye P, Wicklund B, Breitkreutz M. et al. Design and Usability of an Electronic Health Record-Integrated, Point-of-Care, Clinical Decision Support Tool for Modeling and Simulation of Antihemophilic Factors. Appl Clin Inform 2020; 11 (02) 253-64
• 70 Patterson ES, DiLoreto GN, Vanam R, Hade E, Hebert C. Enhancing Usefulness and Usability of a Clinical Decision Support Prototype for Antibiotic Stewardship. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 61-5
• 71 Kandaswamy S, Hettinger AZ, Hoffman DJ, Ratwani RM, Marquard J. Communication through the electronic health record: frequency and implications of free text orders. JAMIA Open 2020; 3 (02) 154-9
• 72 Bockhacker M, Syrek H, Elstermann Von Elster M, Schmitt S, Roehl H. Evaluating Usability of a Touchless Image Viewer in the Operating Room. Appl Clin Inform 2020; 11 (01) 88-94
• 73 Wyatt KD, Willaert BN, Lohse CM, Pallagi PJ, Yiannias JA, Hellmich TR. Experiences of Health Care Providers Using a Mobile Medical Photography Application. Appl Clin Inform 2020; 11 (01) 122-9
• 74 Feller DJ, Lor M, Zucker J, Yin MT, Olender S, Ferris DC. et al. An investigation of the information technology needs associated with delivering chronic disease care to large clinical populations. Int J Med Inform [Internet]. 2020 137. (October 2019):104099 Available from: https://doi.org/10.1016/j.ijmedinf.2020.104099
• 75 Esposito M, Rocq PL, Novy E, Remen T, Losser MR, Guerci P. Smartphone to-do list application to improve workflow in an intensive care unit: A superiority quasi-experimental study. Int J Med Inform [Internet]. 2020 136. (January):104085. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104085
• 76 Hilliard RW, Haskell J, Gardner RL. Are specific elements of electronic health record use associated with clinician burnout more than others?. J Am Med Inform Assoc 2020; 27 (09) 1401-10
• 77 Van Laere S, Cornu P, Buyl R. A cross-sectional study of the Belgian community pharmacist's satisfaction with the implementation of the electronic prescription. Int J Med Inform [Internet]. 2020 135. (December 2019):104069. Available from: https://doi.org/10.1016/j.ijmedinf.2019.104069
• 78 Rogith D, Satterly T, Singh H, Sittig DF, Russo E, Smith MW. et al. Application of Human Factors Methods to Understand Missed Follow-up of Abnormal Test Results. Appl Clin Inform 2020; 11 (05) 692-8
• 79 Dullabh P, Hovey L, Heaney-Huls K, Rajendran N, Wright A, Sittig DF. Application Programming Interfaces in Health Care: Findings from a Current-State Sociotechnical Assessmen. Appl Clin Inform 2020; 11 (01) 59-69
• 80 Sinsky CA, Rule A, Cohen G, Arndt BG, Shanafelt TD, Sharp CD. et al. Metrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc 2020; 27 (04) 639-43
• 81 Valvona SN, Rayo MF, Abdel-Rasoul M, Locke LJ, Rizer MK, Moffatt-Bruce SD. et al. Comparative Effectiveness of Best Practice Alerts with Active and Passive Presentations: A Retrospective Study. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 105-9
• 82 Klappe ES, De Keizer NF, Cornet R. Factors Influencing Problem List Use in Electronic Health Records-Application of the Unified Theory of Acceptance and Use of Technology. Appl Clin Inform 2020; 11 (03) 415-26
• 83 Askari M, Tam JLYY, Aarnoutse MF, Meulendijk M. Perceived effectiveness of clinical pathway software: A before-after study in the Netherlands. Int J Med Inform. 2020 135. (December 2019)
• 84 Khasnabish S, Burns Z, Couch M, Mullin M, Newmark R, Dykes PC. Best practices for data visualization: creating and evaluating a report for an evidence-based fall prevention program. J Am Med Inform Assoc 2020; 27 (02) 308-14
• 85 Lin CJ, Ho SH. Prediction of the use of mobile device interfaces in the progressive aging process with the model of Fitts’ law. J Biomed Inform [Internet] 2020; 107 (43) 103457 . Available from: https://doi.org/10.1016/j.jbi.2020.103457
• 86 Orenstein EW, Ferro DF, Bonafide CP, Landrigan CP, Gillespie S, Muthu N. Hidden health IT hazards: a qualitative analysis of clinically meaningful documentation discrepancies at transfer out of the pediatric intensive care unit. JAMIA Open 2019; 2 (03) 392-8
• 87 Romero-Brufau S, Wyatt KD, Boyum P, Mickelson M, Moore M, Cognetta-Rieke C. A lesson in implementation: A pre-post study of providers’ experience with artificial intelligence-based clinical decision support. Int J Med Inform [Internet]. 2020 137. (December 2019):104072. Available from: https://doi.org/10.1016/j.ijmedinf.2019.104072
• 88 Gill E, Dykes PC, Rudin RS, Storm M, McGrath K, Bates DW. Technology-facilitated care coordination in rural areas: What is needed?. Int J Med Inform [Internet]. 2020 137. (February):104102. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104102
• 89 Rangraz Jeddi F, Nabovati E, Bigham R, Khajouei R. Usability evaluation of a comprehensive national health information system: relationship of quality components to users’ characteristics. Int J Med Inform [Internet]. 2020 133. (October 2018):104026. Available from: https://doi.org/10.1016/j.ijmedinf.2019.104026
• 90 Pennathur PR. A Theoretical Framework for Understanding Creator-Consumer Information Interaction Behaviors in Healthcare Documentation Systems. Appl Ergon [Internet]. 2020 84. (December 2019):103034. Available from: https://doi.org/10.1016/j.apergo.2019.103034
• 91 El Mikati HK, Yazel-Smith L, Grout RW, Downs SM, Carroll AE, Hannon TS. Clinician Perceptions of a Computerized Decision Support System for Pediatric Type 2 Diabetes Screening. Appl Clin Inform 2020; 11 (02) 350-5
• 92 Lasko TA, Owens DA, Fabbri D, Wanderer JP, Genkins JZ, Novak LL. User-Centered Clinical Display Design Issues for Inpatient Providers. Appl Clin Inform 2020; 11 (05) 700-9
• 93 Blackley SV, Schubert VD, Goss FR, Al Assad W, Garabedian PM, Zhou L. Physician use of speech recognition versus typing in clinical documentation: A controlled observational study. Int J Med Inform [Internet]. 2020 141. (April):104178. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104178
• 94 Kaur D, Panos RJ, Badawi O, Bapat SS, Wang L, Gupta A. Evaluation of clinician interaction with alerts to enhance performance of the tele-critical care medical environment. Int J Med Inform [Internet]. 2020 139. (April):104165. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104165
• 95 Devin J, Costello J, McCallion N, Higgins E, Kehoe B, Cleary BJ. et al. Impact of an electronic health record on task time distribution in a neonatal intensive care unit. Int J Med Inform [Internet]. 2021 145. (April 2020):104307. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104307
• 96 Reese TJ, Del Fiol G, Tonna JE, Kawamoto K, Segall N, Weir C. et al. Impact of integrated graphical display on expert and novice diagnostic performance in critical care. J Am Med Inform Assoc 2020; 27 (08) 1287-92
• 97 Leander DM, Gifford AH, Mecchella JN, Sabadosa KA, Van Citters A, Snide JA. et al. Design and Implementation of a Novel Electronic Health Record Tool to Enhance the Care of Individuals with Cystic Fibrosis: The Cystic Fibrosis Note Template. ACI Open 2019; 03 (01) e26-36
• 98 Reading Turchioe M, Grossman LV, Myers AC, Baik D, Goyal P, Masterson Creber RM. Visual analogies, not graphs, increase patients’ comprehension of changes in their health status. J Am Med Inform Assoc 2020; 27 (05) 677-89
• 99 Williams A, Turer C, Smith J, Nievera I, McCulloch L, Wareg N. et al. Adoption of an Electronic Medical Record Tool for Childhood Obesity by Primary Care Providers. Appl Clin Inform 2020; 11 (02) 210-7
• 100 Fraczkowski D, Matson J, Lopez KD. Nurse workarounds in the electronic health record: An integrative review. J Am Med Inform Assoc 2020; 27 (07) 1149-65
• 101 Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: Time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc 2020; 27 (04) 531-8
• 102 McLaurin E, Bass EJ, Bowles KH, Sockolow P. Assessing the Usability of Medication Information Layouts in a Home Healthcare Electronic Health Record. Proc Int Symp Hum Factors Ergon Heal Care 2020; 9 (01) 183-5
• 103 Kaipio J, Kuusisto A, Hyppönen H, Heponiemi T, Lääveri T. Physicians’ and nurses’ experiences on EHR usability: Comparison between the professional groups by employment sector and system brand. Int J Med Inform [Internet]. 2020 134. (June 2019):104018. Available from: https://doi.org/10.1016/j.ijmedinf.2019.104018
• 104 Moore EC, Tolley CL, Bates DW, Slight SP. A systematic review of the impact of health information technology on nurses’ time. J Am Med Inform Assoc 2020; 27 (05) 798-807
• 105 Kandaswamy S, Hettinger AZ, Ratwani RM. What Did You Order? Developing Models to Measure the Impact of Usability on Emergency Physician Accuracy Using Computerized Provider Order Entry. Proc Hum Factors Ergon Soc Annu Meet 2019; 63 (01) 713-7
• 106 Kandaswamy S, Wei J, Will A, Savage E, Ratwani RM, Hettinger AZ. et al. Variation in Electronic Health Record Workflow Patterns: A Multisite study. ACI Open 2020; 04 (01) e83-90
• 107 Diprose WK, Buist N, Hua N, Thurier Q, Shand G, Robinson R. Physician understanding, explainability, and trust in a hypothetical machine learning risk calculator. J Am Med Inform Assoc 2020; 27 (04) 592-600
• 108 Wu D, Lowry PB, Zhang D, Parks RF. Patients’ compliance behavior in a personalized mobile patient education system (PMPES) setting: Rational, social, or personal choices?. Int J Med Inform. 2021 145. (September 2020)
• 109 Wong JIS, Steitz BD, Rosenbloom ST. Characterizing the impact of health literacy, computer ability, patient demographics, and portal usage on patient satisfaction with a patient portal. JAMIA Open 2019; 2 (04) 456-64
• 110 Oh J, Jang S, Kim H, Kim JJ. Efficacy of mobile app-based interactive cognitive behavioral therapy using a chatbot for panic disorder. Int J Med Inform [Internet]. 2020 140. (May):104171. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104171
• 111 Richards S, Carlson K, Matthias T, Birge J. Perception versus reality: Does provider documentation behavior change when clinic notes are shared electronically with patients?. Int J Med Inform [Internet]. 2021 145. (September 2020):104304. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104304
• 112 Li RC, Pereira RAM, Jatobá A, Vidal MCR, de Carvalho PVR, Grindrod K. et al. Information Technology Systems at the sharp end of medication therapy management. Proc Hum Factors Ergon Soc Annu Meet 2019; 63 (01) 698-702
• 113 Fortman E, Hettinger AZ, Howe JL, Fong A, Pruitt Z, Miller K. et al. Varying rates of patient identity verification when using computerized provider order entry. J Am Med Inform Assoc 2020; 27 (06) 924-8
• 114 Butler JM, Gibson B, Lewis L, Reiber G, Kramer H, Rupper R. et al. Patient-centered care and the electronic health record: exploring functionality and gaps. JAMIA Open 2020; 3 (October): 360-8
• 115 Saleem JJ, Read JM, Loehr BM, Frisbee KL, Wilck NR, Murphy JJ. et al. Veterans’ response to an automated text messaging protocol during the COVID-19 pandemic. J Am Med Inform Assoc 2020; 27 (08) 1300-5
• 116 Wang H, Tao D, Yu N, Qu X. Understanding consumer acceptance of healthcare wearable devices: An integrated model of UTAUT and TTF. Int J Med Inform. 2020 139. (October 2019)
• 117 O’Grady M, Kapoor S, Gilmer E, Neighbors C, Conigliaro J, Kwon N. et al. Point-of-Care Mobile Application to Guide Health Care Professionals in Conducting Substance Use Screening and Intervention: A Mixed-Methods User Experience Study. ACI Open 2019; 03 (01) e1-12
• 118 Tomasi JN, Hamilton MV, Fan M, Pinkney SJ, Middaugh KL, Parshuram CS. et al. Assessing the electronic Bedside Paediatric Early Warning System: A simulation study on decision-making and usability. Int J Med Inform [Internet]. 2020 133. (June 2019):103969. Available from: https://doi.org/10.1016/j.ijmedinf.2019.103969
• 119 Oyugi B, Makunja S, Kabuti W, Nyongesa C, Schömburg M, Kibe V. et al. Improving the management of hypertension and diabetes: An implementation evaluation of an electronic medical record system in Nairobi County, Kenya. Int J Med Inform [Internet]. 2020 141. (May):104220. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104220
• 120 Richter Lagha R, Burningham Z, Sauer BC, Leng J, Peters C, Huynh T. et al. Usability Testing a Potentially Inappropriate Medication Dashboard: A Core Component of the Dashboard Development Process. Appl Clin Inform 2020; 11 (04) 528-34
• 121 Morey SA, Stuck RE, Chong AW, Barg-Walkow LH, Mitzner TL, Rogers WA. Mobile Health Apps: Improving Usability for Older Adult Users. Ergon Des 2019; 27 (04) 4-13
• 122 Avila-Garcia P, Hernandez-Ramos R, Nouri SS, Cemballi A, Sarkar U, Lyles CR. et al. Engaging users in the design of an mHealth, text message-based intervention to increase physical activity at a safety-net health care system. JAMIA Open 2019; 2 (04) 489-97
• 123 Fuller TE, Garabedian PM, Lemonias DP, Joyce E, Schnipper JL, Harry EM. et al. Assessing the cognitive and work load of an inpatient safety dashboard in the context of opioid management. Appl Ergon [Internet]. 2020 85. (December 2019):103047. Available from: https://doi.org/10.1016/j.apergo.2020.103047
• 124 Bersani K, Fuller TE, Garabedian P, Espares J, Mlaver E, Businger A. et al. Use, Perceived Usability, and Barriers to Implementation of a Patient Safety Dashboard Integrated within a Vendor EHR. Appl Clin Inform 2020; 11 (01) 34-45
• 125 Fischer SH, Safran C, Gajos KZ, Wright A. Visualization of Electronic Health Record Data for Decision-Making in Diabetes and Congestive Heart Failure. ACI Open 2020; 04 (01) e35-43
• 126 Tsang JY, Brown B, Peek N, Campbell S, Blakeman T. Mixed methods evaluation of a computerised audit and feedback dashboard to improve patient safety through targeting acute kidney injury (AKI) in primary care. Int J Med Inform [Internet]. 2021 145. 104299 . Available from: https://doi.org/10.1016/j.ijmedinf.2020.104299
• 127 Navarro-Alamán J, Lacuesta R, Garcia-Magariño I, Gallardo J. A methodology for the design and development of gamified mobile apps for monitoring cancer survivors. J Biomed Inform [Internet]. 2020 106. (April):103439. Available from: https://doi.org/10.1016/j.jbi.2020.103439
• 128 Dykes PC, Cullen TA. AMIA 2019 Annual Meeting Full Proceedings. 2019
• 129 Olakotan OO, Yusof MM. Evaluating the alert appropriateness of clinical decision support systems in supporting clinical workflow. J Biomed Inform [Internet]. 2020 106. (October 2019):103453. Available from: https://doi.org/10.1016/j.jbi.2020.103453
• 130 Rundo L, Pirrone R, Vitabile S, Sala E, Gambino O. Recent advances of HCI in decision-making tasks for optimized clinical workflows and precision medicine. J Biomed Inform [Internet]. 2020 108. (April):103479. Available from: https://doi.org/10.1016/j.jbi.2020.103479
• 131 Yang X, Ma L, Zhao X, Kankanhalli A. Factors influencing user's adherence to physical activity applications: A scoping literature review and future directions. Int J Med Inform [Internet]. 2020 134. (November 2019):104039. Available from: https://doi.org/10.1016/j.ijmedinf.2019.104039
• 132 Babione JN, Ocampo W, Haubrich S, Yang C, Zuk T, Kaufman J. et al. Human-centred design processes for clinical decision support: A pulmonary embolism case study. Int J Med Inform [Internet]. 2020 142. (March):104196. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104196
• 133 Hochheiser H, Valdez RS. Human-Computer Interaction, Ethics, and Biomedical Informatics. Yearb Med Inform 2020; 93-8
• 134 Humphrey KE, Mirica M, Phansalkar S, Ozonoff A, Harper MB. Clinician Perceptions of Timing and Presentation of Drug-Drug Interaction Alerts. Appl Clin Inform 2020; 11 (03) 487-96
• 135 Rajaram A, Hickey Z, Patel N, Newbigging J, Wolfrom B. Training medical students and residents in the use of electronic health records: A systematic review of the literature. J Am Med Inform Assoc 2020; 27 (01) 174-80
• 136 Patel VL, Kannampallil TG. Cognitive informatics in biomedicine and healthcare. J Biomed Inform [Internet] 2015; 53: 3-14 . Available from: http://dx.doi.org/10.1016/j.jbi.2014.12.007
• 137 Holden RJ, Professor A, Carayon P, Gurses AP, Professor A, Hoonakker P. et al. SEIPS 2.0: A human factors framework for studying and improving the work of healthcare professionals and patients NIH Public Access. Ergonomics [Internet] 2013; 56 (11) 1-30 . Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3835697/pdf/nihms521772.pdf
• 138 Carayon P, Schoofs Hundt A, Karsh BT, Gurses AP, Alvarado CJ, Smith M. et al. Work system design for patient safety: The SEIPS model. Qual Saf Heal Care 2006; 15 (SUPPL. 1): 50-8
• 139 Sittig. Dean F, Singh H. A New Socio-technical Model for Studying Health Information Technology in Complex Adaptive Healthcare Systems. Qual Saf Heal Care 2010; 19 (Suppl 3): i68-74
• 140 Kermani F, Orooji A, Sheikhtaheri A. Teleoncology for children with cancer: A scoping review on applications and outcomes. Int J Med Inform [Internet]. 2020 139. (November 2019):104118. Available from: https://doi.org/10.1016/j.ijmedinf.2020.104118
• 141 Kondylakis H, Koumakis L, Tsiknakis M, Kiefer S. Personally Managed Health Data: Barriers, Approaches, and a Roadmap for the Future. J Biomed Inform 2020; 106
• 142 Gambino O, Rundo L, Pirrone R, Vitabile S. HCI for biomedical decision-making: From diagnosis to therapy. J Biomed Inform 2020; 111: 103593
• 143 McGreevey JD, Mallozzi CP, Perkins RM, Shelov E, Schreiber R. Reducing Alert Burden in Electronic Health Records: State of the Art Recommendations from Four Health Systems. Appl Clin Inform 2020; 11 (01) 1-12
• 144 Venkatesh V, Morris MG, Davis GB, Davis FD. Quarterly. 2003; 27 (03) 425-78
• 145 Zhou T, Lu Y, Wang B. Integrating TTF and UTAUT to explain mobile banking user adoption. Comput Human Behav [Internet] 2010; 26 (04) 760-7 . Available from: http://dx.doi.org/10.1016/j.chb.2010.01.013