Evidence-Based Health Informatics as the Foundation for the COVID-19 Response: A Joint Call for Action

Abstract

Background As a major public health crisis, the novel coronavirus disease 2019 (COVID-19) pandemic demonstrates the urgent need for safe, effective, and evidence-based implementations of digital health. The urgency stems from the frequent tendency to focus attention on seemingly high promising digital health interventions despite being poorly validated in times of crisis.

Aim In this paper, we describe a joint call for action to use and leverage evidence-based health informatics as the foundation for the COVID-19 response and public health interventions. Tangible examples are provided for how the working groups and special interest groups of the International Medical Informatics Association (IMIA) are helping to build an evidence-based response to this crisis.

Methods Leaders of working and special interest groups of the IMIA, a total of 26 groups, were contacted via e-mail to provide a summary of the scientific-based efforts taken to combat COVID-19 pandemic and participate in the discussion toward the creation of this manuscript. A total of 13 groups participated in this manuscript.

Results Various efforts were exerted by members of IMIA including (1) developing evidence-based guidelines for the design and deployment of digital health solutions during COVID-19; (2) surveying clinical informaticians internationally about key digital solutions deployed to combat COVID-19 and the challenges faced when implementing and using them; and (3) offering necessary resources for clinicians about the use of digital tools in clinical practice, education, and research during COVID-19.

Discussion Rigor and evidence need to be taken into consideration when designing, implementing, and using digital tools to combat COVID-19 to avoid delays and unforeseen negative consequences. It is paramount to employ a multidisciplinary approach for the development and implementation of digital health tools that have been rapidly deployed in response to the pandemic bearing in mind human factors, ethics, data privacy, and the diversity of context at the local, national, and international levels. The training and capacity building of front-line workers is crucial and must be linked to a clear strategy for evaluation of ongoing experiences.

Keywords

mobile health - medical informatics - public health - COVID-19

Introduction

The current COVID-19 pandemic demonstrates that there is an urgent need to focus on evidence-based implementation of digital health. This gains paramount importance since there is frequently a tendency to focus attention on what could be highly promising despite being poorly validated in times of urgency. Rigor and evidence need to be taken into consideration to avoid delays and unforeseen negative consequences.

Further, this pandemic has created a unique opportunity to create quality data which can enable the achievement of the “Learning Healthcare System (LHS)” paradigm which implies that knowledge generated within the health care systems in the daily practice is used systematically to produce the continual improvement in care. This involves the intersection of digital platforms to create seamless efficient delivery of health care that also aligns itself continually to changing demands.[1] [2]

This, in turn will result in a long-term improvement of the quality of the health care system. This article is a joint call to action from the working groups and special interest groups of the International Medical Informatics Association (IMIA) for adopting an evidence-based approach toward deployment of digital health technologies during the current COVID-19 crisis. We believe that embracing an interdisciplinary scientific approach will not only consolidate the response, but will also reduce the risk of increasing health disparities and increase our digital preparedness for future challenges (e.g., pandemics and natural disasters).

Evidence-Based Health Informatics in Times of Crisis

The current COVID-19 pandemic is affecting health care systems across the globe in an unprecedented scale affecting not only the prevention and management of the COVID-19 emergency but also how health care is delivered as a whole. There is a rapid expansion of the use of health informatics innovations often overcoming legal and organizational barriers that have been in place for decades. The World Health Organization (WHO) defines in its web site eHealth as “the use of information and communication technologies for health care purposes.”[3] Further, the 71st World Health Assembly (WHA) in 2018 highlighted the importance of using digital technologies to reinforce public health resilience including “to build capacity for rapid response to disease incidents and public health emergencies,” as stayed in the minutes of the WHA.[4] These digital technologies include many different types of approaches and subtypes (e.g., telehealth, mobile health, online health and digital therapeutics), but all of them require, as any health technology, to be built on the best practices and evidence. The use of information technology (IT) in medicine and health care continues to evolve into different branches and focus areas. The discipline of health informatics is the interdisciplinary research field focused on how ITs can support the practice of health care and public health, this can be seen as a crucial part of medical informatics or more broadly biomedical informatics,[5] [6] thus encompassing the scientific foundations of innovation such as telemedicine, eHealth, mHealth, and a long list of terms to describe those informatics-based innovations in the health sector.

In the context of a public health crisis, access to accurate evidence-based information about how safe and effective the health informatics technologies are in aiding the public health interventions is of paramount importance.[7] [8] Clinical trials and data of different types (such as public health registries or Electronic Health Records) are rapidly emerging to validate therapeutic and preventive pharmacological treatments. However, eHealth solutions are deployed at a large scale often without rigorous and methodologically sound scientific assessment. Evidence has been published about the use of eHealth tools in previous public health emergencies such as the Ebola virus[9] [10] [11] [12] and the Zika virus.[13] In many instances, there have been reports about the need to ensure methodologically sound evaluation which will support evidence-based decision-making and building a strategy to reinforce and strengthen the health care systems to increase resilience and preparedness for the next crisis.[14]

Evidence is broadly defined as the available body of information that attests whether a proposition is valid. Sackett et al, in his definition of evidence-based medicine, perhaps the most well-developed area of evidence-based practice, have defined the field in terms of the “conscientious, explicit, and judicious use of the current best evidence in making decisions about the care of individual patients,”[15] and it has been further expanded into areas such as public health.[16] The contexts into which digital health systems are implemented are usually highly complex making randomized controlled trials (RCTs) challenging and often infeasible. This means that following a hierarchy of evidence that relies solely on RCTs and meta-analyses may not be appropriate nor practical for evidence-based health informatics where much of the body of available information on the effectiveness of informatics comes from observational studies, although there are fast growing numbers of RCTs in health informatics. Notwithstanding these differences, evidence should remain be the basis of clinical and public health practice. It enables decision-making to be based on the best available peer-reviewed quantitative and qualitative researches. This would mean systematic usage of data/information systems, application of planning frameworks such as learning health care systems models, community involvement in decision-making, and evaluation. Dissemination of knowledge to the key stakeholders is important, since acceptance of technology is often related to how it has been communicated and perceived, also in addition to the training and skills of intended users.

One of the main challenges stems from the sociotechnical nature of digital health technologies. These are complex interventions that require well-planned integration into complex organizational settings where human factors, as well as privacy and security, play a major role. Consequently, building evidence-based digital health strategies requires an interdisciplinary and intersectoral approach combining expertise across disciplines (e.g., health care, informatics, and management,) and sectors (e.g., health care systems, higher education, health technology industry, and policymakers) to avoid negative or unintended consequences.[17] We know that implementation challenges, including training and human factors, might hamper the translation of evidence into health practice, thus requiring an interdisciplinary approach.[8] Research on implementation challenges have been acquiring extra attention in recent years, including in eHealth, as a mean to facilitate the meaningful introduction of new solutions into the health domain.[18]

As in many health innovations, including pharmacological treatments, formative research needs to happen before clinical trials. In health informatics, usability and design research can provide early evidence on how innovations will be accepted into clinical practice. An additional challenge during the current novel coronavirus disease 2019 (COVID-19) pandemic is to ensure that experiences are shared rapidly as part of a “crisis informatics” approach,[19] and ensuring high-quality data.[20]

An interdisciplinary approach to study the use of ITs in the health care field is not new, and the field of health informatics has been an interdisciplinary endeavor for over 60 years. An example of such scholarly community is IMIA, which with 53 years of existence encompasses over 60 national medical informatics societies and regional associations, and over 20 groups dealing with special aspects via special interest groups, task forces, and working groups. Furthermore, the global approach of such type of scientific societies does allow for rapid sharing of knowledge and expertise not only across disciplines but also across geographies that might represent very different socioeconomic and cultural contexts. In addition, scientific societies do represent independent bodies where knowledge can be freely shared within the scientific principles. IMIA and its national and regional member societies and associations have already published some core recommendations in the context of COVID-19, aiming at guiding public health organizations.[19]

Objective

In this paper, we urge for a joint call for action to use and leverage evidence-based health informatics as the foundation for the COVID-19 response and public health interventions. We provide an overview of how the health informatics scientific community is helping to support the COVID-19 crisis response through tangible examples of how the working groups and special interest groups of IMIA are helping to build an evidence-based response to this crisis. Further, we provide some recommendations on key aspects that should be addressed and/or avoided related to the use of digital health during the current crisis based on decades of experience in health informatics research.

Methods

Using a qualitative approach, the IMIA board approached leaders of working groups and special interest groups via e-mail for an overview of the scientific efforts being taken as the COVID-19 pandemic was spreading across the globe. Each working group and special interest group compiled its activities and submitted a short summary of their efforts.

The process for creating this manuscript was led by the IMIA Vice President for Working Groups and Special Interest Groups. The IMIA Chief Executive Officer (CEO) sent e-mail invitations to the leadership of the working groups using the IMIA mailing database (currently 26 groups). That e-mail included short questions (web form) to describe the role of the different working groups (WGs)/special interest groups (SIG) during the COVID-19 crisis. After that, the WGs and SIGs were invited to participate in two brainstorming sessions using video conference where the role of WGs and SIGs were discussed and early versions of the manuscript were developed. Once a more matured version of the manuscript was ready, a second round of invitations to all WGs/SIGs was sent for comments or additional contributions. Discussions on the latest version of the manuscript were done by circulating the word documents for comments.

The Role of the Health Informatics Scientific Community and the International Medical Informatics Association

The IMIA WGs and SIGs have made contributions in the context of COVID-19 that is summarized in [Table 1].

Table 1 Current activities of selected IMIA WGs and SIGs related to COVID-19

WG/SIG	Description of the WG/SIG	WG/SIG contribution regarding COVID-19
Telehealth	The IMIA telehealth working group provides evidence and shares experience on the use of telemedicine and telehealth technologies including: ethical considerations of telehealth implementations,[27] training of health care professionals,[28] and governance[29]	Review global Telehealth initiatives for the management of COVID-19 such as the provision of virtual care. Compare strategies across countries to develop global telehealth guidelines for COVID-19 response and mitigation
Technology assessment and quality development	This working group promoted the evaluation of health technologies related to medical informatics, including evaluation of safety aspects[30] and other interdisciplinary aspects.[31] Including to promote the theory and practice of evidence-based eHealth by developing evaluation methods and tools to examine effects of IT intervention on health care structure, process and patient outcomes	Development of evidence-based guidelines for the design and deployment of digital health solutions during COVID-19. Including, supporting technology assessment for pandemic management at local, national and regional levels through advocacy and capacity building. We are calling for: 1. Evidence-based approach to IT interventions to ensure they are safe and effective 2. Rapid and pragmatic evaluation prior to deployment at front line, including iterative improvement cycles to ensure interventions have a plausible chance of working
Ethics, privacy, and security of health informatics	The Ethics, Privacy and Security of Health Informatics working group deals with the ethical handling of personal health information collected, used, and disclosed from treatment to analysis, reporting, and research. This WG is cross-cutting across many areas and inherently interdisciplinary, specially to address human factors	A particular element of key interest in the context of COVID-19 has been how to apply privacy protections on technologies design for surveillance and contact tracing
Language and meaning in BioMedicine	This working group focuses on formal and natural languages for expressing information and knowledge in the biomedical domain. This encompasses natural language processing, knowledge representation languages, design and use of biomedical ontologies, and global semantic interoperability. These include the application of Natural Language Processing technics, standardization, and also the use of best practices in data sharing[32] [33]	Data harmonization and initiatives related to COVID-19 sharing of data-driven knowledge under the FAIR (findability, accessibility, interoperability, and reusability) principles. Harmonization of data and of data harmonization efforts, in sync with Research Data Alliance (RDA) and Virus Outbreak Data Network (VODAN) To collaborate on a technology-agnostic semantic specification of data elements. To build up corpora of free text as multilingual training material for natural language processing. To foster collaboration between data creators, data modelers, and data users
Open source	Creation of medical software using the principles of open source, including the orchestration of social coding experiences, such as hackathons, and open science[34] by sharing core (e.g., in artificial intelligence applications) while protecting privacy[35]	1. Investigating CIVIC Tech (civil action with open source software and open data)[36] and promote it against COVID-19 pandemics and infodemics. A campaign “STAY HOME AND WRITE CODE, SAVE MORE LIVES” 2. Promotion to the activities against COVID-19 on GitHub. 3. Open data to the public organization
Students and emerging professionals	The group's role is to inform the new generation of informatics professionals and promote collaboration, placing a special emphasis on supporting interdisciplinary research in health informatics	The group collaboratively created a survey asking international clinical informaticians about key solutions and challenges in which health information technology helps to respond to COIVD-19 challenges. Preliminary survey findings can be found at http://covidhitimpact.com/
Nursing informatics	The focus of IMIA-NI is to foster collaboration among nurses and others who are interested in Nursing Informatics to facilitate development in the field. We aim to share knowledge, experience and ideas with nurses and health care providers worldwide about the practice of nursing informatics and the benefits of enhanced information management	IMIA-NI SIG and the European Federation of Medical Informatics Nursing Informatics (EFMI NI) are collaborating to offer resources to support nurses with materials about the use of digital tools during COVID-19 in clinical practice and education.[37] These include videos, articles and presentations, about how to use digital tools in clinical practice and education addressing daily practice, education and Research & Development
Participatory health and social media	This WG engages members from the international health informatics community, across sectors, to identify, explore, collaborate, and disseminate research on the use of social media for health. Of particular interest are the drivers of change, barriers, facilitators, and policies necessary for the application of the various social media categories in the health domain	Involved in several infoveillance studies analyzing COVID-19 related issues on social media Reviewing the existing evidence on the role of participatory health informatics in managing and detecting pandemics
Accident and emergency informatics	There is a need to interconnect the IT systems in the early rescue chain of the alerting, responding, and curing instances. This WG aims to foster sharing and semantic linkage of health data with environmental sensor data from smart implants and wearables to smart vehicles and homes, as well as future smart cities	In pandemic events, automatic exchange of information is needed across smart devices such as wearables, vehicles, or homes. We develop concepts to transform smart devices into diagnostic spaces including secured communication channels and semantic interoperability
Organizational and social issues	Given the increased implementation of health information technology and the focus on approaches, such as big data, patient participatory medicine and collaborative care delivery, it is more important than ever to ensure that organizational and social contexts are considered and studied as part of the design and evaluation of informatics-based solution Our objective is to develop and promote scholarly approaches for organizational and social issues in medical informatics research and care delivery	The global COVID-19 pandemic response has exposed significant gaps in information systems and processes to enable timely health decision-making. Our WG proposes to collaborate with the AMIA Global Health Informatics WG, AMIA Consumer and Pervasive Health Informatics WG to identify, review and summarize organizational issues related to information technology in health care, for example, care delivery models, access to care and technology, and effectiveness. Specifically, we will examine how the use of informatics could help support COVID-19 care delivery, and accelerate knowledge discovery bring to the forefront organizational issues
Smart homes and ambient assisted living	The aim of this working group is the study and promotion of research and development in the area of smart homes and ambient assisted living applications. While the situation at hospitals is receiving much of today's attention, a large part of the population has been or is still confined at their homes without proper access to health services or supervision. A “smart home” is a residential setting equipped with a set of advanced electronics, sensors and automated devices specifically designed for care delivery, remote monitoring, early detection of problems or emergency cases and promotion of residential safety and quality of life	Since capacities in hospitals are limited, most “mild” COVID-19 cases have been sent for quarantine at their homes, frequently without follow-up and limited possibilities for monitoring and exchange with medical professionals. A myriad of ease-to-use and affordable health monitoring solutions and other appliances for Smart Homes have been developed, amongst others by members of the WG, to help people who decide to remain at their homes and for health professionals to keep contact with their patients, including •Developing new models of virtual care to support remote monitoring and care planning due to COVID-19 •Exploring adaption and use of smart home, sensor technologies and wearable devices that can be applied to the management of individuals self-isolating at home for COVID-19 symptom development and for symptom management in the community
Health informatics for patient safety	The working group will focus on the following areas where health information systems are concerned: (1) Identifying and documenting how health information systems and their associated devices can best be designed, implemented and applied to improve patient safety), (2) Identifying and documenting software safety issues involving health information systems and their associated devices	We are currently involved in the following activities: •Evaluating the safety of health technologies that are being used to monitor and mitigate COVID-19's spread in the community The focus of our work has been on the following technologies: public health information systems, remote monitoring technologies for symptom monitoring, information systems to monitor the deployment of technologies focused on COVID-19 management, decision support systems for patients' self-assessment of symptoms and health professional decision support systems for diagnosis and management of COVID-19, and virtual care solutions[38]
Human factors engineering for health care informatics	Human Factors Engineering is the field of study which is concerned with the understanding of interactions of humans with elements of their work system, especially with the cognitive aspects of their interactions with health care technology.[39] This working group explores methods and practices in design and evaluation for studying the human–computer interaction in health care. We aim to enhance the understanding of the impact of interactive health technology design on health care processes to build evidence regarding design guidelines for optimal and safe interface designs for health informatics software[40]	Due to COVID-19, the uptake and use of interactive health technology by health care professionals and citizens has taken a flight forward. With regard to human factors research for health care informatics, we are currently performing a global research on the design aspects and acceptation factors of the official applications that have been introduced to monitor and mitigate the outbreaks of the COVID-19 pandemic. In addition, we are working on the development of a model to promote and support the performance of ethical review board assessment of user centered design research of health information technology[41] The objective of this model is to promote the performance of these studies in a way that respects the participants' integrity without undermining the innovation and the responsiveness of research teams, a prerequisite for coping with fast-spreading pandemics such as that of COVID-19

Abbreviations: COVID-19, novel coronavirus disease 2019; IMIA-NI, International Medical Informatics Association Nursing Informatics; IT, information technology; SIG, special interest group; WG, working group.

Results: A Call for Evidence-Based Informatics Response to COVID-19

With the COVID-19 outbreak, research concerned with forecasting and predictive analytics for syndromic surveillance[21] have received remarkable media attention. Increasing reliability and validity of forecasting or developing mechanisms for blending official datasets, like case statistics published by the World Health Organization or Centers for Disease Control and Prevention (CDC), with unofficial channels, such as data feeds from social media or telecommunications service providers,[22] seem to be “the” most important concern right now. Already, there are examples of meaningful data sharing initiatives such as the international consortium 4CE,[23] (p3) where electronic health record (EHR) data of COVID-19 patients from nearly a hundred hospitals is being shared. However, to a much lesser extent, researchers are focusing on organizational preparedness and postcrisis learning.[24] Even though there is strong evidence that a coordinated approach and small but directed changes in culture, processes, and IT-reliant solutions may prevent a breakdown of health care providers in times of crisis,[25] relatively little efforts have been made on this topic (as compared with crisis response). Our call to action is not only directed toward the crisis response but actually addressing a long-term perspective including preparedness and postcrisis learning.

Based on the combined discussion among the scientific working groups of the IMIA, we have created a list of actions that should take place during the current COVID-19 crisis ([Table 2]), as a mean to reinforce the response and health care systems with the best evidence-based knowledge in health informatics. Underpinning these recommendations is the expertise of the IMIA community in the multidisciplinary perspectives, understanding of human factors, and thoughtful and critical, ethical considerations that should be of central importance in the development and implementation of digital health tools that have been rapidly deployed in response to the pandemic. With these foundations in mind, this involves both things to avoid and things to promote. We should consider that the right approach will enable the creation of the global Learning Health System built on real-world evidence and robust scientific foundations. We consider that training and capacity building is of crucial importance to ensure recovery and preparedness for the next crisis. This needs to be linked to a clear strategy for evaluation of ongoing experiences, and the fair and meaningful practices for data sharing and privacy. All these aspects need to be considered at the local, national, and international levels through methodological planification and guidelines which include addressing ethics and human factors.

Table 2 IMIA WG/SIG Recommendations during the COVID-19 pandemic

What needs doing	What should be avoided
Training and capacity building: reinforcement of training of health care professionals and also students (both undergraduate and graduate) on the use of digital health tools for different tasks such as triage, surveillance, diagnosis, treatment and rehabilitation. This includes engaging students and emerging health informaticians in creating solutions for COVID-19 pandemics. Community Health workers, who are the major providers in developing countries, must be empowered with evidence-based tools, including mobile health tools, to help them acquire accurate information about COVID-19, help treat and diagnose their patients, and educate their communities	Disempowering patients by not engaging patients in systems design or not providing patient education and counseling using digital tools Increasing the digital divide and health inequalities across communities and countries by creating better services for people with better technological means
Evaluation: Consolidate evidence on real-world applications used during the COVID-19 pandemic, including an assessment of how COVID-19 has impacted health/clinical practice using digital tools to define a threshold for future health care delivery. Take an evidence-based approach to IT based interventions to ensure they are safe and effective. IT interventions should be evaluated prior to deployment at the front line, but ensuring that evaluation should is rapid and pragmatic, including iterative improvement cycles to ensure they have a plausible chance of working	Developing initiatives without involving multiple stakeholders relevant for sustainability (e.g., clinicians, patients, payors, and regulators) Initiate pilots or any initiative without an assessment of sustainability in the long run
Data sharing: define strategies for sharing structured and standardized data relevant to the crisis, including trained models for risk prediction. Also, establishing automatic exchange of information, e.g., COVID-19 test results to ensure complete data, better statistics, and avoids delays. Including the use of Findable, Accessible, Interoperable, and Reusable (FAIR) principles,[42] standardized terminologies and classification systems	Creating data silos and sharing data of low quality that might lead to misguiding conclusions
Data privacy: to ensure privacy we should apply the principles of privacy by design which minimizes potential risks before any system is launched. Including the prevention of potential cyberattacks to health information systems or the design of contact tracing solutions that pose a risk to the privacy of citizens. This includes the need of combining telemedicine with the secure and standardized transmission of health information.[43] Emphasize the need of combining telemedicine with the secure and standardized transmission of health information	Eroding an individual's universal right to privacy in the midst of a crisis situation such as publicly releasing anonymized information on morbidity and mortality that could reidentify individuals, leading to racial discrimination, stigma and bias Adoption of less secure technologies for the transmission of personal data such as unsecured short messaging systems (SMS) versus secured electronic prescription or other encrypted systems
Planification: development of national and international guidelines on how  •To provide telemedicine/eHealth services including when/how to prescribe it  •To protect patient safety and privacy, including data confidentiality  •To pilot and validate of health care devices, technologies, and biomedical testing during times of crisis  •To tackle social media misinformation  •To ensure that digital health interventions are well positioned with the organization or country's existing national strategic strategies and infrastructure  •To involve health care professionals, patients, payors, and regulatory bodies on the organization of telemedicine when face-to-face care delivery is not possible due to epidemiological crisis  •Understand contextual differences across health systems and its impact on our ability to share informatics strategies	Development of unregulated telemedicine practices that put into legal risks both patients and health care professionals Implementation of telemedicine without considering patient safety, local culture, and other contextual factors Lack of analysis of impact of new technologies in the workload of health care professionals. Run into data lock-in, project lock-in, or vendor lock-in
Ethics and human factors: define potential ethical impacts of rapid deployment of health technologies, including impact on stigmatization of segments of the population, increase of health disparities, and any other human and ethical factors. Involving professionals, patients, and civil society in a systematic way is the best approach to minimize unintended negative consequence of health technologies	Deploying digital health technologies without assessing its impact on ethical, social and organizational considerations, as well as its impact on reducing disparities in access and delivery of health care services

Abbreviations: COVID-19, novel coronavirus disease 2019; IMIA, International Medical Informatics Association; IT, information technology; SIG, special interest group; WG, working group.

Conclusion: A Call for Interdisciplinary Collaboration in Digital Health during the COVID-19 Pandemic

Collaboration is our recommendation as the best way forward toward a more robust and equitable global public health system after the COVID-19 pandemic. The involvement and collaboration of multidisciplinary stakeholders across sectors (i.e., policymakers, governments, research institutes, consumers, and others) can foster and enable the desired outcomes and health system. Therefore, we do call on other scientific societies and any stakeholders involved in the crisis response, including consumers of health care services, to proactively seek collaboration with the IMIA working groups, as well as with national and regional associations that do have also related working groups. In this paper, we provide a substantial corpus of knowledge and evidence; however, we should consider it to be limited due to the exponential growth on research and implementation of digital health. To get actionable insights from the implementation of digital health during the COVID-19 is going to be a research tasks for many years to come.

Together, we can move digital health from hope and hype to reality and in the service of consumers and public health. To do that, we would like to encourage the wider scientific communities to raise awareness about evidence-based digital approaches for COVID-19 by disseminating them in social media, publishing complementary viewpoints, and consensus statements, so we can be better prepared for the next crisis both at the microlevel (e.g., patient interaction), mesolevel (health care organization and community), and macrolavel (e.g., policy).[26]

Conflict of Interest

L.F.L. is Chief Scientific Officer and shareholder at Adhera Health Inc (USA). All the other authors report no conflict of interest.

Note: For complete IMIA affiliations, please refer to Appendix A.

• References

• 1 Friedman C, Rigby M. Conceptualising and creating a global learning health system. Int J Med Inform 2013; 82 (04) e63-e71
• 2 Institute of Medicine (US). Grossmann C, Powers B, McGinnis JM. eds. Digital Infrastructure for the Learning Health System: The Foundation for Continuous Improvement in Health and Health Care: Workshop Series Summary. Washington, DC: National Academies Press (U.S.); 2011
• 3 WHA eHealth. Resolutions and decisions. Accessed March 2, 2021 at: https://apps.who.int/iris/bitstream/handle/10665/20378/WHA58_28-en.pdf;jsessionid=731E7D57BD9F308B4E115BC4FA03EBBB?sequence=1
• 4 Digital Health WHO. Seventy first World Health Assembly. Accessed March 2, 2021 at: https://apps.who.int/gb/ebwha/pdf_files/WHA71/A71_R7-en.pdf?ua=1
• 5 Kulikowski CA, Shortliffe EH, Currie LM. et al. AMIA Board white paper: definition of biomedical informatics and specification of core competencies for graduate education in the discipline. J Am Med Inform Assoc 2012; 19 (06) 931-938
• 6 Bath PA. Health informatics: current issues and challenges. J Inf Sci 2008; 34 (04) DOI: 10.1177/0165551508092267.
• 7 Ammenwerth E, Schnell-Inderst P, Siebert U. Vision and challenges of Evidence-Based Health Informatics: a case study of a CPOE meta-analysis. Int J Med Inform 2010; 79 (04) e83-e88
• 8 Ammenwerth E. Evidence-based health informatics: how do we know what we know?. Methods Inf Med 2015; 54 (04) 298-307
• 9 Kummervold PE, Schulz WS, Smout E, Fernandez-Luque L, Larson HJ. Controversial Ebola vaccine trials in Ghana: a thematic analysis of critiques and rebuttals in digital news. BMC Public Health 2017; 17 (01) 642
• 10 Sacks JA, Zehe E, Redick C. et al. Introduction of mobile health tools to support ebola surveillance and contact tracing in Guinea. Glob Health Sci Pract 2015; 3 (04) 646-659
• 11 Tom-Aba D, Olaleye A, Olayinka AT. et al. Innovative technological approach to ebola virus disease outbreak response in nigeria using the open data kit and form hub technology. PLoS One 2015; 10 (06) e0131000
• 12 Mandl KD. Ebola in the United States: EHRs as a public health tool at the point of care. JAMA 2014; 312 (23) 2499-2500
• 13 Faria NR, Sabino EC, Nunes MRT, Alcantara LCJ, Loman NJ, Pybus OG. Mobile real-time surveillance of Zika virus in Brazil. Genome Med 2016; 8 (01) 97
• 14 World Health Organization. WHO guideline Recommendations on Digital Interventions for Health System Strengthening. Accessed march 2, 2021 at: https://www.ncbi.nlm.nih.gov/books/NBK541902/pdf/Bookshelf_NBK541902.pdf
• 15 Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ 1996; 312 (7023): 71-72
• 16 Brownson RC, Baker EA, Deshpande AD, Gillespie KN. Evidence-Based Public Health. Third Edition. New York, NY: Oxford University Press; 2017
• 17 Koppel R, Chen Y. Consequences SE for the IYSS on U. Unintended consequences: new problems, new solutions. Yearb Med Inform 2016; 25 (01) 87-92
• 18 Glasgow RE, Phillips SM, Sanchez MA. Implementation science approaches for integrating eHealth research into practice and policy. Int J Med Inform 2014; 83 (07) e1-e11
• 19 –International Medical Informatics Association. IAHSI statement to WHO on the use of informatics in pandemic situations. Accessed December 1, 2020 at: https://imia-medinfo.org/wp/statement-from-the-international-academy-for-health-sciences-informatics-iahsi-the-academy-of-the-international-medical-informatics-association-imia-to-the-director-general-of-the-who-on-the-use/
• 20 Galvin CJ, Fernandez-Luque L, Li Y-C. Accelerating the global response against the exponentially growing COVID-19 outbreak through decent data sharing. Diagn Microbiol Infect Dis 2020; (e-pub ahead of print) DOI: 10.1016/j.diagmicrobio.2020.115070.
• 21 Samaras L, García-Barriocanal E, Sicilia M-A. Syndromic surveillance using web data: a systematic review. Innov Health Inform 2020; 39-77 DOI: 10.1016/B978-0-12-819043-2.00002-2.
• 22 Nambisan P, Luo Z, Kapoor A, Patrick TB, Cisler RA. Social Media, Big Data, and Public Health Informatics: Ruminating Behavior of Depression Revealed through Twitter. Presented in 48th Hawaii International Conference on System Sciences; January 5–8, 2015; Kauai, HI:
• 23 Brat GA, Weber GM, Gehlenborg N. et al. International electronic health record-derived COVID-19 clinical course profiles: the 4CE consortium. NPJ Digit Med 2020; 3 (01) 109
• 24 Williams TA, Gruber DA, Sutcliffe KM, Shepherd DA, Zhao EY. Organizational response to adversity: fusing crisis management and resilience research streams. Acad Manag Ann 2017; 11 (02) 733-769
• 25 Veenema TG. ed. Disaster Nursing and Emergency Preparedness for Chemical, Biological, and Radiological Terrorism, and Other Hazards. New York, NY: Springer Publishing Company; 2019
• 26 World Health Organization. Innovative Care for Chronic Conditions: Building Blocks for Action: Global Report. Accessed March 2, 2021 at: https://www.who.int/chp/knowledge/publications/icccglobalreport.pdf?ua=1
• 27 Kuziemsky CE, Hunter I, Gogia SB. et al. Ethics in Telehealth: Comparison between Guidelines and Practice-based Experience -the Case for Learning Health Systems. Yearb Med Inform 2020; 29 (01) 44-50
• 28 Kulatunga GGAK, Marasinghe RB, Karunathilake IM, Dissanayake VHW. Development and implementation of a web-based continuing professional development (CPD) programme on medical genetics. J Telemed Telecare 2013; 19 (07) 388-392
• 29 Marcelo A, Ganesh J, Mohan J. et al. Governance and management of national telehealth programs in Asia. Stud Health Technol Inform 2015; 209: 95-101
• 30 Ammenwerth E, Rigby M. eds. Evidence-Based Health Informatics: Promoting Safety and Efficiency through Scientific Methods and Ethical Policy. Studies in Health Technology and Informatics). Amsterdam, the Netherlands: IOS Press; 2016
• 31 Scott P, de Keizer N, Georgiou A. eds. Applied Interdisciplinary Theory in Health Informatics: A Knowledge Base for Practitioners (Studies in Health Technology and Informatics). Amsterdam, the Netherlands: IOS Press; 2019
• 32 Balkanyi L, Cornet R. The interplay of knowledge representation with various fields of artificial intelligence in medicine. Yearb Med Inform 2019; 28 (01) 27-34
• 33 Jacobsen A, de Miranda Azevedo R, Juty N. et al. FAIR Principles: Interpretations and Implementation Considerations. Data Intell 2019; 2 (1,2): 10-29
• 34 Paton C, Kobayashi S. An open science approach to artificial intelligence in healthcare. Yearb Med Inform 2019; 28 (01) 47-51
• 35 Kobayashi S, Kane TB, Paton C. The privacy and security implications of open data in healthcare. Yearb Med Inform 2018; 27 (01) 41-47
• 36 MOSS, Japan. CIVIC Tech challenges to COVID-19. https://www.moss.gr.jp/post/612095052398919680/civic-tech-challenges-to-covid-19 . Accessed December 1, 2020
• 37 IMIA NI. IMIA. Published January 25, 2016. Accessed December 1, 2020 at: https://imia-medinfo.org/wp/sig-ni-nursing-informatics/
• 38 Borycki E, Cummings E, Dexheimer JW. et al; Contribution of the IMIA Working Group on Patient Safety. Patient-centred coordinated care in times of emerging diseases and epidemics. Yearb Med Inform 2015; 10 (01) 207-215
• 39 Beuscart-Zéphir MC, Borycki E, Carayon P, Jaspers MW, Pelayo S. Evolution of human factors research and studies of health information technologies: the role of patient safety. Yearb Med Inform 2013; 8 (01) 67-77
• 40 Marcilly R, Peute L, Beuscart-Zéphir M-C. From usability engineering to evidence-based usability in health IT. Stud Health Technol Inform 2016; 222: 126-138
• 41 Peute LW, Lichtner V, Baysari MT. et al. Challenges and best practices in ethical review of human and organizational factors studies in health technology: a synthesis of testimonies. Yearb Med Inform 2020; 29 (01) 58-70
• 42 Wilkinson MD, Dumontier M, Aalbersberg IJJ. et al. The FAIR guiding principles for scientific data management and stewardship. Sci Data 2016; 3: 160018
• 43 Kuziemsky CE, Gogia SB, Househ M, Petersen C, Basu A. Balancing health information exchange and privacy governance from a patient-centred connected health and telehealth perspective. Yearb Med Inform 2018; 27 (01) 48-54

Publication History

Received: 04 December 2020

Accepted: 19 January 2021

Article published online:
11 May 2021