International Comparison of Six Basic eHealth Indicators Across 14 Countries: An eHealth Benchmarking Study

Keywords

health information exchange - health information systems - continuity of patient care - patient-centered care - eHealth - international health

Introduction

Modern health care strongly depends on the availability of health-related information. Information and communication systems, which form the technical foundation of eHealth, are thus increasingly introduced in health care institutions worldwide.

The aim of eHealth is to increase the quality and efficiency of care, reduce costs for clinical services, reduce administrative costs of the health care system, and enable new models of health care delivery.[1] Evidence shows that eHealth can support the quality and efficiency of patient care.[2] [3]

In many countries, we can see a shift from institution-centered to patient-centered information processing. This means that the fragmented pieces of a patient's health data distributed in isolated silos throughout the different institutions involved in his/her care are now consolidated within a common patient record, which integrates data across the boundaries of health care institutions.[4] eHealth, which is understood as “a commitment for networked, global thinking, to improve health care locally, regionally, and worldwide,”[5] facilitates this shift.

Countries show different progress using eHealth to support information exchange and patient-centered information processing. International organizations have designed and deployed eHealth indicators to explain differences between countries. For example, the Organization for Economic Cooperation and Development (OECD) has developed a guide for measuring eHealth indicators.[1] The Nordic eHealth Research Network published eHealth indicators to be used in the Nordic region.[6] [7] The European Commission publishes a report on the dissemination of eHealth functionality among general practitioners within the European Union (EU) member states in every 5 years.[8] The World Health Organization (WHO) provides statistics for several eHealth indicators in European member countries.[9] In the United States, the electronic health record (EHR) meaningful use incentive program requires participating care providers to demonstrate fulfillment of several functionality criteria as a prerequisite for receiving incentive payments.[10]

While these eHealth benchmarking approaches give important insights into eHealth adoption in different countries, they are not able to provide a clear picture of the cross-institutional availability of clinical information for clinicians and patients. Clinical information is at the core of eHealth and its availability to clinicians and patients across institutional boundaries is a precondition for patient-centered care.

In this paper, we apply six eHealth indicators and present the results of these indicators in 14 countries. We then compare our approaches to other available eHealth benchmarks.

Objectives

This study was aimed to present eHealth indicators on cross-institutional availability of major types of patient data for health care professionals and patients and their caregivers across 14 countries. “Availability of patient data” was defined as having the ability to access data and to add data to the patient record.

This study also focused to extend and to highlight changes for those seven countries that already participated in the eHealth indicator survey in 2017.[11]

Comparing our indicators and the resulting data for the examined countries with other related eHealth benchmarks was also studied.

Methods

Indicators and Survey Instrument

The survey instrument focuses on six basic eHealth indicators that cover two major aspects. First, the immediate access of health care professionals, patients, and (informal) caregivers (e.g., relatives) to patients' major relevant health data stored in EHRs. Second, the possibility of health care professionals, patients, and their caregivers to immediately add data to an EHR.

The indicators were defined and tested in a survey of six countries in 2017.[11] Details of the development of the items and the underlying motivation for selecting these items have been published.[11]

The survey contains 60 items that we aggregate to the six major eHealth indicators as presented in [Table 1].

Table 1 Six basic eHealth indicators used in the eHealth indicator survey 2017 and 2019

Abbreviations	eHealth indicator	Explanation
AH	Access of healthcare professionals to their patients' data:immediate access of physicians, nurses, and pharmacists to their patients' EHRs' major relevant data from other health care institutions where the patient received care	The 21 items assessed whether diagnoses, medication, and problems documented in one institution were available for health care professionals in other institutions.
AP	Access of patients to their data:immediate access of patients to their EHRs' major relevant data from selected health care institutions where the patient received care	The 15 items assessed whether diagnoses, medication, and problems documented by a healthcare institution were available for the patient.
AC	Access of caregivers to patients' data:immediate access of caregivers to the patients' EHRs' major relevant data from selected health care institutions where the patient received care	The 15 items assessed whether diagnoses, medication, and problems documented by a healthcare institution were available for authorized caregivers of the patient.
EH	Adding data by health care professionals:physicians, nurses, and pharmacists of selected health care institutions are able to add data immediately to the patients' EHR(s)	The seven items assessed whether health care professionals were able to electronically document patient-related data in a (local or shared) EHR system.
EP	Adding data by patients:patients are able to add data immediately to their EHR(s)	The item assessed whether the patient is able to add data in electronic form to a (local or shared) EHR system to make them available to health care professionals and themselves.
EC	Adding data by caregivers:caregivers are able to add data immediately to the patients' EHR(s) in a respective country	The item assessed whether the patient's caregivers are able to add data in electronic form to a (local or shared) EHR system to make it available to health care professionals and themselves.

Abbreviations: AC, access of caregivers to the patients' health record data; AH, access of healthcare professionals to their patients' data; AP, access of patients to their health record data; EC, enabling caregivers to add data to the patients' health record(s); EH, adding data by health care professional(s); EHR, electronic health record; EP, enabling patients to add data to their health record(s).

Note: The explicit scope of all questions was the experts' respective country.

Please note that throughout the survey, we chose the following naming conventions:

• EHR: to accommodate for various interpretations of this term in the participating countries, we defined EHR as “any patient-centered electronic collection of clinical data or documents generated as a by-product of patient care.” According to ISO 20514 (ISO: International Organization for Standardization), an EHR may be managed by local (institutional) EHR systems, as well as by shared (i.e., transinstitutional, regional, or national) EHR systems.

• Health care institution: a health care institution provides inpatient and/or outpatient care for a patient. We focused on selected health care institutions expected to exist in most participating countries. These institutions include hospitals, medical offices (e.g., physicians in private practice or with health maintenance organization [HMO]), contract(physicians in an ambulatory clinic), nursing homes (institutions for long-term nursing care of elderly or chronically ill patients), outpatient nursing organizations (providing ambulatory nursing care at the patient's home), and pharmacies.

• Major relevant data: EHRs contain a large variety of data elements. We focused on major data elements considered central for patient care, namely, diagnoses, medication, and problems (e.g., allergies).

• Immediate access and immediate adding of data: accessing and adding data are seen as “immediate” if they are available through electronic means and without the need for special requests; paper-based information processing cannot provide this.

• Caregivers: persons informally taking care of patients such as family members or parents (for their children).

Organization of the Survey

Two experts from each of the following countries were invited to provide the indicator data for their country: Argentina, Australia, Austria, Finland, Germany, Hong Kong, Israel, Japan, Jordan, Kenya, South Korea, Sweden, Turkey, and the United States.

The rationale behind the selection of the 14 countries was as follows: Seven countries (Austria, Finland, Germany, Hong Kong, South Korea, Sweden, and the United States) already participated in the eHealth indicator survey 2017 and were thus chosen to allow an analysis of an eventual progress made. The remaining seven countries were chosen according to our goal to include representative countries from the remaining three continents (we did not consider Antarctica) and including a broad spectrum of different health care systems and various levels of eHealth adoption. The selection of the experts from each country was based on their known scientific activity in national eHealth projects.

As eHealth activities are advancing in many of these countries, all responses had to reflect the situation in the respective country on August 1, 2019 as date of reference.

The two experts in each participating country were asked to complete the survey, to solve any discrepancies through discussion, and to submit a consensus version. For each item, available answering options were as follows:

• “−”: not given, meaning that the respective indicator item is not fulfilled or only fulfilled in a clear minority of institutions, health care professionals, or patients (<20%).

• “+”: partly given, meaning that the respective indicator item is fulfilled in some (but not all) institutions, health care professionals, or patients.

• “++”: given, meaning that the respective indicator item is fulfilled in the large majority of institutions, health care professionals, or patients.

• N/A: not applicable.

The experts were also asked to provide short descriptions for all “partly given” answers and where they saw the need to provide additional context. E.A. and R.H. carefully reviewed and discussed all answers, contacted the experts in the event of clarification needed, and documented revised answers accordingly.

All authors explicitly released the final version of their respective survey results. Complying with rules for good scientific practice, all final questionnaires are available from the corresponding author.

Calculation of Indicators

[Table 2] summarizes the formulas for the 2019 survey. Based on the 60 items, we calculated the six eHealth indicators for each country using mainly the published formulas and motivations for the 2017 survey[11] with the following two clarifications:

• Indicator AH (access of healthcare professionals to their patients' data): for pharmacies to achieve an overall “+,” it is now sufficient if they have at least access (i.e., “+”) to medication data. Rationale: in many countries, there is no plan for pharmacies to have access to diagnoses and problems at all. Thus, grading pharmacies as “−” that have access to medication data, but not to diagnoses and problems, does not seem fair. Therefore, we changed this calculation in the 2019 survey. The changed wording is highlighted in [Table 2].

• Indicator EH (enabling health care professionals to add data to their patients' health record[s]; adding data by health care professionals): we now award an overall “+” if three or more outcome values were at least “+” and one or more outcome value was “N/A.” Rationale: In one country (Jordan), not all listed types of health care institutions exist independently, thus “N/A” is a valid answer and calculating it as “−” as in the 2017 survey did not seem adequate. The changed wording is highlighted in [Table 2].

Table 2 Indicator definition of the six eHealth indicator survey 2019. Outcome values can be given (++, green), partially given (+, yellow) or not given (−, red)

No.	Abbreviation	Indicator calculation
1	AH	Calculation of this eHealth indicator is based on 3 × 7 = 21 outcome values:access to (1) diagnoses, (2) medication, and (3) problemsfrom hospitals (by (1) physicians, (2) nurses, (3) pharmacists), from (4) medical offices (by physicians), from (5) nursing homes (by nurses), from (6) outpatient nursing organizations (by nurses), and from (7)pharmacies (by pharmacists). Outcome values on access to diagnoses, medication, and problems from each health care institution by the selected health care professionals were defined as follows: a) Access is ++ if all three outcome values for diagnoses, medication, and problems are ++ or if two are ++ and one is +. b) Access is + if one outcome value is ++ and two are + or if all three outcome values are +; exception: for pharmacies, a + was given if the outcome value for medication is at least +. c) Access is − in all other cases. Based on the then remaining sevenvalues this eHealth indicator was calculated in the sequence: 1) Indicator is ++ if ≥ 4 outcome values are ++. 2) Indicator is + if ≥ 2 outcome values are ++ or ≥ 3 values are ++ or +. 3) Indicator is − in all other cases.
2, 3	AP, AC	Calculation of these eHealth indicators is based on 3 × 5 = 15 outcome values: Access to (1) diagnoses, (2) medication, and (3) problemsfrom (1) hospitals, (2) medical offices, (3) nursing homes, (4) outpatient nursing organizations, and (5) pharmacies. For the values of diagnoses, medication, and problems for each selected health care institution we used the same definition of values as for indicator AHr. Based on the remaining five values this eHealth indicator was calculated in the sequence: 1) Indicator is ++ if ≥ 3 outcome values are ++ and ≤ 2 outcome values are -. 2) Indicator is + if ≥ 2 outcome values are ++ or + 3) Indicator is − in all other cases.
4	EH	This eHealth indicator is based on seven outcome values, adding data in hospitals (by (1) physicians, (2) nurses, (3) pharmacists), in (4) medical offices (by physicians), in (5) nursing homes (by nurses), in (6) outpatient nursing organizations (by nurses), and in (7) pharmacies (by pharmacists). It was calculated in the sequence: 1) Indicator is ++ if ≥ 4 outcome values are ++. 2) Indicator is + if ≥ 2 outcome values are ++ or ≥ 4 outcome values are at least +or ≥ 3 outcome values are at least + and ≥ 1 outcome value is N/A. 3) Indicator is − in all other cases.
5, 6	EP, EC	The outcome value itself served as the eHealth indicator. No calculation was necessary.

Abbreviations: AC, access of caregivers to the patients' health record data; AH, access of healthcare professionals to their patients' data; AP, access of patients to their health record data; EC, enabling caregivers to add data to the patients' health record(s); EH, adding data by health care professional(s); EP, enabling patients to add data to their health record(s).

Notes: Indicator definitions are taken from the eHealth indicator survey 2017 (p. 708), with two clarifications (underlined; see main text for explanation).

Additional value not applicable is visualized as N/A and in black.

Results

We received responses from all invited countries. With the exception of Kenya, for all countries, at least two experts provided consensus data.

[Table 3] presents the outcome for the eHealth indicator AH, as well as changes compared to the 2017 survey. Survey responders added additional explanations that are summarized in [Appendix 1].

Table 3 Outcome for eHealth indicator AH as of August 1, 2019, for Argentina (RA), Australia (AUS), Austria (A), Finland (FIN), Germany (D), Hong Kong (HK), Israel (IL), Japan (J), Jordan (JOR), Kenya (EAK), South Korea (ROK), Sweden (S), Turkey (TR), and the United States (USA)

Can selected health care professionals from selected health care institutions immediately access relevant patient data from other institutions?		Countries
Selectedhealth care institutions	Selected groups ofhealth careprofessionals	A	AUS	D	EAK	FIN	HK	IL	J	JOR	RA	ROK	S	TR	USA
Hospitals	Physicians	→+	+	→−	−	→++	+	++	+	−	+	→++	++	→+
	Nurses	→+	+	→−	−	→++	+	++	+	−	+	→++	+	→+
	Pharmacists	+	+	→−	−	→++	+	++	+	−	+	→+	++	→+
Medical offices	Physicians	+	+	→−	−	→++	++	+	−	+	→++	+	→+
Nursing homes	Nurses	→−	−	→−	−	→++	+	++	N/A	−	→−	→+	+	→−
Outpatient nursing org.	Nurses	→−	+	→−	−	→++	++	N/A	−	→−	++	+	→−
Pharmacies	Pharmacists	+	+	→−	−	→+	→-	+	−	→−	→+	++	→+
Indicator 1 AH		→+	+	→−	−	→++	+	++	+	−	+	++	+	→+

Abbreviation: AH, access of healthcare professionals to their patients' data.

Note: For those countries, who participated also in the 2017 survey: → same indicator as 2017, indicator changed from − to + or from + to ++. N/A = not applicable. Survey responders added additional explanations that are summarized in [Appendix 1].

Overall, four countries (Finland, Hong Kong, Japan, and Sweden) received a “++” in this indicator. Compared to the 2017 survey, Austria, South Korea, and Sweden improved in some items and South Korea, as well as Sweden, also improved their overall indictor AH.

[Table 4] presents the outcome for eHealth indicator AP(access of patients to their health record data) and eHealth indicator AC (access of caregivers to the patients' health record data), as well as changes compared to the 2017 survey. Survey responders added additional explanations that are summarized in [Appendix 2].

Table 4 Outcome for eHealth indicators AP and AC as of August 1, 2019, for Argentina (RA), Australia (AUS), Austria (A), Finland (FIN), Germany (D), Hong Kong (HK), Israel (IL), Japan (J), Jordan (JOR), Kenya (EAK), South Korea (ROK), Sweden (S), Turkey (TR), and the United States (USA)

Can the patient and patient's caregivers access major patient data from selected health care institutions?	Country
Selected health care institutions	A	AUS	D	EAK	FIN	HK	IL	J	JOR	RA	ROK	Sa	TR	USA
Hospitals	++	+	→−	−	→++	→−	−	→++	++	++	→+
Medical offices	→−	+	→−	−	→++	→−	−	→++	++	+	→+
Nursing homes	→−	−	→−	−	→++	→−	−	→++	→−	−	→−
Outpatient nursing organizations	→−	−	→−	−	→++	→−	−	→++	→−	+	→−
Pharmacies	+	+	→−	−	→++	→−	−	→+	−	→−
Indicators 2 APr and 3 ACr	+	+	→−	−	→++	→−	−	→++	→+	+	→+

Abbreviations: Abbreviations: AC, access of caregivers to the patients' health record data; AP, access of patients to their health record data.

Note: For those countries, who participated also in the 2017 survey: → same indicator as 2017, indicator changed from – to + or from + to ++. N/A = not applicable. Survey responders added additional explanations that are summarized in[Appendix 1].

^a In Sweden, access of patients and access of caregivers differ. The entry in the table is with respect to patients. For caregivers in Sweden, all entries are “−” indicator AC decreases to “−”.

Overall, two countries (Finland and South Korea) received a “++” in these indicators. Compared to the 2017 survey, only Austria and Sweden improved some items. In the case of Austria, this led to an improvement of the overall indicators AP and AC.

[Table 5] presents the outcome for the eHealth indicator EH, as well as changes compared to the 2017 survey. Survey responders added additional explanations that are summarized in [Appendix 3].

Table 5 Outcome for eHealth indicator EH as of August 1, 2019, for Argentina (RA), Australia (AUS), Austria (A), Finland (FIN), Germany (D), Hong Kong (HK), Israel (IL), Japan (J), Jordan (JOR), Kenya (EAK), South Korea (ROK), Sweden (S), Turkey (TR), and the United States of America (USA)

Can selected health care professionals from selected health care institutions add relevant patient data to a (local or shared) electronic health record?		Country
Selected health care institutions	Selected groups of health care professionals	A	AUS	D	EAK	FIN	HK	IL	J	JOR	RA	ROK	S	TR	USA
Hospitals	Physicians	→++	++	→++	+	→++	++	+	→++	→++	++	→++
	Nurses	→++	++	→++	+	→++	+	++	+	→++	→++	++	→++
	Pharmacists	→−	++	→+	+	→++	+	−	++	−	+	→−	→−	++	→++
Medical offices	Physicians	→++	++	→++	+	→++	++	+	→++	→++	+	→++
Nursing homes	Nurses	+	+	→-	+	→++	→−	++	N/A	−	→++	→++	−	→+
Outpatient nursing org.	Nurses	→−	+	→−	+	→++	+	−	++	N/A	−	→++	→++	+	→+
Pharmacies	Pharmacists	+	+	→+	−	→+	→−	−	→−	→−	−	→++
Indicator 4 EHr		→+	++	→+	+	→++	→+	++	+	→++	→++	+	→++

Abbreviation: EH_r, adding data by health care professional(s).

Note: For those countries, who participated also in the 2017 survey: → same indicator as 2017, indicator changed from − to + or from + to ++. N/A = not applicable. Survey responders added additional explanations that are summarized in [Appendix 1].

Overall, seven countries (Australia, Finland, Israel, Japan, South Korea, Sweden, and the United States) received a “++” in this indicator. Compared to the 2017 survey, Austria and Hong Kong improved some items but without effect on the overall indicator.

[Table 6] presents the outcome for eHealth indicators EP (enabling patients to add data to their health record[s]) and EC(enabling caregivers to add data to the patients' health record[s]), as well as changes compared to the 2017 survey. Survey responders added additional explanations that are summarized in [Appendix 4].

Table 6 Outcome for eHealth indicators EP and EC as of August 1^st, 2019, for Argentina (RA), Australia (AUS), Austria (A), Finland (FIN), Germany (D), Hong Kong (HK), Israel (IL), Japan (J), Jordan (JOR), Kenya (EAK), South Korea (ROK), Sweden (S), Turkey (TR), and the United States (USA)

Indicator				c	o	u	n	t	y
No.	Abbreviation	Name	A	AUS	D	EAK	FIN	HK	IL	J	JOR	RA	ROK	S[a]	TR[b]	USA
5	EP	Enabling patients to add data to their health record(s).	→−	+	→−	−	→+	→−	−	+	→−	→+	++	→+
6	EC	Enabling caregivers to add data to the patients' health record(s).

Abbreviations: EC, enabling caregivers to add data to the patients' health record(s); EP, enabling patients to add data to their health record(s).

Note: For those countries, who participated also in the 2017 survey: → same indicator as 2017, indicator changed from – to + or from + to ++. Survey responders added additional explanations that are summarized in [Appendix 1].

^a In Sweden, enabling of patients and of caregivers to add data differ. The entry in the table is with respect to patients. For caregivers in Sweden, the entry is “−.” Indicator EC thus decreases to “−.”

^b In Turkey, enabling of patients and of caregivers to add data differ. The entry in the table is with respect to patients. For caregivers in Turkey, the entry is “+.”

Overall, only Turkey received a “++” in this indicator. Compared to the 2017 survey, no country showed an improved indicator.

[Table 7] summarizes the six eHealth indicators for all countries. No country received a “++” for all indicators. Finland received a “++” for four indicators, South Korea for three indicators, Japan and Sweden for two indicators, and Australia, Hong Kong, Israel, Turkey, and the United States for one indicator.

Table 7 Summary table for the outcomes for the six eHealth indicators as of August 1, 2019, for Argentina (RA), Australia (AUS), Austria (A), Finland (FIN), Germany (D), Hong Kong (HK), Israel (IL), Japan (J), Jordan (JOR), Kenya (EAK), South Korea (ROK), Sweden (S), Turkey (TR), and the United States (USA)

	eHealth indicator	Country
No.	Abbreviation	Name	A	AUS	D	EAK	FIN	HK	IL	J	JOR	RA	ROK	S	TR	USA
1	AH	Access of health care professionals to their patients' health record data.	→+	+	→−	−	→++	+	++	+	−	+	++	+	→+
2	AP	Access of patients to their health record data.	+	+	→−	−	→++	→−	−	→++	→+	+	→+
3	AC	Access of caregivers tothe patients' health record data.	+	+	→−	−	→++	→−	−	→++	→+	+	→+
4	EH	Enabling health care professionals to add data to their patients' health record(s).	→+	++	→+	+	→++	→+	++	+	→++	+	→++
5	EP	Enabling patients to add data to their health record(s).	→−	+	→−	−	→+	→−	−	+	→−	→+	++	→+
6	EC	Enabling caregivers to add data to the patients' health record(s).	→−	+	→−	−	→+	→−	−	+	→−	+	→+

Abbreviations: AC, access of caregivers to the patients' health record data; AH, access of healthcare professionals to their patients' data; AP, access of patients to their health record data; EC, enabling caregivers to add data to the patients' health record(s); EH, adding data by health care professional(s); EP, enabling patients to add data to their health record(s).

Note: For those countries, who participated also in the 2017 survey: → same indicator as 2017, indicator changed from − to + or from + to ++, indicator changed from ++ to + or from + to −.

Discussion

Summary of Findings

The 14 participating countries demonstrated diverse cross-institutional availability of patient-related information.

Finland, Hong Kong, Japan, and Sweden provide health care professionals from various inpatient and outpatient organizations with best access to patients' cross-institutional health record data ([Table 3]). Overall, 11 countries offer hospital staff fully or partly access to cross-institutional data, and 10 countries offer this for medical offices. In only six countries, nursing homes and outpatient nursing organizations have access to cross-institutional data. Three countries do not allow any access to any professional group.

Finland and South Korea allow patients and their caregivers the best access to their health record data ([Table 4]), five countries offer at least partial access. In all these countries, patients receive the largest amount of health record data from hospitals and from medical offices, as access to these data is fully or partly offered in seven countries. Access to data from nursing homes is only available in two countries. Three countries allow access to data from outpatient nursing organizations. Six countries do not allow patients and theiry caregivers any access to their data.

In all countries, health care professionals are partly or fully able to add patient data to a patient health record ([Table 5]). This reflects the fact that nearly all health care institutions have some sort of organizational EHR system where patient data is documented. Full support for electronic documentation is especially available in hospitals (12 countries) and medical offices (11 countries), to less extent also in nursing homes (5 countries) and outpatient nursing organizations (4 countries). Pharmacies typically only add medication data to the health record.

Only six countries allow patients and their caregivers to add data to the patient EHR, with Turkey supporting this best ([Table 6]). Eight countries do not allow patients to add any data.

Finland showed the highest fulfillment of all six analysed eHealth indicators, followed by South Korea, Japan, and Sweden ([Table 7]). No countries showed full support for all eHealth indicators, indicating opportunities for further improvement in all countries.

The WHO considers eHealth as an integral part of delivering improvements in health and sees a “substantial increase in the number and range of solutions.”[13] Our survey shows that in most of the 14 observed countries, the vision of universal access to major patient data is not yet fully reached. Nevertheless, according to the WHO, there has been a steady growth in the adoption of national EHR systems in the last years.[13] We found three countries (Argentina, Germany, and Kenya; [Table 3]) that did not allow health care professionals interorganizational access to relevant patient data from any of the selected health care institutions. We found six countries (Argentina, Germany, Hong Kong, Japan, Jordan, and Kenya; [Table 4]) that did not allow patients and their caregivers to access relevant health record data from any of the selected health care institutions.

Advancement in eHealth depends on contextual factors such as health care organization and health politics. Often cited barriers to national EHR systems are lack of funding, technical infrastructures, human resources capacity, effectiveness, and legal frameworks.[13] These barriers may hamper fast improvements in eHealth. However, countries like Finland show that strategic planning and long-term investments may result in effective eHealth. eHealth must show a positive impact on costeffectiveness[13] and on the quality of care[14] to increase the likelihood of being supported and implemented. Building such an evidencebase on the impact of eHealth seems crucial to further foster advancement in eHealth.[15]

Limitations

To increase the objectivity of the data, survey responses were based on consensus from two experts for all (but one) country. We tried to select eHealth experts for each country, but other experts may have come to other conclusions. All responses were reviewed and any implausible responses were rechecked with the experts.

Using eHealth expert surveys provided subjective results of the different indicators. Future research should be directed toward more direct measures that can provide a more objective picture by measuring completeness of information in electronic records. Other areas of future work will be to include allied health care providers, as well as professions, who support the health sector such as social workers.

Different from other surveys,[16] we did not provide quantitative ranges for “−,” “+,” and “++.” Instead, we aimed to make responding easier and accommodate for different interpretations and goals. However, not providing ranges might have led to different interpretation especially of “−” and “+.” To avoid this, we asked the experts to choose “−” if less than 20% of institutions or stakeholders in their country would have a positive indicator.

While attempting to carefully define and explain the core elements such as “EHR,” we found that some experts had difficulties to understand the scope of some questions. Based on a different understanding of the term EHR, some misunderstanding arose. We tried to accommodate this during verification of the responses through personal contact.

As a general limitation of every survey, the selection of indicators implies a certain perspective (and potential bias), with emphasis on some aspects and lack of coverage of other potentially important aspects. For example, we did not address questions such as the usefulness of EHR data for research.

Changes for Countries that Participated in the Survey of 2017

The eHealth indicator AH saw the biggest improvement between surveys with two (South Korea and Sweden) out of seven countries able to raise their overall score compared to the 2017 survey ([Table 7]). Considering that AH already showed the second highest overall scores in 2017 (“++” for two countries, “+” for three countries, and two “−”), this additional improvement seems remarkable and reflects better interoperability of electronic records. Hospital-based health care professionals made the most progress in accessing their patients' data ([Table 3]).

For eHealth indicators AP and AC, only Austria improved its overall score (from “−” to “+”). Most progress could be made in accessing data of hospitals (now “++” in Austria and Sweden, see [Table 4]).

For the other three indicators, EH, EP, and EC, none of the seven countries was able to make any visible improvements. As EH was the top scoring indicator already in 2017 (“++” for four countries and “+” for three countries), this seems less problematic than the stagnation for EP and EC. The latter indicators had low outcomes in 2017 (four “−” and three “+”), suggesting that substantial room for improvement was not exploited.

Comparisons to other eHealth Benchmarks

The OECD has developed a guide for measuring eHealth indicators. It is currently only available in a draft version.[1] The guide includes surveys to be answered by caregivers, as well as chief information officers of health institutions. The surveys address the availability of different types of electronic health information (e.g., problem lists, medication lists, and allergies) for caregivers, as well as patients, and distinguish between data-generated inside or outside the own organization. A smaller subset of seven indicators (e.g., management of electronic patient information by primary care providers, exchange of radiology images, and patient access to test results online) was used in a survey performed within 38 countries.[16] Compared to this OECD eHealth indicator survey, our survey is broader in regard to the included health care institutions by not only covering primary care providers and hospitals but also nursing homes, as these represent important contributors to patient-centered care. Comparable to the OECD survey, we also focused on problem lists, diagnoses, and allergies that we find essential for patient-centered care. The OECD survey, however, adds additional information types, such as demographics and vital signs, which makes the survey somewhat longer.

Six countries were covered both by the OECD survey[16] and our survey, and thus allowed us to compare the results. For the indicator EH, “enabling to add data,” with a focus on primary-care physicians (line 4 in [Table 5]), five of these six countries (Austria, Germany, Finland, Israel, and Sweden) scored “++” in our survey and “75 to 100%” in the OECD survey. Turkey scored “+” in our survey (with the comment that some private offices do not use EHRs) and “ > 75%” in the OECD survey. The survey results thus are congruent.

Regarding the indicator AP/AC, “access to EHR major data by patients or caregivers,” with a focus on primary care physicians (line 2 in [Table 4]), both surveys reported similarly (“++,” respectively, “maturity” for Finland and Israel and “−,” respectively, “ < 50%” for Germany). For Turkey, our survey found “+” and the OECD survey “ < 50%” that can be seen as corresponding. For Sweden, we found “++” and OECD <50%.” The reason may be the different survey dates (2016 vs. 2019) as in our surveys Sweden scored “+” in 2017 and “++” in 2019.

In 2013, the Nordic eHealth Research Network published a specification of eHealth indicators to be used in the Nordic region.[6] It is based on the OECD work and refines it in several aspects. The specification focused on three indicators for its initial test phase. These indicators address the availability of a list of currently prescribed medications to every caregiver that the patient sees, the availability and use of electronic transmission of medication prescriptions, and the availability and use of secure messaging between caregivers and patients for appointment booking. The initial set of indicators was extended to 49 indicators in 2015,[7] which address the availability and access of various types of health information (e.g., clinical notes, lab results, and medication prescriptions) by caregivers and patients. Comparable to the Nordic eHealth Research Network, our survey focusses on the availability of certain clinical information (e.g., clinical notes, lab results, and medication prescriptions). The Nordic indicator results are, however, limited to Nordic countries, while we covered countries from all over the world.

Two countries provided data in the Nordic survey[7] and were also covered in our survey, Finland and Sweden. In our survey, Finland scored “ + +” for the indicator AH, “access of health care professionals to patient data,” with regard to access from hospitals and from medical offices (lines 1–4 in [Table 3]). In the Nordic survey, Finland scored high for access to clinical notes (95%), patient summaries (85%), and prescriptions (100%) from specialized care institutions and reasonably high (78, 63, and 100%) for access to the same type of information from primary care institutions. Both survey results thus were congruent. In our survey, Sweden scored “++” for the indicator AH with regard to access from hospitals and from medical offices. In the Nordic survey, Sweden scored 100% for access to clinical notes, patient summaries, and prescriptions both from primary and specialized care institutions. These results thus were also congruent.

The European Commission publishes a report on the dissemination of eHealth functionality among general practitioners (GPs) within the EU member states in a 5-year interval.[8] This report provides an overview of the usage frequency of different types of content within the GP's EHR systems, such as laboratory test results, radiology reports, and drug-related information. In contrast to our survey, it does not focus on cross-institutional access to this data, thus we must expect that answers primarily refer to access within the GPs' local EHR systems. The survey further shows the extent of the exchange of these data with other care providers. Overall, our survey is broader, as we also cover hospitals, pharmacies, and nursing homes.

Four countries (Austria, Germany, Finland, and Sweden) were covered both by the GP survey[8] and our survey allowing comparison of the results. Regarding indicator AH, with focus on medical offices (line 4 in [Table 3]), the surveys were congruent with Austria/Finland/Sweden scoring “+”/“++”/“++” in our survey and 1.6/2.9/2.9 (out of a maximum of 4 points) for indicator “health information exchange of clinical data” in the GP survey. Germany scored 1.6 points in the GP survey and “−” in our survey. The reasons for the divergent conclusions between Germany (1.6/−) and Austria (1.6/+) in both surveys could be in the recent changes in Austria. At the moment, Austria is introducing the national EHR system ELGA (Elektronische Gesundheitsakte) leading to improvement of the indicator AH to “+” in our survey. ELGA may not have been covered by the GP survey from 2018.

The WHO provides statistics for several eHealth indicators among the European members of the WHO within its European Health Information Gateway.[9] The data originated from the WHO's global eHealth survey from 2015.[13] A separate report is available for the European region.[17] The indicators primarily focus on issues such as funding, strategies, legislation, and the existence of certain eHealth programs. These WHO indicators thus do not cover the availability of certain types of health information for certain types of caregivers or the patient as our indicators do.

In the United States, the EHR meaningful use incentive program requires participating care providers to demonstrate the satisfaction of several criteria as a prerequisite for funding.[10] Criteria focus on health information exchange and on providing patients electronic access to their health information. These criteria are more focused on available functionality and less on available Information.

While Esdar et al used an existing diffusion model to calculate the diffusion dynamics of EHRs in German and U.S. hospital care between 2007 and 2017,[18] access to different types of health information by different types of care providers was not addressed by them.

Naumann et al compared eHealth adoption in Austrian, German, and Swiss hospitals in regard to diffusion of EHRs, health information exchange, and electronic communication with patients.[19] Similar to the present work, they analyzed the availability of medical discharge letters for external care providers.

Tsai and Koch developed a framework for eHealth evaluation and monitoring in Sweden.[20] It focuses on 19 specific eHealth outcomes and provides indicators for assessing the outcomes.

Our survey provides an international perspective on the availability of basic clinical data for important stakeholders involved in patient-centered care, including patients and their caregivers. We focus on available and reproducible information and not on functionalities or technical infrastructure. In accordance with the Clinical Adoption Metamodel,[21] we see availability and usage of information systems as the precondition of information availability which is an important precondition for information logistics (“right information at the right place to the right person”[22]). Information availability thus is also the precondition to modification of clinical behaviors and clinical decisions that may, in the end, affect patient outcomes.

Conclusion

Advancement in eHealth depends on contextual factors such as health care organization, national health politics, and health financing. Improvements in eHealth are thus often slow. Our survey shows, however, that some countries were able to improve at least some indicators between 2017 and 2019. Further improvements can be expected in the future. Countries such as Finland show that strategic planning and long-term investments allowed fostering eHealth successfully in the end.

We must remember that the indicators we chose do not directly reflect the quality of health care. Quality of care may be quite high, even when eHealth indicators may (still) be low. Nevertheless, we expect that easy access to relevant patient data is an important part of high-quality patient care.

• References

• 1 Organisation for Economic Co-operation and Development (OECD). Draft OECD Guide to Measuring ICTs in the Health Sector. Available at: https://www.oecd.org/health/health-systems/Draft-oecd-guide-to-measuring-icts-in-the-health-sector.pdf . Accessed July 15, 2020
• 2 Kruse CS, Beane A. Health information technology continues to show positive effect on medical outcomes: systematic review. J Med Internet Res 2018; 20 (02) e41
• 3 Hersh WR, Totten AM, Eden KB. et al. Outcomes from health information exchange: systematic review and future research needs. JMIR Med Inform 2015; 3 (04) e39
• 4 Kuhn KA, Giuse DA. From hospital information systems to health information systems. Problems, challenges, perspectives. Methods Inf Med 2001; 40 (04) 275-287IS
• 5 Eysenbach G. What is e-health?. J Med Internet Res 2001; 3 (02) E20
• 6 Hyppönen H, Faxvaag A, Gilstad H. et al. Nordic eHealth indicators: organisation of research, first results and plan for the future. Stud Health Technol Inform 2013; 192: 273-277
• 7 Nordic Council of Ministers. Nordic EHealth Benchmarking. Available at: https://norden.diva-portal.org/smash/get/diva2:1093162/FULLTEXT01.pdf . Accessed July 25, 2020
• 8 European Commission. Benchmarking Deployment of EHealth among General Practitioners. Available at: http://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=4897 . Accessed July 15, 2020
• 9 World Health Organization. E-health: Indicators & visualizations. Available at: https://gateway.euro.who.int/en/themes/e-health/indicators-visualizations/ . Accessed July 15, 2020
• 10 Centers for Disease Control and Prevention. Public Health and Promoting Interoperability Programs. Available at: https://www.cdc.gov/ehrmeaningfuluse/introduction.html . Accessed July 15, 2020
• 11 Haux R, Ammenwerth E, Koch S, et al. A Brief Survey on Six Basic and Reduced eHealth Indicators in Seven Countries in 2017. Appl Clin Inform. 2018;9(3):704–713. doi:10.1055/s-0038-1669458
• 12 ISO. ISO/TR 20514:2005. Health informatics—Electronic health record—Definition, scope and context. Available at: https://www.iso.org/standard/39525.html . Accessed July 15, 2020
• 13 World Health Organization. Global Diffusion of EHealth: Making Universal Health Coverage Achievable. Available at: https://apps.who.int/iris/bitstream/handle/10665/252529/9789241511780-eng.pdf;jsessionid=B7E7B4CF6BA92C6F4CC674C962BC395B?sequence=1 . Accessed July 15, 2020
• 14 Granja C, Janssen W, Johansen MA. Factors determining the success and failure of ehealth interventions: systematic review of the literature. J Med Internet Res 2018; 20 (05) e10235
• 15 Rigby M, Ammenwerth E, Talmon J. Forward outlook: the need for evidence and for action in health informatics. Stud Health Technol Inform 2016; 222: 355-363
• 16 Zelmer J, Ronchi E, Hyppönen H. et al. International health IT benchmarking: learning from cross-country comparisons. J Am Med Inform Assoc 2017; 24 (02) 371-379
• 17 World Health Organization. From Innovation to Implementation. EHealth in the WHO European Region(2016). Available at: https://www.euro.who.int/__data/assets/pdf_file/0012/302331/From-Innovation-to-Implementation-eHealth-Report-EU.pdf . Accessed July 15, 2020
• 18 Esdar M, Hüsers J, Weiß JP, Rauch J, Hübner U. Diffusion dynamics of electronic health records: a longitudinal observational study comparing data from hospitals in Germany and the United States. Int J Med Inform 2019; 131: 103952
• 19 Naumann L, Esdar M, Ammenwerth E, Baumberger D, Hübner U. Same goals, yet different outcomes: analysing the current state of ehealth adoption and policies in Austria, Germany, and Switzerland using a mixed methods approach. Stud Health Technol Inform 2019; 264: 1012-1016
• 20 Tsai CH, Koch S. Towards a framework for national ehealth evaluation and monitoring: a combined top-down and bottom-up approach using sweden as example. Stud Health Technol Inform 2019; 264: 954-958
• 21 Price M, Lau F. The clinical adoption meta-model: a temporal meta-model describing the clinical adoption of health information systems. BMC Med Inform Decis Mak 2014; 14 (01) 43
• 22 Winter A, Haux R, Ammenwerth E, Brigl B, Hellrung N, Jahn F. Health Information Systems: Architectures and Strategies. 2nd ed. New York, NY: Springer-Verlag; 2011