Using Existing Clinical Information Models for Dutch Quality Registries to Reuse Data and Follow COUMT Paradigm

Abstract

Background Reuse of health care data for various purposes, such as the care process, for quality measurement, research, and finance, will become increasingly important in the future; therefore, “Collect Once Use Many Times” (COUMT). Clinical information models (CIMs) can be used for content standardization. Data collection for national quality registries (NQRs) often requires manual data entry or batch processing. Preferably, NQRs collect required data by extracting data recorded during the health care process and stored in the electronic health record.

Objectives The first objective of this study was to analyze the level of coverage of data elements in NQRs with developed Dutch CIMs (DCIMs). The second objective was to analyze the most predominant DCIMs, both in terms of the coverage of data elements as well as in their prevalence across existing NQRs.

Methods For the first objective, a mapping method was used which consisted of six steps, ranging from a description of the clinical pathway to a detailed mapping of data elements. For the second objective, the total number of data elements that matched with a specific DCIM was counted and divided by the total number of evaluated data elements.

Results An average of 83.0% (standard deviation: 11.8%) of data elements in studied NQRs could be mapped to existing DCIMs . In total, 5 out of 100 DCIMs were needed to map 48.6% of the data elements.

Conclusion This study substantiates the potential of using existing DCIMs for data collection in Dutch NQRs and gives direction to further implementation of DCIMs. The developed method is applicable to other domains. For NQRs, implementation should start with the five DCIMs that are most prevalently used in the NQRs. Furthermore, a national agreement on the leading principle of COUMT for the use and implementation for DCIMs and (inter)national code lists is needed.

Keywords

electronic health record - secondary use - data reuse - clinical information models - national quality registries - administrative burden - interoperability - FAIR

Background and Significance

National Quality Registries

Over the last decades, measuring quality of care has become a common practice in health care. For these measurements, various types of data are required about patients and the diagnoses and treatments they have been given. Results of national quality registries (NQRs) are increasingly used for improving quality of care, for informing patients in the shared-decision-making process, and for performance comparisons among health care institutions.[1] At the start, around 2000 to 2010, Dutch NQRs were based on manual data entry only. Currently, most NQRs have the option of batch processing or more manual extraction. The batch processing option requires specific queries, customized for each hospital, to extract data directly from specific fields in the electronic health records (EHRs).[2]

Data Reuse of Electronic Health Records

EHRs play a key role in providing access to data that can improve individual care as well as support quality improvements, clinical research, and the achievement of public health objectives. Preferably, NQRs extract machine-readable data recorded during the care process and stored in the EHR, without any additional manual actions. This requires structured and standardized registration of the characteristics of a patient, the diagnostic work-up and various aspects of the disease, treatment, and outcomes in the EHR. Some additional benefits of direct extraction of NQR data from EHRs would be the avoidance of misinterpretation of source data by a registrar—this could reduce the need for extensive data verification and contribute to a reduction of the administrative burden on health care professionals. Nevertheless, data verification by short-cycle feedback will still be required.

However, this is not an easy transition as there are different EHR systems in use and there is a lack of uniform registration in EHRs.[2] Ideally, data needed for reuse are entered once in an EHR, stored in a structured way, and are subsequently able to be extracted for multiple purposes (care process, research, quality registries, and so on). Internationally, this type of data reuse is referred to as the COUMT paradigm (“Collect Once Use Many Times”).[3]

Dutch Clinical Information Models

In order to make the transition from manual data entry in an NQR to extracting data directly from EHRs, a novel approach to data collection, storage, and retrieval needs to be developed and applied. Clinical information models (CIMs) are models that structure data in a way to reuse them.[4] [5] A CIM describes a (clinical) concept in a structured and detailed method.[5] Preferably, CIMs are structured in such a way that the COUMT paradigm is followed and international terminologies like SNOMED CT are used in order to make the data machine readable and suitable for international use. Different types of CIMs exist, such as for example HL7 templates and open EHR archetypes,[6] and many synonyms for CIMs are being used: detailed clinical models, clinical building blocks, clinical content models, national information models, and so on.[7] In 2012 a national system of 100 Dutch CIMs (DCIMs and in Dutch “Zorginformatiebouwstenen”) was designed in order to support reuse of the clinical data registered in the daily care process for multiple purposes (see [Supplementary Appendix 1], available in the online version).[8] The Basic Set (Basisgegevensset Zorg [BgZ]) CIMs are based on the International Patient Summary[9] and consist of 28 DCIMs like “Problem,” “Patient,” and “Procedure.”[10] DCIM “Problem” for example covers complaints, symptoms, diagnosis, starting date, end date, etc.[11] Dutch hospitals have implemented the Basic Set, but EHR vendors have implemented them in different ways which complicates data exchange between EHR systems. NQRs are not yet based on DCIMs and it is currently unknown how many of the data elements needed for NQRs are covered with DCIMs.

Objective

In the Netherlands, national goals are set to follow COUMT and use EHRs as a source of data information.[12] In 2018, the Dutch Ministry of Health and the representative organizations of patients, clinicians, nurses, hospitals, and health insurers agreed on a program aiming to improve data exchange through increased structuring and standardization of documentation and subsequent reuse of data.[12] An additional agreement was made for the NQRs stipulating that they also should be standardized and the required data should be directly retrievable from the EHRs. The Dutch Association of Medical Specialists started a study (“Verduurzamen Kwaliteitsregistraties”) aiming to fulfill this agreement. The first objective of this study was to analyze the level of coverage of data elements in NQRs with existing DCIMs in order to evaluate whether it is realistic to use EHR data based on DCIMs for NQRs. Eventually this could enable automated quality measurement with limited administrative burden and near real-time feedback from NQR to hospitals for adjustment and improvement of care. The second objective of this study was to analyze the most predominant DCIMs both in terms of data element coverage and in their prevalence across existing NQRs.

Method

Introduction

This study was conducted to determine whether the content of existing DCIMs was sufficient to cover the necessary data input for the NQRs. Most NQRs were disease-specific and did not include patient-reported outcome measures nor patient-reported experience measures. All Dutch NQRs, currently around 60 in total, were invited to participate in this study. Thirty-six NQRs applied and 31 NQRs had health professionals available to work with the study team. The developed mapping method was applied to these 31 NQRs by executing an in-depth analysis of each data element of the NQR and linking it to existing DCIMs. Biases in mapping were prevented by working with two persons from the study team on one NQR.

Mapping Method for the First Study Objective

In this study, a method to map the data elements per NQR to the DCIMs was developed. Our method was inspired by the approach that originated from the Dutch Program Registration at the Source (“Registratie aan de Bron”).[13] This existing approach consisted of linking each data element of an NQR to the corresponding DCIM. This approach was enriched through alignment with clinical pathways to be able to retrieve the exact data needed for an NQR. Additionally, extra levels of detail were added as we linked each data element to the corresponding element in the code list (for example, code list Tobacco use and exposure) used in NQR and the DCIM. This step was needed since correspondence between the code lists used for the NQR and those used for the DCIM is a prerequisite for eventual implementation.

Overall Mapping Method

The overall mapping method which was developed is summarized in [Fig. 1].

Fig. 1 Overall mapping method.

Each single step of the approach is further explained in the next paragraphs. Each mapping was executed by at least two members of a small overall study team consisting of nurses and health scientists with information technology expertise. For each NQR mapping, two to four clinicians with expertise in the specific disease were added to the study team.

Mapping Method Step-by-Step

In the first step, the high-level clinical pathway for a disease was described. A clinical pathway, also known as a (integrated) care pathway, is one of the main tools for standardization of the care process. Clinical pathways are used to reduce variation, improve quality of care, and maximize outcomes for specific groups of patients.[14] In this study, a high-level description of the clinical pathway was executed for each of the 31 NQRs.

The study team made a first proposal based on documentation from the participating clinicians which was then validated by the clinicians. Although clinical pathways may differ per hospital on a more detailed level, the high-level clinical pathway is nationally agreed on in guidelines and thus can be considered common practice. The Hospital Reference Architecture (Ziekenhuis Referentie Architectuur [ZiRA]) process model was used to describe the clinical pathway and an example of the clinical pathway for a patient with (suspected) prostate cancer is depicted in [Fig. 2].[15]

Fig. 2 High-level clinical pathway prostate cancer.

In the second step, the clinical pathway workflow was linked to the data workflow per patient to gain insight into three main issues: which data were required during each phase of the workflow, which data should be registered in each part of the clinical pathway, and who should register the data (e.g., urologist, radiologist, etc.). The main reason to combine the clinical pathway with the NQR is that both are necessary for the selection of the right data element from the EHR. For example, for the NQR of morbid obesity, body weight measurements before and after the operation are needed. Therefore, the system must facilitate the recording of body weight measurements in specific months before and after the procedure, so they are registered and ready for reuse. After completion of step 2, it is clear which data elements are registered in the EHR, during which part of the care process they are registered, and what is the source of the data (e.g., digital referral and outpatient clinic). See [Supplementary Appendix 2] (available in the online version) for the example of prostate cancer.

For the third step, every data element of the NQR was critically reviewed from the perspective of efficient data use. For example, a body mass index does not need to be registered when body weight and length are already registered. Also, the clinicians looked critically at their dataset again and data elements that were no longer relevant for the purpose of health care improvement were dropped.

In the fourth step, all data elements from the NQR were mapped onto the DCIMs. For example, all data elements concerning patient characteristics, such as their date of birth, were mapped onto the DCIM “Patient.”

In the fifth step, the results of steps 2 and 4 were combined. Every single data element of an NQR that was matched to a DCIM in step 4 was then, in step 5, plotted to the corresponding part of the clinical pathway from step 2. In this way, it was clear in which part of the clinical pathway the specific data element is registered.

The sixth step included the most detailed mapping. Each data element of the NQR was mapped with the corresponding values of the corresponding code list of the data element of the DCIM. For example, the data element “Smoking” which was already linked to DCIM “Tobacco use” in step 4, was linked in this step to the corresponding entity in the code list based on the international terminology of SNOMED CT: 365980008 Tobacco use and exposure.

Application of the Mapping Method

For all 31 NQRs we used exactly the same method to map the NQR data element with the corresponding data element(s) from the appropriate DCIM. To evaluate the mapping method, we analyzed to what extent every single data element could be mapped, after detailed analysis, onto existing DCIMs. Each data element was assigned to one of the following categories ([Table 1]).

Table 1 Definitions of mapping categories

Categories	Definition
Basic Set DCIM mapping possible	According to definition Basic Set DCIMs[9]
Other DCIM mapping possible	Other than Basic Set DCIM
Future DCIM mapping possible	DCIM which will be released in near future
No mapping possible with DCIM	No match with current or near-future DCIM
Other data model possible	For pathology data, there is a separate data model in the Netherlands
Smart registry possible	Data element can be retrieved by using or combining other data elements which are already in the NQR dataset
Data element dropped	Data element no longer clinically relevant

Abbreviation: DCIM, Dutch Clinical Information Model; NQR, national quality registry.

Methodology for the Second Study Objective: Most Predominant DCIMs

To determine which DCIMs were the most used in mapping, the total number of data elements that matched with a specific DCIM was counted and divided by the total number of evaluated data elements. For analysis of the prevalence of the DCIMs, we counted the number of NQRs in which the specific DCIM was used and divided that by the total number of NQRs analyzed.

Results

Participation of National Quality Registries

The 31 participating NQRs represent different diseases and/or procedures and are categorized as follows: 8 oncology, 5 neurology and neurosurgery, 6 surgery, 3 gynecology, 4 internal medicine, and 5 miscellaneous NQRs ([Table 2]).

Table 2 Results of application of the mapping method for 31 national quality registries (datasets 2019)

Category	NQR	Name of NQR	Reference	Total data elements	Basic Set DCIM	Other DCIM	Future DCIM	Smart Registry	Information model pathology	Dropped	No DCIM
Oncology
1	Breast cancer	NBCA	15	121	46	23	6	14	28		4
2	Colorectal cancer	DCRA	16	236	117	54	56	9
3	Stomach and esophagus cancer	DUCA	17	303	172	39	16	2			74
4	Liver and gallbladder cancer	DHBA		247	223	19					5
5	Lung cancer	DLCA	18–20	661	337	121		6	68		129
6	Melanoma	DMTR	21	544	220	186		119			19
7	Pancreatic cancer	DPCA	22	265	101	72	14	5	20	10	43
8	Prostate cancer	tbd		146	80	51	11				4
Other surgery
9	Aneurysm	DSAA	23	131	106	17		3			5
10	Carotid interventions	DACI	24	78	58	18					2
11	Children surgery	EPSA		468	380	37		20			31
12	Clubfoot	LROI-children-clubfoot		142	135			7
13	Morbid obesity	DATO	25	162	124	22		6			10
14	Schisis	tbd		230	200	3		1			26
Neurology and neurosurgery
15	Glioblastoma	QRNS-glioblastoma		31	27			4
16	Hypophysis	QRNS-hypophysis		57	39	9		4			5
17	Multiple sclerosis	MS		60	29	15					16
18	Subarachnoid hemorrhage	SAB		44	35	7		2
19	Stroke	DASA	26	39	23	10		3			3
Internal diseases
20	Chronic kidney disease	Renine		48	39	4		2			3
21	Colorectal endoscopy	DRCE	27	106	60	10		3		22	11
22	Diabetes	DPARD		131	94	5		7		9	16
23	Rheumatoid arthritis	DQRA		40	22	2	8				8
Gynecology/obstetrics
24	Endoscopy	NVOG-endoscopy		169	151	15					3
25	IVF and IUI	NVOG-IVF IUI		44	31	2				5	6
26	Mesh	NVOG Mesh		194	180	2					12
Miscellaneous
27	Age-related macula degeneration	LMD		49	47						2
28	Breast implant	DBIR	28	111	105	1					5
29	Hip fracture	DHFA	29	85	51	19		9			6
30	Intensive care	NICE		121	91	4		1			25
31	Percutaneous coronary interventions	NHR-PCI		43	33	8					2
		Total		5,106	3,356	775	111	227	116	46	475

Abbreviations: NBCA, NABON Breast Cancer Audit[16]; DCRA, Dutch Colorectal Audit[17]; DUCA, Dutch Upper GI Audit[18]; DHBA, Dutch Hepato Biliary Audit; DLCA, Dutch Lung Cancer Audit[19] [20] [21]; DMTR, Dutch Melanoma Treatment Registry[22]; DPCA, Dutch Pancreatic Cancer Audit[23]; DSAA, Dutch Surgical Aneurysm Audit[24]; DACI, Dutch Audit for Carotid Interventions[25]; EPSA, European Pediatric Surgical Audit; LROI, Landelijke Registratie Orthopedische interventie/Dutch Registry for Orthopedic Interventions; DATO, Dutch Audit for Treatment of Obesity[26]; QNRS, Quality Registry Neurosurgery; DASA, Dutch Acute Stroke Audit[27]; DRCE, Dutch Registration of Complications in Endoscopy[28]; DPARD, Dutch Pediatric and Adult Registration of Diabetes; DQRA, Dutch Quality registry Rheumatoid Arthritis; NVOG, Nederlandse Vereniging voor Obstetrie en Gynaecologie/Dutch Society for Obstetrics and Gynecology; LMD, leeftijdsgebonden maculadegeneratie/age-related macular degeneration; DBIR, Dutch Breast Implant Registry[29]; DHFA, Dutch Hip Fracture Audit[30]; NICE, Nationale Intensive Care Evaluatie/National Intensive Care Evaluation; NHR-PCI, Nederlandse Hart Registratie/Dutch Heart Registry ; Percutaneous Coronary Intervention.

Level of Coverage of Data Elements for 31 NQRs

Each of the 31 NQRs was analyzed using the mapping method. [Table 2] describes the main results. Using a detailed mapping, 80.9% (4,131 of 5,106) data elements could eventually be matched with an existing DCIM, 65.7% (3,356) to a Basic Set DCIM (and 15.2% (775) to another DCIM), 2.2% (111 data elements) could be linked to a future DCIM, 2.3% (116) to the pathology information model, 4.4% (227) could be retrieved using smart registry, 0.9% (46) was dropped, and 9.3% (475) could not be matched to a DCIM, for example, because they were related to financial information or structure indicators such as number of medical specialists. The average coverage with existing DCIMs was 83.0% with a standard deviation of 11.8%.

There was variation in the level of coverage of the different categories per NQR. However, no single NQR had less than 50% coverage by the Basic Healthcare Data Set DCIMs or the Other DCIMs. The relative results per NQR are listed in [Fig. 3].

Fig. 3 Overview of the relative results of 31 national quality registries per mapping category.

Overall Coverage and Prevalence of DCIMs Per NQR

The 31 NQRs consisted of in total 5,106 data elements, ranging from 31 (for glioblastoma NQR) to 661 data elements (for lung cancer NQR), with a median of 121 data elements per NQR. In total 1,006 data elements (19.7%) could be matched with DCIM Problem; 863 data elements (16.9%) with DCIM Procedure, 240 (4.7%) with DCIM Patient, 204 (4.0%) with DCIM Laboratory Test Result, and 168 (3.3%) with DCIM General Measurement. In total, 5 out of the 100 DCIMs were needed to map 48.6% of the data elements. [Fig. 4] illustrates this.

Fig. 4 Predominantly used Dutch Clinical Information Models (DCIMs) for mapping data elements from national quality registries (NQRs).

The analysis of the prevalence of DCIMs in NQRs demonstrated that 8 out of the 100 DCIMs occurred frequently, with each being mapped to over half of the 31 studied NQRs. The figure underneath depicts the prevalence per DCIM over the total of 31 NQRs ([Fig. 5]).

Fig. 5 Prevalence of Dutch Clinical Information Models (DCIMs) in national quality registries (NQRs).

Discussion

The reuse of health care data for various purposes will become increasingly important in the future. To enable the reuse of clinical data, structured and standardized registration and documentation and standardized exchange are conditional. DCIMs are agreements on characteristics of a care concept about content and are crucial for registration, documentation, and improve interoperability and reuse of health care data. In this study, we focused on DCIMs and demonstrated that the potential of mapping NQRs with existing DCIMs is substantial; on average 80.9% of data elements could be matched to an existing DCIM and 4.6% could be linked to a future DCIM or to the information model for pathology data. Overall coverage of DCIMs showed that 5 out of 100 DCIMs covered 48.6% of data elements needed for 31 NQRs.

Detailed mapping with DCIMs led to several other insights. There was a huge variety in set-up and data use between the NQRs. Only 1 out of 31 NQRs was partially based on international terminologies, such as SNOMED CT and LOINC.[31] [32]

Also, we found a lack of standardized implementation of national code lists by both NQRs and EHR systems. For example, each EHR system used a different code list for smoking.

The effort to map an NQR took about 200 hours for an average-sized NQR, 160 hours for the study team, and 40 hours for the team of health care professionals. Standardized implementation of (the Basic Set of) DCIMs in hospitals could potentially lead to a significant reduction of the administrative burdens for NQRs, as 80.9% of data elements of the 31 NQRs could be mapped on an existing DCIM. With only four DCIMs with care content (Problem, Procedure, General Measurement, and Laboratory Test Result), 43.9% of the data elements of the studied NQRs are covered. This study also demonstrates the advantage of linking the data elements to the care process.

Comparisons with Other Studies

Reuse, or secondary use, of data concerns the use of routinely collected clinical data for a different purpose other than the one for which it was originally collected. Often these data are reused for research or quality-of-care measures. Literature about reuse of data in general is voluminous[33]; however, to our knowledge this is the first study which analyzes the potential of using existing DCIMs for data in NQRs. In a recent Swedish study, a patient-centered information model with data annotation was developed and successfully implemented for one care pathway.[34] Their study emphasized that an information model should follow and support clinical pathways in order to generate data for myriad purposes such as clinical research and NQRs. When comparing the data elements of the clinical pathway for chronic obstructive pulmonary disease (COPD) with the data elements required for the COPD NQR, they found that many data elements were similar. The study's authors expect the burden for registering data for NQRs to be significantly reduced once a full implementation is made. They concluded that unless the information model is flexible in supporting use of clinical pathways in an accessible way, with methods where the professionals are part of the construction, system-level inertia from professional roles, administration systems, payment systems, and poor information technology will prevent health care development.

Reuse of data has been of interest in (pharmaco) epidemiology. Projects such as the Observational Medical Outcomes Partnership (OMOP) have demonstrated the value of these data compared to more traditional databases.[35] The Observational Health Data Sciences and Informatics (OHDSI) collaborative is a volunteer collaborative international network of researchers and is the successor of OMOP.[36] OMOP facilitates the transformation of data contained in different health care databases into a harmonized format (Common Data Model [CDM]), and uses common representations (terminologies, vocabularies, and coding schemes). Health data include insurance claims, EHR, and hospital billing data. The CDM makes large-scale analytics possible, allowing access to billions of deidentified health records for observational health research.[37] A fundamental tool developed from OMOP is the Standard Vocabulary, based on global standards such as SNOMED CT and LOINC, which enables interoperability between systems.[38] OMOP is an overall CIM, whereas DCIMs are more detailed CIMs of concepts which are also present in OMOP. For example, OMOP has a concept “observation” and the DCIM Blood Pressure is a detailed elaboration of the OMOP observation Blood Pressure.

Our results contribute to the European discussion on the use of different interoperability standards across Europe and supports the importance of standardized taxonomies such as SNOMED CT.[39] No comparable studies in other countries have been found, yet our approach could be used in analogous efforts in other countries exploring the use of DCIMs.

Strengths of this Study

A strength of this study is that a detailed analysis has been executed of 5,106 elements of 31 NQRs. In the Netherlands there are currently around 60 NQRs for different diseases, so this research covers about 50% of all Dutch NQRs. Another strength is that the mapping method is reproducible, as each mapping was executed by the same small overall study team. In weekly meetings all questions that came up during mapping were discussed with the overall study team, to make sure all decisions were made consistently throughout the whole study. For example, how to discern whether imaging has taken place before or after a procedure or how to make a distinction between first operation and revision surgery.

Limitations of this Study

Although about half of all NQRs participated, there was a slight overrepresentation of oncological NQRs. Furthermore, this study is limited to the development and application of a mapping methodology of hospital data and no implementation has taken place during this study. As hospitals only implemented the Basic Set DCIMs, we can start using a part of the results of this study in day-to-day practice.

Lessons Learned from Current Practice in the Netherlands

For data reuse in health care, the words “Registration at the Source” have a widespread use in the Netherlands. However, the focus in most projects is still on single-use perspectives instead of multiple-use purposes. Unfortunately, the COUMT paradigm is not yet seen as fundamental in many of the current nationwide projects. Also, making the data FAIR (i.e., meeting the principles of Findability, Accessibility, Interoperability, and Reusability)[40] is unfortunately not yet a goal for EHR systems in the Netherlands. The Netherlands has an oligopoly of hospital EHR vendors and thus multiple EHR systems; this fragmentation delays implementation of (inter)national standards and DCIMs.[41]

To make reuse of data possible, some adjustments are needed. EHR systems should be upgraded from digital notebooks or financial registration systems to systems that support the clinical pathway and workflow for each patient.[42] As mentioned above, tracking and tracing of a data element that is registered in the EHR is another prerequisite. Furthermore, to increase semantic interoperability, the use of standard code lists and international terminologies such as SNOMED-CT and LOINC should be obligatory in order to achieve a common vocabulary.[43] Structured and standardized reporting and documentation is preferred when reuse of data is desirable. Research has shown that structured documentation can improve provider efficiency, decrease documentation time,[44] and increases the quality of notes in the EHR.[45] The adoption of structured reporting by health care professionals is related to usability and compliance to the clinical pathway and the workflow.

A national agreement on the leading principle of COUMT for the use and implementation for DCIMs and (inter)national code lists is needed. To confirm the feasibility and added value of COUMT for health care data, it is recommended to start with the implementation of at least the five most important DCIMs (Problem, Procedure, Patient, Laboratory Test Result, and General Measurement) for NQRs. This means that EHR systems and NQRs should be adapted accordingly.

Future Research

High-quality machine-readable data have the potential to increase safety and quality of care, allow near real-time feedback for NQRs, reduce the administrative burden, and eventually reduce costs. This study is the first step in applying DCIMs to NQRs. Efforts should be made to evaluate the coverage and use of DCIMs for other NQRs and also for different use cases, such as research purposes and in other health care segments such as primary care, mental care, etc. Another study to analyze the coverage for other NQRs, using the same mapping method, has already started. The results of our study raise questions for future studies about the benefits and pitfalls of implementation of DCIMs in different areas while taking the COUMT paradigm as an overarching goal. These questions include for example the effect of structured documentation systems on time and effort and also the possible short-cycle data feedback and verification possibilities with NQRs based on EHR data. Future research would also benefit from studying the most efficient adjustment of NQRs to a DCIM format and implementing the most impactful DCIMs in a controlled setting in EHR systems.

Conclusion

This study shows the potential of using existing DCIMs for data capture for NQRs, gives direction to further implementation of DCIMs in the Netherlands, and facilitates the set-up for new NQRs according to DCIMs. In addition, this method can be used for other domains, such as primary care or mental health care and other purposes such as research. The next step will be the validation of this work in practice, by applying DCIMs in EHRs and adapting NQRs to DCIMs following the COUMT paradigm. Given the current lack of reusability of data and poor interoperability across EHRs, a transition to COUMT is needed and only feasible with national orchestration.

Clinical Relevance Statement

The reuse of health care data for various purposes will become increasingly important in the future. Reuse of EHR data is possible when the COUMT paradigm is followed and CIMs are implemented. The potential of using existing DCIMs for 31 NQRs is high. Implementing DCIMs could potentially reduce the administrative burden substantially. In addition, reuse of data by implementing the DCIMs will also allow near real-time feedback and contribute to patient safety and quality of care. The described method can also be used for other domains, such as primary care or mental health care and other purposes such as research.

Multiple-Choice Questions

1. Which of the statements about reusing electronic health record (EHR) data is true?

   • i. Reusing EHR data for national quality registries is common practice in the Netherlands.

   • ii. The potential of reusing EHR data is high for national quality registries.

     • I is true, II is false.

     • I is true, II is true.

     • I is false, II is false.

     • I is false, II is true.

     Correct Answer: The correct answer is option d. The potential of reusing EHR data for national quality registries is high, but is not yet common practice in the Netherlands.

2. When mapping the data elements for a national quality registry to clinical information models do we have to take the clinical pathway into account?

   • No, just mapping the data elements is sufficient.

   • No, this is only necessary if patients have recurrent diseases.

   • Yes, this is needed to specify the exact needed data element.

   • Yes, this is needed because often different clinicians treat one patient.

   Correct Answer: The correct answer is option c. Including the clinical pathway in the mapping method is needed in order to specify the required data element exactly as described in the section “Mapping Method Step-by-Step” with the example of morbid obesity.

Conflict of Interest

None declared.

Acknowledgements

We thank Esther Snoek for her cooperation and contribution in the analysis of the 31 NQRs and Nicolette de Keizer, Ronald Cornet, and Ferdinand Roelfsema for critically reviewing the manuscript.

Authors' Contributions

M.S., A.T., and E.v.d.V. developed the study design and the mapping method. M.S. executed the analysis for this manuscript. M.v.H., M.W., and S.R. supervised the analyses for this manuscript. M.S., A.T., M.v.H., S.R., and M.W. contributed to the draft of the manuscript. All authors contributed to the manuscript review and editing.

Protection of Human and Animal Subjects

No human subjects were involved.

• References

• 1 Emilsson L, Lindahl B, Köster M, Lambe M, Ludvigsson JF. Review of 103 Swedish healthcare quality registries. J Intern Med 2015; 277 (01) 94-136
• 2 Hoeijmakers F, Beck N, Wouters MWJM, Prins HA, Steup WH. National quality registries: how to improve the quality of data?. J Thorac Dis 2018; 10 (Suppl. 29) S3490-S3499
• 3 Joukes E, Cornet R, de Keizer N, de Bruijne M. Collect once, use many times: end-users don't practice what they preach. Stud Health Technol Inform 2016; 228: 252-256
• 4 Goossen WTF. Detailed clinical models: representing knowledge, data and semantics in healthcare information technology. Healthc Inform Res 2014; 20 (03) 163-172
• 5 Oniki TA, Coyle JF, Parker CG, Huff SM. Lessons learned in detailed clinical modeling at Intermountain Healthcare. J Am Med Inform Assoc 2014; 21 (06) 1076-1081
• 6 Moreno-Conde A, Moner D, Cruz WD. et al. Clinical information modeling processes for semantic interoperability of electronic health records: systematic review and inductive analysis. J Am Med Inform Assoc 2015; 22 (04) 925-934
• 7 Goossen W, Goossen-Baremans A, van der Zel M. Detailed clinical models: a review. Healthc Inform Res 2010; 16 (04) 201-214
• 8 Zorginformatiebouwstenen. Accessed November 16, 2021 at: https://www.registratieaandebron.nl/zorginformatiebouwstenen
• 9 World Health Organization. Guidelines on the European patient summary dataset. Eurohealth 2014; 20 (01) 25-28 Accessed December 9, 2022 at: https://apps.who.int/iris/handle/10665/332849
• 10 Patient Summary BGZ. Accessed April 5, 2022 at: https://www.nictiz.nl/patient-summary-bgz/
• 11 Zib probleem. Accessed August 18, 2022 at: https://zibs.nl/wiki/Probleem-v4.4(2020NL)
• 12 Bestuurlijk akkoord medisch-specialistische zorg 2019–2022. Accessed October 1, 2021 at: https://eerstekamer.nl/9370000/1/j4nvjlhjvvt9eu4_j9vvkfvj6b325az/vkoyotm271z2
• 13 Doeboek kwaliteitsregistraties. Accessed October 1, 2021 at: https://www.registratieaandebron.nl/files/Doeboek_kwaliteitsregistraties_versie_1.0.pdf
• 14 Lawal AK, Rotter T, Kinsman L. et al. What is a clinical pathway? Refinement of an operational definition to identify clinical pathway studies for a Cochrane systematic review. BMC Med 2016; 14: 35
• 15 ZIRA. Accessed October 11, 2021 at: https://www.nictiz.nl/standaardisatie/referentiedomeinmodellen/zira
• 16 van Bommel AC, Spronk PE, Vrancken Peeters MT. et al; NABON Breast Cancer Audit. Clinical auditing as an instrument for quality improvement in breast cancer care in the Netherlands: the national NABON Breast Cancer Audit. J Surg Oncol 2017; 115 (03) 243-249
• 17 Van Leersum NJ, Snijders HS, Henneman D. et al; Dutch Surgical Colorectal Cancer Audit Group. The Dutch surgical colorectal audit. Eur J Surg Oncol 2013; 39 (10) 1063-1070
• 18 Busweiler LA, Wijnhoven BP, van Berge Henegouwen MI. et al; Dutch Upper Gastrointestinal Cancer Audit (DUCA) Group. Early outcomes from the Dutch Upper Gastrointestinal Cancer Audit. Br J Surg 2016; 103 (13) 1855-1863
• 19 Beck N, Hoeijmakers F, Wiegman EM. et al. Lessons learned from the Dutch Institute for Clinical Auditing: the Dutch model for quality assurance in lung cancer treatment. J Thorac Dis 2018; 10 (Suppl. 29) S3472-S3485
• 20 Ten Berge M, Beck N, Heineman DJ. et al. Dutch lung surgery audit: a national audit comprising lung and thoracic surgery patients. Ann Thorac Surg 2018; 106 (02) 390-397
• 21 Ismail RK, Schramel FMNH, van Dartel M. et al; Dutch Lung Cancer Audit Scientific Committee. The Dutch Lung Cancer Audit: nationwide quality of care evaluation of lung cancer patients. Lung Cancer 2020; 149: 68-77
• 22 Jochems A, Schouwenburg MG, Leeneman B. et al. Dutch Melanoma Treatment Registry: quality assurance in the care of patients with metastatic melanoma in the Netherlands. Eur J Cancer 2017; 72: 156-165
• 23 van Rijssen LB, Koerkamp BG, Zwart MJ. et al; Dutch Pancreatic Cancer Group. Nationwide prospective audit of pancreatic surgery: design, accuracy, and outcomes of the Dutch Pancreatic Cancer Audit. HPB (Oxford) 2017; 19 (10) 919-926
• 24 Alberga AJ, Karthaus EG, Wilschut JA. et al. Treatment outcome trends for non-ruptured abdominal aortic aneurysms: a nationwide prospective cohort study. Eur J Vasc Endovasc Surg 2022; 10: S1078-S5884
• 25 Karthaus EG, Vahl A, Kuhrij LS. et al; Dutch Society of Vascular Surgery, Steering Committee of the Dutch Audit for Carotid Interventions, Dutch Institute for Clinical Auditing. The Dutch audit of carotid interventions: transparency in quality of carotid endarterectomy in symptomatic patients in the Netherlands. Eur J Vasc Endovasc Surg 2018; 56 (04) 476-485
• 26 Poelemeijer YQM, Liem RSL, Nienhuijs SW. A Dutch nationwide bariatric quality registry: DATO. Obes Surg 2018; 28 (06) 1602-1610
• 27 Kuhrij LS, Wouters MW, van den Berg-Vos RM, de Leeuw FE, Nederkoorn PJ. The Dutch Acute Stroke Audit: benchmarking acute stroke care in the Netherlands. Eur Stroke J 2018; 3 (04) 361-368
• 28 de Neree Tot Babberich MPM, Ledeboer M, van Leerdam ME. et al. Dutch Gastrointestinal Endoscopy Audit: automated extraction of colonoscopy data for quality assessment and improvement. Gastrointest Endosc 2020; 92 (01) 154.e1-162.e1
• 29 Spronk PER, Becherer BE, Hommes J. et al. How to improve patient safety and quality of care in breast implant surgery? First outcomes from the Dutch Breast Implant Registry (2015-2017). J Plast Reconstr Aesthet Surg 2019; 72 (10) 1607-1615
• 30 Voeten SC, Arends AJ, Wouters MWJM. et al; Dutch Hip Fracture Audit (DHFA) Group. The Dutch Hip Fracture Audit: evaluation of the quality of multidisciplinary hip fracture care in the Netherlands. Arch Osteoporos 2019; 14 (01) 28
• 31 SNOMED CT. Accessed November 19, 2021 at: https://www.snomed.org
• 32 LOINC. Accessed November 19, 2021 at: https://loinc.org
• 33 Schlegel DR, Ficheur G. Secondary use of patient data: review of the literature published in 2016. Yearb Med Inform 2017; 26 (01) 68-71
• 34 Ingvar M, Blom MC, Winsnes C, Robinson G, Vanfleteren L, Huff S. On the annotation of health care pathways to allow the application of care-plans that generate data for multiple purposes. Front Digit Health 2021; 3: 688218
• 35 Ryan PB, Stang PE, Overhage JM. et al. A comparison of the empirical performance of methods for a risk identification system. Drug Saf 2013; 36 (Suppl. 01) S143-S158
• 36 OMOP – OHDSI. Accessed November 19, 2021 at: https://ohdsi.org/omop/
• 37 Hripcsak G, Duke JD, Shah NH. et al. Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers. Stud Health Technol Inform 2015; 216: 574-578
• 38 OHDSI. Accessed November 19, 2021 at: https://athena.ohdsi.org
• 39 TEHDAS. Report on secondary use of health data through European case studies. 2022. Accessed December 9, 2022 at: https://tehdas.eu/app/uploads/2022/08/tehdas-report-on-secondary-use-of-health-data-through-european-case-studies-.pdf
• 40 Wilkinson MD, Dumontier M, Aalbersberg IJ. et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 2016; 3: 160018
• 41 OECD. Towards an integrated health information system in the Netherlands. Paris: OECD Publishing; 2022. Accessed September 12, 2022 at: https://doi.org/10.1787/a1568975-en
• 42 Mertens S. Het EPD van nu is een soort digitaal kladblok. Med Contemp 2021; 39: 36
• 43 European Commission, Directorate-General for the Information Society and Media, Virtanen M, Ustun B, Rodrigues J, et al., eds. Semantic Interoperability for Better Health and Safer Healthcare: Deployment and Research Roadmap for Europe. Publications Office, 2013. Accessed December 9, 2021 at: https://data.europa.eu/doi/10.2759/38514
• 44 Ebbers T, Takes RP, Smeele LE, Kool RB, van den Broek GB, Dirven R. The implementation of a multidisciplinary, Electronic Health Record embedded care pathway to improve structured data recording and decrease EHR burden; a before and after study (Unpublished manuscript). Department of Head and Neck Oncology, Radboud University Medical Center. 2022
• 45 Ebbers T, Kool RB, Smeele LE. et al. The impact of structured and standardized documentation on documentation quality; a multicenter, retrospective study. J Med Syst 2022; 46 (07) 46
• 46 Zorginformatiebouwstenen 2017. Accessed December 9, 2021 at: https://zibs.nl/wiki/ZIB_Publicatie_2017(NL)

Publication History

Received: 14 October 2022

Accepted: 02 January 2023

Article published online:
03 May 2023