Keywords
diagnoses - electronic health records - patient portals - patient-friendly terms -
SNOMED CT
Background and Significance
Background and Significance
Medical data can be difficult to comprehend for patients; nonetheless, patients increasingly
access their electronic health records (EHRs) through patient portals and personal
health records.[1]
[2]
[3] Many patients prefer patient-friendly, lay language and may require easier synonyms,
definitions, and explanations to clarify medical terminology.[2]
[3]
[4]
[5]
[6] In previous research, it has been shown that clarifications increase the comprehension
of deidentified notes.[7]
[8]
[9]
[10] However, these studies were not performed with the actual EHRs of patients themselves,
have not reported the coverage of their tools in clinical practice, i.e., how many
difficult terms were covered by their functionality, and neither have they assessed
how much the clarification functionality was used in clinical practice, which is important
to determine whether it fulfils an information need from patients. In an earlier study
we performed, we found that patients appreciate clarifications of their personal EHR
notes, but that only few difficult terms were clarified by the functionality, due
to limited coverage of the terminology used and issues with free-text annotation.[11] Data are ideally encoded with medical terminology systems, such that no ambiguity
may arise in identifying the medical concepts that are represented by the terms in
certain text ([Supplementary Appendix 1], available in the online version).
Fortunately, the SNOMED CT Netherlands edition (Nictiz, The Hague) contains the patient-friendly
Dutch language reference set (PFRS) that “states which descriptions are appropriate
to show to patients, caregivers and other stakeholders who have not received care-related
training.” This provides the opportunity to clarify medical data to patients. SNOMED
CT is not used for all types of medical data, but in particular, diagnoses are registered
with the interface terminology Diagnosethesaurus (DT; DHD, Utrecht, Netherlands) in
the Netherlands. DT diagnoses are mapped to SNOMED CT clinical findings and disorders,
through which PFRS descriptions can be obtained. For instance, “phlebitis” in [Table 1] can be clarified by providing a synonym and definition from the PFRS. However, the
coverage of DT diagnoses by PFRS descriptions was low, 1.2% in 2018.[12] Therefore, we developed an algorithm to generalize diagnoses to one or more general,
supertype concepts that do have patient-friendly terms and definitions in the PFRS,
by employing the SNOMED CT hierarchy.[12]
[13] For instance, “pulmonic valve regurgitation” in [Table 1] can be generalized to the medical concept heart valve regurgitation, to provide
the PFRS synonym “leaky heart valve” and the PFRS definition as a clarification. We
showed that this algorithm increases the coverage of diagnoses by clarifications significantly
to 71%.[12] This algorithm is especially relevant for languages such as Dutch that have limited
resources available for medical terminology and language processing,[14] because few patient-friendly terms were needed to clarify a large number of medical
concepts.[12] Two raters with a medical and terminological background validated these generalizations
and considered more than 80% of the clarifications to be correct and acceptable to
use in clinical practice.[15] The clarifications had not previously been evaluated by actual patient portal users
in a real-life setting.
Table 1
Examples of diagnoses registered in Dutch problem lists of medical records and their
corresponding clarifications that can be displayed after clicking on the diagnosis
description or info button
|
Medical diagnosis description
|
Clarification
|
|
Phlebitis
|
Another word for “phlebitis” is vein inflammation: inflammation of a vein, which makes
it red, swollen, and painful.
|
|
Pulmonic valve regurgitation
|
A type of leaky heart valve. Leaky heart valve: this is a heart valve that closes
poorly so that oxygen-rich blood no longer flows properly through the body. This causes
complaints such as shortness of breath, fatigue after exertion, and dizziness.
|
|
Congenital cyst of adrenal gland
|
A type of inborn abnormality and hormonal disorder. Cyst: cavities in the body filled
with liquid.
|
|
Lowe syndrome
|
A type of inborn abnormality, mental disorder, and disorder of brain, kidney, eye,
and metabolism. It is hereditary.
|
Objective
The current study aims to evaluate the implementation of these diagnosis clarifications
in a patient portal problem list, which contains diagnoses, complications, and attention
notes. First, we aimed to evaluate to what extent the clarification functionality
met patient portal users' information needs by assessing the coverage of their diagnoses
by clarifications and by analyzing to what extent they actually use the clarification
functionality when they view their problem list. Second, we evaluated the quality
of the clarifications from the perspective of the users and explored differences in
user, patient and diagnosis characteristics for those users who view the problem list,
and those who use the clarification functionality.
Study Context
The study was performed at the teaching hospital Franciscus Gasthuis & Vlietland (Franciscus).[16] The hospital used the health information system HiX and its patient portal (version
6.2; ChipSoft B.V., Amsterdam, Netherlands). Patients, or their authorized proxies,
use the patient portal, for instance, to view their medical data, schedule appointments,
securely message their health care provider, and complete questionnaires. Proxy users
can be anyone authorized by the hospital or the patients (depending on their age),
such as informal caregivers, case managers, or the parents of a child. The diagnosis
clarifications were implemented in the problem list as illustrated in [Fig. 1]. The description of the diagnosis was highlighted, underlined, and provided with
an info icon if a clarification was available. When clicked, the diagnosis description
and a clarification of the diagnosis were displayed. [Figs. 2] and [3] illustrate the clarifications and functionality to provide feedback. Details on
the terminology implemented in the system are provided in [Supplementary Appendix 1] (available in the online version).
Fig. 1 Problem list with diagnoses, complications, and attention notes. Diagnoses with clarifications
have info buttons: these are highlighted, underlined and followed by an information
icon. Users can click on the info button to view a pop-up with a diagnosis clarification.
(all data presented in [Fig. 1] are not real and completely imaginary)
Fig. 2 Example of diagnosis clarification for the diagnosis “Osteoarthritis of knee.” This
clarification defines the supertype osteoarthritis. Users can provide a rating of
the clarification from (1) very bad to (7) very good. Below, a warning is provided
that the clarification was generated automatically, and questions can be addressed
to the clinician. Additionally, information is provided that the feedback is used
to improve the clarifications and a link is provided to gain further information about
the research.
Fig. 3 Example of diagnosis clarification for the diagnosis “Obesity.” This clarification
consists of a definition from the Dutch SNOMED CT patient-friendly extension. Users
can motivate their rating in free text.
Methods
Study Design
We performed a prospective postimplementation evaluation study with quality improvement
feedback and the reuse of routinely collected data.
Participants
During the 9-week study period from Monday, April 4 to Monday, June 6, 2022, all patient
portal users were included. We analyzed usage data about the logins into the patient
portal, problem list views, diagnoses displayed on the problem list, the number of
diagnoses with clarifications, and which info buttons were clicked on. Users could
log in, view the problem list, display clarifications, and provide feedback multiple
times. This thus resulted in a convenience sample with those users who went through
each of these steps, which we refer to as “conversion steps” and we call the percentage
of users that convert from one step (e.g., logging in) to another (e.g., view the
problem list) conversion rates.
Conversion steps:
-
Login into the patient portal.
-
View the problem list.
-
Click on the info button to view the clarification.
-
Provide feedback on the clarification.
Outcome Measures
The coverage of diagnosis by clarifications was measured as the diagnosis clarification
token coverage[17]: the number of diagnoses with a clarification divided by the total number of diagnoses
viewed on the problem list. The use of the clarification functionality was measured
as the info button click conversion: the number of info buttons clicked on divided
by the total number of info buttons viewed. For each conversion step from login to
rating the clarifications, we reported the percentage of users that converted to that
step, the number of actions (i.e., logins, views, clicks, ratings) they performed,
and the number of unique problems, diagnoses, and info buttons where the actions were
performed on. We aggregated user, patient, and diagnosis characteristics for each
step, to compare differences between subgroups in the conversion rates. User characteristics
were user type (patient or proxy account), and the age group, gender, latest diagnosis
year, and the number of diagnoses of the patient for whom the user used the patient
portal. Diagnosis characteristics were DT concept, clarification type, and medical
specialty.
Data Acquisition
We prospectively reused EHR and audit trail data to derive which diagnoses were viewed
by patients, by which account types, and for which diagnoses users clicked on the
info button. We retrospectively reused diagnoses, age, and gender already registered
in the EHR to explore differences in user, patient, and diagnosis characteristics.
The feedback questions asked were simple and minimally invasive: (1) Please rate this
explanation (1: very bad; 7: very good)? (2) Why? Can you explain your score? The
questionnaire functionality of the EHR was used for this purpose. The feedback was
monitored by the hospital staff, to assess whether it contained questions or if any
issues arose that needed to be addressed.
Aggregated data on all patient portal users during the study period were exported
from the EHR. To protect the privacy of the patients, variables such as gender and
age were aggregated in separate tables so they could not be combined. Free text from
the feedback provided was anonymized by an authorized hospital functionary. Anonymization
was performed by removing directly identifying data, such as dates, places, and names
of patients, clinicians, or others. EHR, audit trail, and free-text data were made
available by the hospital without any directly identifying personal information.
Statistical Analysis
Conversion rates were calculated and aggregated by the different outcome measure levels.
For the number of actions, the interquartile ranges (IQRs) and the maximum number
of actions per user were reported. We calculated the total diagnosis clarification
token coverage and info button click conversion rate, and calculated it for each patient
and took the median and IQR per patient. For the clarification quality ratings, we
used the median and IQR of the median rating per patient and clarification. The difference
in ratings for clarifications with patient-friendly synonyms and definitions compared
with clarifications with generalizations to concepts with patient-friendly synonyms
and definitions was tested with the Mann–Whitney U test.[18] Two authors (H.J.T.v.M. and G.E.G.H.) analyzed the comments thematically and summarized
them narratively. Any differences were discussed until consensus was achieved. Differences
among users and diagnosis characteristics in the proportions of problem list views
and info button clicks were tested with the Fisher exact test.[19] The p-values were corrected for false discovery rate with the Benjamini–Yekutieli method.[20] Odds ratios (ORs) were calculated post-hoc for each variable to estimate the associations
between the characteristics and the views and clicks, comparing the odds of the particular
variable (e.g., patients of the female gender) with a reference group (e.g., male
gender). We took the largest group as the reference group. Data were analyzed using
the R programming language (R Foundation for Statistical Computing, Vienna, Austria;
Version: 4.2.1, 2022–06–23) in RStudio (RStudio Inc., Boston, Massachusetts, United
States; Version: 2022.07.1). See the R script in [Supplementary Appendix 2] (available in the online version).
Results
Study Population
In total, for 19,961 patients users had logged in at least once during the 9-week
study period. Logins came from all age groups, the largest group logged in for patients
in their 30s (18.1%), followed by 60s (17.4%) and 50s (16.5%). Relatively more logins
were for women (61.8%) and few users logged in with proxy accounts (0.2%). [Table 2] shows the user characteristics for each step. [Supplementary Appendix 3] (available in the online version) contains the complete results dataset.
Table 2
The number of users that logged in per patient characteristic and user account type,
additionally for whether they viewed the problem list, viewed info buttons on the
problem list, clicked on info buttons on the problem list and provided feedback, with
the number of patients and percentages of the total number of patients for which that
action was performed
|
Statistic
|
Value
|
Logged in
|
Viewed problem list
|
Viewed info buttons on list
|
Clicked on info buttons
|
Provided feedback
|
|
n
|
(%)
|
n
|
(%)
|
n
|
(%)
|
n
|
(%)
|
n
|
(%)
|
|
Age
|
00–09
|
373
|
(1.9)
|
130
|
(2.0)
|
54
|
(2.3)
|
32
|
(2.5)
|
2
|
(2)
|
|
10–19
|
492
|
(2.5)
|
166
|
(2.5)
|
58
|
(2.5)
|
33
|
(2.6)
|
0
|
(0)
|
|
20–29
|
2,280
|
(11.4)
|
764
|
(11.7)
|
181
|
(7.7)
|
95
|
(7.4)
|
6
|
(6)
|
|
30–39
|
3,612
|
(18.1)
|
1,085
|
(16.6)
|
265
|
(11.2)
|
138
|
(10.7)
|
7
|
(6)
|
|
40–49
|
2,807
|
(14.1)
|
929
|
(14.2)
|
333
|
(14.1)
|
191
|
(14.8)
|
17
|
(15)
|
|
50–59
|
3,284
|
(16.5)
|
1,110
|
(17.0)
|
442
|
(18.7)
|
259
|
(20.1)
|
18
|
(16)
|
|
60–69
|
3,478
|
(17.4)
|
1,193
|
(18.3)
|
487
|
(20.6)
|
264
|
(20.5)
|
28
|
(25)
|
|
70–79
|
2,861
|
(14.3)
|
923
|
(14.1)
|
417
|
(17.7)
|
218
|
(16.9)
|
25
|
(23)
|
|
80–89
|
711
|
(3.6)
|
211
|
(3.2)
|
114
|
(4.8)
|
56
|
(4.3)
|
5
|
(5)
|
|
90–
|
63
|
(0.3)
|
19
|
(0.3)
|
12
|
(0.5)
|
5
|
(0.4)
|
0
|
(0)
|
|
Age
subgroup
|
00
|
17
|
(0.1)
|
7
|
(0.1)
|
2
|
(0.1)
|
1
|
(0.1)
|
0
|
(0)
|
|
01–09
|
356
|
(1.8)
|
123
|
(1.9)
|
52
|
(2.2)
|
31
|
(2.4)
|
2
|
(2)
|
|
10–11
|
54
|
(0.3)
|
14
|
(0.2)
|
3
|
(0.1)
|
1
|
(0.1)
|
0
|
(0)
|
|
12–15
|
131
|
(0.7)
|
51
|
(0.8)
|
24
|
(1.0)
|
15
|
(1.2)
|
0
|
(0)
|
|
16–17
|
94
|
(0.5)
|
33
|
(0.5)
|
13
|
(0.6)
|
6
|
(0.5)
|
0
|
(0)
|
|
18–19
|
213
|
(1.1)
|
68
|
(1.1)
|
18
|
(0.8)
|
11
|
(0.9)
|
0
|
(0)
|
|
Gender
|
Male
|
7,629
|
(38.2)
|
2,375
|
(36.4)
|
856
|
(36.2)
|
471
|
(36.5)
|
46
|
(43)
|
|
Female
|
12,332
|
(61.8)
|
4,155
|
(63.6)
|
1,507
|
(63.8)
|
820
|
(63.5)
|
62
|
(57)
|
|
Account
|
Proxy
|
42
|
(0.2)
|
12
|
(0.2)
|
7
|
(0.3)
|
2
|
(0.2)
|
0
|
(0)
|
|
Patient
|
19,923
|
(99.8)
|
6,519
|
(99.8)
|
2,357
|
(99.8)
|
1,289
|
(99.9)
|
108
|
(100)
|
|
Total
|
|
19,961
|
(100.0)
|
6,530
|
(100.0)
|
2,363
|
(100.0)
|
1,291
|
(100.0)
|
108
|
(100)
|
Problem List and Clarification Views
[Fig. 4] and [Table 3] show the conversion rates and the number of actions performed for each step. This
provides detailed insight into the usage patterns. The problem list of 6,530 patients
was viewed (32.7% of the patients for whom users had logged in), 2,660 (13.3%) had
viewed DT-encoded diagnoses on their problem list, and 2,363 (11.8%) had viewed info
buttons on their problem list on which they could have clicked. Therefore, for 88.8%
(2,363/2,660) of patients of whom DT encoded diagnoses on their problem list were
viewed, an info button was available to view a clarification. When info buttons were
available, a median of 1 (IQR: 1–2; maximum: 10) info button was on their problem
list. The diagnosis clarification token coverage was 81.7% (4,977/4,069) and the coverage
per patient had a mean of 100% (IQR: 75–100%). One or more info buttons were clicked
on for diagnoses of 1,291 patients, which is 54.6% of the patients for whom info buttons
were viewed and 6.5% of the patients for whom was logged in. On average, users clicked
twice (IQR: 1–3; maximum: 31) on one info button (IQR: 1–1; maximum: 8). The info
button click conversion rate for all info buttons viewed was 43.5% (1,770/4,069) with
a median click conversion of 50% (IQR: 0–100%) per patient. Of the patients who clicked
on an info button, 108 (8.4%) provided a rating (0.5% of the patients who had logged
in).
Fig. 4 The number and percentage (conversion rates) of patients for whom users went through
the conversion steps. Not all patients had problems, diagnoses, or info buttons on
their problem list, which is illustrated with the dashed arrows.
Table 3
Percentages of logins, views, clicks and ratings, with the number of patients (n) for whom was logged in, the percentage of patients for whom the problem list was
viewed, for whom a problem was viewed on the problem list, for whom diagnoses were
viewed, for whom info buttons were viewed, for whom an info button was clicked on
and for whom a clarification was rated, of the total number of patients for whom users
logged in; it additionally shows the total number of actions, with the quartiles min,
25%, median, 75%, and max, and the average number of actions per patient; the number
of actions is distinguished from the number of problems or diagnoses that were viewed
(i.e., the number of times ratings were provided for which number of diagnoses)
|
Statistic
|
Patients
|
Actions
|
|
n
|
%
|
Level
|
Total
|
Min
|
25%
|
Median
|
75%
|
Max
|
Average
|
|
Logged in
|
19,961
|
100.0
|
Logins
|
69,112
|
1
|
1
|
2
|
3
|
260
|
3.5
|
|
Viewed problem list
|
6,530
|
32.7
|
Views
|
17,414
|
1
|
1
|
2
|
3
|
63
|
2.7
|
|
Viewed problems
|
3,961
|
19.8
|
Views
|
34,539
|
1
|
2
|
4
|
9
|
474
|
8.7
|
|
Problems
|
11,145
|
1
|
1
|
2
|
4
|
20
|
2.8
|
|
Viewed diagnoses
|
2,660
|
13.3
|
Views
|
16,012
|
1
|
2
|
3
|
6
|
197
|
6.0
|
|
Diagnoses
|
4,977
|
1
|
1
|
1
|
2
|
16
|
1.9
|
|
Viewed info buttons
|
2,363
|
11.8
|
Views
|
13,235
|
1
|
1
|
3
|
6
|
165
|
5.6
|
|
Diagnoses
|
4,069
|
1
|
1
|
1
|
2
|
10
|
1.7
|
|
Clicked on info buttons
|
1,291
|
6.5
|
Clicks
|
2,979
|
1
|
1
|
2
|
3
|
31
|
2.3
|
|
Diagnoses
|
1,770
|
1
|
1
|
1
|
1
|
8
|
1.4
|
|
Rated clarifications
|
108
|
0.5
|
Ratings
|
133
|
1
|
1
|
1
|
1
|
4
|
1.2
|
|
Diagnoses
|
127
|
1
|
1
|
1
|
1
|
4
|
1.2
|
Clarification Quality Ratings
A total of 108 users rated the quality of 127 diagnoses (103 unique diagnoses with
95 unique clarifications). Users rated the clarifications with a median of 6 (IQR:
4–7; [Fig. 5]). Clarifications with synonyms and definitions were rated higher than clarifications
with generalizations to supertypes (median: 6, compared with median: 5.5; p = 0.0379), see [Fig. 6]. Users provided a comment on 66 of the 127 diagnoses (56%). We identified 16 themes
in the comments and the most common ones were that they found the clarification clear
(n = 25; 38%) or incomplete (n = 10; 15%), provided input for improvement (n = 10; 15%), found the clarification unclear (n = 5; 8%), or disagreed with the diagnosis rather than the clarification (n = 4; 6%). Additionally, some users (n ≤ 3; ≤5%) commented they recognized the clarification based on their own experience,
that they found the clarification was right or useful, asked for a solution for their
health problem, disagreed with the treatment, clarification, and/or diagnosis (sometimes
not a clear distinction), or mentioned the existence of alternative sources of clarifications.
Fig. 5 Bar plot and box plot of the median ratings per patient. The bar plot shows the median
ratings per patient for each rating from (1) very bad to (7) very good and the percentage,
and the number of patients (n). The box plot below the bar plot shows the median (median: 6), interquartile range
(IQR: 4–7), and jittered scatter of the ratings.
Fig. 6 Box plots of ratings from (1) very bad to (7) very good for all clarifications (“All,”
left), clarifications with patient-friendly synonyms and definitions (“Descriptions,”
center), and clarifications with generalization to supertypes with patient-friendly
synonyms and definitions (“Generalizations,” right).
Differences between Subgroups
After correcting for the false discovery rate (see [Supplementary Table S2] in [Supplementary Appendix 4], available in the online version), differences in the proportion of users who viewed
the problem list were found significant for gender (p = 0.0037) and latest diagnosis year (p = 0.0037). The odds of viewing the problem list were lower for male compared with
female patients (OR: 0.89; confidence interval [CI]: 0.84–0.95) and higher for patients
having the latest diagnosis in the year 2022 (when the study was performed) compared
with those having no diagnosis (OR: 1.35; CI: 1.20–1.53). Differences in the proportion
of users who clicked on an info button were significant for the latest diagnosis year
(p = 0.0003) and medical specialty (p = 0.0037). The odds of clicking on an info button were higher for patients having
the latest diagnosis in 2022 (OR: 3.08; CI: 2.30–4.15) and 2021 (OR: 1.33; CI: 1.02–1.74)
compared with 2020. Compared with orthopaedics, the odds of clicking were lower for
ear, nose and throat surgery (OR: 0.78; CI: 0.61–0.99), dermatology (OR: 0.56; CI:
0.42–0.74), surgery (OR: 0.75; CI: 0.56–0.99), ophthalmology (OR: 0.64; 0.48–0.85),
urology (OR: 0.52; CI: 0.34–0.78), plastic surgery (OR: 0.51; CI: 0.28–0.93), and
gynecology (OR: 0.51; CI: 0.26–0.98). See [Supplementary Appendix 4] (available in the online version) and [Supplementary Tables]
[Supplementary Table S3]–[S8] (available in the online version) for the proportions and ORs of the subgroup variables.
Unexpected Observations
During monitoring, we noticed two events that were not expected. One user (rating:
5) wrote “I have this pain already for [x] years, why can they not do anything about it, life
keeps getting more unbearable.” The hospital verified whether the patient required a follow-up, but there already
was a follow-up scheduled. Therefore, it was decided that further action was not necessary.
In a second case (rating: 1), a user commented he or she did not have the diagnosis
and that this was confirmed by the clinician.
Discussion
This study provided insight into patient portal user information needs by measuring
and evaluating the actual coverage and use of a clarification functionality for the
problem list. The coverage of diagnoses by clarifications was high, with almost 90%
of patients having clarifications for one or more diagnoses on their problem list.
More than half of the users who could use the info buttons clicked on them during
the study period and on average they clicked on half of the info buttons available
in their problem list. Overall, clarifications were rated as having good quality.
Clarifications by synonyms and definitions of supertypes were rated relatively lower
than clarifications with synonyms and definitions of the diagnoses themselves. The
odds that the problem list was viewed were relatively higher for patients of the female
gender and with a more recent diagnosis. The odds that info buttons were clicked to
view clarifications were relatively higher for patients with a recent diagnosis and
relatively lower (compared with orthopaedics) for diagnoses from the specialties ear,
nose and throat surgery, dermatology, surgery, ophthalmology, urology, plastic surgery,
and gynecology.
Similar studies have not performed an evaluation study in clinical practice but relied
on online surveys,[21] laboratory situations,[7]
[10]
[21] or only performed expert evaluation.[22]
[23]
[24] Additionally, previous studies did not use personal medical data and were focused
on notes, rather than encoded diagnoses. Therefore, the current study is novel in
that we prospectively evaluated clarifications in a real-time patient portal with
patients' personal medical data, showing that end-users use and appreciate clarification
functionality. Patients have been reported to find errors in their notes and to consider
some medical record content to be judgmental and offensive.[25]
[26]
[27] It appears that the clarifications help users to verify whether the diagnosis is
correct, as our second example in the “Unexpected Observations” section illustrates.
Some authors[25]
[28] argue that medical jargon should be replaced by language that treats patients less
belittling, passive, childish, and blamable. The evaluated solution in the present
study, however, does not require clinicians to change the way they register their
data. It combines the strength of more professional phrasing, as the content was already
encoded with terminology systems, with clarification by the functionality.
To our knowledge, this is the first study that evaluates clarifications in a patient
portal. Reusing existing log and EHR data provides a more representative picture of
users and their behavior than making patients or laymen fill out surveys and using
fabricated nonpersonal data,[7]
[10]
[21] as we were able to include a wide variety of users in the convenience samples of
each conversion step. The brief quality ratings were minimally invasive for end users.
Some users disagreed with the diagnosis and one with their treatment, and accordingly
rated the clarification as very bad. Conversely, a user commented that the clarification
was a good addition to the drawing a clinician made and rated the clarification as
very good. Where users did not comment, we could not verify whether they based the
rating on the clarification only or also on the diagnosis or experience with their
clinician. This might affect the ratings and the ratings thus reflect a mix of the
quality of the clarification, the data quality, and the experience with the clinician.
Without the permission of the users, we could not obtain individual patient data to
run a multivariate model. Therefore, this research was limited to aggregate data and
associations could hence not be corrected for confounders. The aggregate data provided
insight into different user groups. However, the few differences in conversion we
found between users and diagnoses were based on sample sizes that lowered along the
conversion steps. Differences might have resulted coincidentally due to multiple testing
and confounding. We tried to minimize the false discovery rate and might have unnecessarily
discarded associations such as age and problem list viewing (e.g., the problem list
appears to be viewed significantly more often for patients in their 30s compared with
patients in their 60s). However, we still were able to provide some insight for further
studies with a rich descriptive dataset.
This study shows that generalization is a useful technique to generate clarifications
from the perspective of actual patient portal users. For terminology developers, the
approach has the potential to make more maintainable terms and definitions that can
be reused among several medical concepts. In further research, tailoring clarifications
to end-users, especially on a more accessible language difficulty level, and developing
clarifications for particular diagnosis classes should be investigated, improving
the clarifications and functionality. The coverage of the current system can be increased
by updating the terminology versions, developing clarifications for other types of
medical data, and applying other clarification methods, such as using relationships
other than is-a relationships in SNOMED CT, such as the finding site (e.g., pancreas)
and associated morphology (e.g., inflammation) to clarify concepts (e.g., deriving
“inflammation of the pancreas” from “pancreatitis”). The associations found indicate
that there are differences in usage between groups, which might reflect that they
have different information needs. The unexpected observations imply that asking for
free-text feedback about diagnosis clarifications should also involve follow-up, as
patients sometimes do not understand or agree with the diagnosis. The hospital decided
to continue showing the clarifications after the study period, but without asking
for free-text feedback, because there was no solution yet for continuing follow-up
and free-text anonymization to share the feedback for clarification quality improvement.
Health care institutions should determine how to deal with these issues before implementing
such functionality, as user input can help improve medical record accuracy and clarification
quality.
Conclusion
The coverage of diagnoses by clarifications based on an algorithm that generalizes
diagnoses to concepts with patient-friendly terms and definitions was high and the
majority of users used the clarification functionality. Overall, users considered
it good clarifications, but they also identified opportunities for improving the clarity
and completeness of some clarifications. Future research should address the improvement
of the clarification coverage and quality, and further investigate differences between
subgroups to assess specific user group needs and prioritize areas of improvement.
Clinical Relevance Statement
Clinical Relevance Statement
While medical data had traditionally been registered for clinical purposes and clinicians
only, patients—who often have not had any medical training—currently access their
health records. This study presents a generic solution to make medical data, in particular
diagnoses, more understandable for patients, without creating an additional administrative
burden for clinicians, because clarifications are provided to data that already are
routinely registered in health records. The functionality is used and appreciated
by patient portal users.
Multiple-Choice Questions
Multiple-Choice Questions
-
What method was applied to increase the coverage of diagnoses by clarifications?
-
Synonymy
-
Generalization
-
Definition
-
Patient education
Correct Answer: The correct answer is option b. The coverage of diagnoses by clarifications was increased
9.4 times by generalizing diagnoses to more general concepts with patient-friendly
terms and definitions.
-
What part of the patient portal users actually views diagnosis clarifications by clicking
on an info button in this study?
Correct Answer: The correct answer is option c. More than half of the users who viewed their problem
list and had one or more info buttons to click on, clicked on an info button to view
a diagnosis clarification.
-
The odds ratios were found to be significantly different for some groups of patients
and diagnoses. For which patients or diagnoses were the odds ratios of viewing clarifications
lower?
-
Diagnoses from dermatology
-
Patients with a recent diagnosis
-
Diagnoses from orthopaedics
-
Patients with female gender
Correct answer: The correct answer is option a. The odds ratio of diagnosis clarifications being
viewed from dermatology compared with orthopaedics was 0.56 (CI: 0.42–0.74). However,
the relative odds of viewing clarifications were higher for patients with a recent
diagnosis and they were not different for female compared with male patients.
