Background and Significance
Background and Significance
Patient portals allow patients to track test results, report self-administered medications,
and communicate with their providers through portal-mediated secure messages (PSMs).[1] Oncology patients rate this opportunity to engage with their care team as more important
than other patients, likely related to the increased complexity of their management.[2] PSMs provide a valuable source of information about patients, including references
to health needs, care coordination, and questions about their treatment plans.[3]
[4]
Some patients prefer that their caregivers have access to their patient portal to
help coordinate care and overcome barriers such as geographic distance.[5] This has led to the development of registered proxy accounts or the setup of separate,
distinct accounts for caregivers.[6]
[7] However, an estimated less than 1% of caregivers utilize formal proxy accounts because
it is more convenient for them to use the patient's account.[5] As such, PSMs are frequently not authored by the patient, requiring careful interpretation
of the messages by the care team.[5]
[8] A study found that 46% of adult diabetic patients had at least one PSM authored
by a caregiver who used the patient's portal credentials to log in (unregistered proxy)
instead of using their proxy account.[9] Identification of unregistered proxy users can enhance patient security through
accurate authentication, ensure that needs expressed in PSMs are appropriately being
responded to whether from patients or caregivers, and help systems leverage technology
to address barriers to portal engagement.
Little is known about the prevalence of unregistered proxy use among oncology patients
or the demographics of oncology patients that may have others send PSMs on their behalf.
This study aimed to determine if sociodemographic differences exist between patients
with a high number of presumed-unregistered proxy messages compared with those with
few to no unregistered messages in our patient population.
Methods
This study took place at Memorial Sloan Kettering Cancer Center (MSK), a National
Comprehensive Cancer Network center with seven regional campuses in New York. MSK's
homegrown patient portal system, MyMSK, started enrollment in 2006 and began supporting
registered proxy accounts in 2017. MSK provides an optimal environment for investigation
given the high rate of patient engagement. As of 2022, 89% of active patients, those
who received treatment within the past year, had MyMSK accounts with 13% of patients
with a MyMSK account having at least one formally registered proxy account. To be
included in our study, PSMs must have been sent from a patient or registered proxy
account to a physician's inbox after December 31, 2016. Additionally, the PSM had
to be the first message in a conversation chain. A total of 1.5 million PSMs met these
criteria. Our cohort was randomly selected based on the power of our available computational
resources.
The use of natural language processing (NLP) models to extract data from unstructured
free text in the electronic health record (EHR) has demonstrated efficiency and potential
to reduce missed clinically significant information compared with manual review.[10]
[11] The complete methodology is described in a manuscript detailing the development
of our NLP model to classify the authorship of PSMs.[12] In brief, we trained an NLP model on a dataset of 1,850 individual messages with
manual annotation of patient or presumed proxy as our gold standard in concordance
with other NLP models categorizing PSMs.[3]
[13]
[14] This model achieved an AUC of 0.93. To then categorize a patient's account, we determined
how often a true proxy user's messages predict as proxy-written utilizing the model.
Running the model on a new set of messages (training data), all written by registered
proxy users determined that, on average, a confirmed registered proxy only had 26.1%
of messages that were flagged by the model. We then performed further analyses to
select a cut-off value of 0.25, which maximized accuracy and sensitivity of the model.[12] Given that unregistered proxy accounts likely behave linguistically similar to registered
proxy accounts, the threshold for classifying a user as an unregistered proxy was
set at a positive prediction for 25% of their messages. We then applied the model
to messages from 12,119 randomly selected patients.
Our primary statistical analyses involve comparing differences in self-reported sociodemographic
variables between accounts that were categorized by the model as presumed proxy and
those that were not. Continuous variables are summarized using median and interquartile
range (IQR, 25th and 75th percentiles) and compared using the Wilcoxon rank sum test.
Categorical variables are described using count and percentage and compared using
the chi-square test.
Results
Our NLP model classified 12,119 individual messages from unique patients, of which
1,253 (10.3%) appeared to be sent by someone other than the patient. Patient demographics
comparing predicted proxy and nonproxy users are summarized in [Table 1]. There was no difference in ethnicity or marital status between cohorts. Patients
predicted to have proxies sending PSM were significantly more likely to be male, older
(68 vs. 61 years old), and identify as a nonwhite race. Additionally, there was a
higher representation of patients with a non-English preferred language, noncitizens,
and public insurance type among presumed proxy accounts.
Table 1
Sociodemographics differences for patients with no proxy messages compared with those
with a presumed proxy
|
Predicted proxy
|
|
|
0 (N = 10,863)
|
1 (N = 1,256)
|
Total (N = 12,119)
|
p-Value
|
|
Patient sex
|
|
Female
|
06299 (57.99%)
|
00591 (47.05%)
|
06890 (56.85%)
|
<0.001[a]
|
|
Male
|
04564 (42.01%)
|
00665 (52.95%)
|
05229 (43.15%)
|
|
Race
|
|
Asian
|
00724 (06.97%)
|
00099 (08.41%)
|
00823 (07.12%)
|
0.010[a]
|
|
Black
|
00517 (04.98%)
|
00065 (05.52%)
|
00582 (05.03%)
|
|
Other
|
00288 (02.77%)
|
00048 (04.08%)
|
00336 (02.90%)
|
|
White
|
08861 (85.28%)
|
00965 (81.99%)
|
09826 (84.95%)
|
|
Unknown
|
00473 (04.35%)
|
00079 (06.29%)
|
00552 (04.55%)
|
|
Ethnicity
|
|
Hispanic
|
00595 (05.74%)
|
00078 (06.62%)
|
00673 (05.83%)
|
0.222
|
|
Non-Hispanic
|
09778 (94.26%)
|
01101 (93.38%)
|
10879 (94.17%)
|
|
Unknown
|
00490 (04.51%)
|
00077 (06.13%)
|
00567 (04.68%)
|
|
Marital status
|
|
Married/life partner
|
07456 (70.83%)
|
00849 (71.05%)
|
08305 (70.85%)
|
0.875
|
|
Not married
|
03071 (29.17%)
|
00346 (28.95%)
|
03417 (29.15%)
|
|
Unknown
|
00336 (03.09%)
|
00061 (04.86%)
|
00397 (03.28%)
|
|
Preferred language
|
|
English
|
10450 (97.44%)
|
01133 (91.82%)
|
11583 (96.86%)
|
<0.001[a]
|
|
Non-English
|
00275 (02.56%)
|
00101 (08.18%)
|
00376 (03.14%)
|
|
Unknown
|
00138 (01.27%)
|
00022 (01.75%)
|
00160 (01.32%)
|
|
Age
|
|
Median (IQR)
|
61.0 (51.0–70.0)
|
68.0 (56.0–78.0)
|
62.0 (51.0–71.0)
|
<0.001[a]
|
|
Range
|
15.00–98.00
|
5.00–99.00
|
5.00–99.00
|
|
n
|
10863
|
1256
|
12119
|
|
Insurance type
|
|
Other
|
00839 (07.77%)
|
00135 (10.80%)
|
00974 (08.08%)
|
<0.001[a]
|
|
Private
|
05400 (49.99%)
|
00373 (29.84%)
|
05773 (47.90%)
|
|
Public
|
04563 (42.24%)
|
00742 (59.36%)
|
05305 (44.02%)
|
|
Unknown
|
00061 (00.56%)
|
00006 (00.48%)
|
00067 (00.55%)
|
Abbreviation: IQR, interquartile range.
a Statistically significant.
Discussion
Our results showed that oncology patients with a presumed unregistered proxy were
more likely to be older, nonwhite, and have a non-English preferred language. This
is in concordance with known disparities in portal use and associated PSM studies
in a diabetic patient cohort where racial minorities, older patients, and patients
with limited English proficiency were more likely to have unregistered proxy-written
PSM.[7] In contrast, an analysis from Mayo Clinic found that up to 16% of PSMs were estimated
to be sent by an unregistered proxy, and no significant differences in race or age
were noted.[6] This may reflect the lack of racial and ethnic diversity in their study, with over
90% of patients identifying as white.
Patient EHR access is linked to improved patient satisfaction and outcomes through
the promotion of patient engagement and shared decision-making.[15]
[16]
[17]
[18] PSMs allow for opportunities for further communication, engagement, and education,
which has been reported to be particularly valuable among oncology patients.[19] Engaging caregivers offers a clinical benefit to patients, with joint access to
patient portals enhancing discussion and agreement about care plans.[20]
[21] However, there are multiple requirements for patients to benefit from patient portal
use. Patients/and or their proxies need internet access, computer literacy, and health
literacy.[15]
[21] Recognizing the impact of demographics on the utilization of portals and messaging
is important to improve the inclusiveness of the patient portal design.
Barriers to the uptake of patient portal use extend to the creation of proxy accounts
with inequitable utilization worsening privacy and security for vulnerable groups
if not addressed. Some portals offer variations in the degree of information access
via proxy accounts, but controlling these various settings may involve increasing
complexity for patients and their families.[22]
[23] Proxy account setup should include easy-to-follow instructions and technical assistance
so that all patients can equitably access important security features. Account setup
should also include options to give proxies full access to information, if the patient
consents, to mitigate the need for proxies logging into the patient account to view
needed information, which can result in confusion for the medical team and issues
related to detecting true account security breaches. Preventing breaches in patient
confidentiality is key to maintaining trust in medical practice and alleviating concerns
regarding the privacy of health information.[16]
[24]
Also, needs expressed in the PSMs may reflect the needs of a caregiver rather than
the patient. The burden experienced by cancer caregivers disproportionately impacts
black and Hispanic caregivers with more caregiving tasks performed, less social support,
and greater financial burden.[25]
[26] PSM (from registered and unregistered proxy accounts) may provide an existing resource
for identifying specific challenges faced by caregivers that can inform intervention
development to support those most in need. Further, reliably distinguishing between
message authors is critical to allow for appropriate response or action and for ongoing
efforts to utilize machine learning to extract information from PSM, such as patient-reported
outcomes.[27]
This study reviewed PSMs sent by patients receiving treatment at a specialized cancer
center with an established patient portal. As such, these results may differ from
other medical settings with less complex care and may not generalize to all populations.
Additionally, reliance on pronoun use to manually annotate messages may have inadvertently
misclassified non-English speakers or those with lower education levels. Messages
sent by a proxy using first-person singular pronouns (i.e., being intentionally deceptive
and posing as the patient) would not have been identified. Utilization of race data
is limited by accuracy and completeness in the EHR.[28] Further investigation is necessary to assess interventions to decrease unregistered
proxies and ensure that patient portal platforms provide the support necessary for
all patients to engage meaningfully and securely. Future work should incorporate clinical
characteristics, caregiver sociodemographics, examine differences between registered
and unregistered proxies, and consider the impact on patient outcomes.
Conclusion
Sociodemographic differences in patients with unregistered proxy users highlight groups
that may not be well supported by the current functionality of patient portals. Identification
of unregistered proxies provides an opportunity to develop a process to educate caregivers
about registered accounts and determine if new initiatives are necessary to assist
caregivers. For diverse patients to benefit from patient portals, portal optimization
and further assessment of barriers are required.
