Use of the Direct Standard for Patient Event Notifications: A Qualitative Study Among Industry Leaders

Abstract Background  Health care organizations seek to maximize efficiency and effectiveness when sending patient event notifications required by the U.S. Centers for Medicare and Medicaid Services Interoperability and Patient Access Final Rule. Objectives  To identify the barriers for senders (i.e., hospitals) and receivers (Department of Veterans Affairs [VA] and non-VA) of patient event notifications using the Direct Standard, the transport method standard that addresses Direct Messaging (DM). Methods  Questions were asked to better understand perceived barriers to sending and receiving event notifications as well as involvement with event notifications with VA. Open coding was used to identify themes in the transcribed interviews. Results  Analysis of the interviews ( n  = 17) showed workflow barriers, including identifying a patient's provider, the provider's Direct address, and whether a patient is an enrolled Veteran, were the most common barriers. Next were technical barriers, such as payload and electronic health record ingestion of event notifications. Less common barriers included content, policy/governance, cost, and organizational issues. The interviews also highlighted a promising pilot between VA and an aggregator. Conclusion  Overall, interviewees felt that event notifications are a benefit from a coordination of care perspective for patients and clinical care teams. Also, interviewees felt that DirectTrust's Implementation Guide for Event Notifications via the Direct Standard has helped guide (and perhaps the DirectTrust Directory could help guide) the industry in sending and more effectively receiving event notifications.


Background and Significance
Health information exchange (HIE) is the act of electronically sharing health data between two or more organizations. 1HIE typically occurs within a network of organizations that trust one another and provide governance of the exchanged data. 2,3Although earlier studies (2015 and prior) suggested a mixed benefit of HIE, [4][5][6][7][8] later studies showed more consistent favorable benefits (e.g., cost savings and reduced readmission) in the use of HIE. 9,10o build on the favorable benefits and maximize effectiveness and efficiency, HIE networks should utilize standards to facilitate interoperable sharing and use of health data.The Direct Standard is the transport method standard that addresses Direct Messaging (DM). 11 Health information service providers (HISPs) encrypt Direct messages with a digital certificate and a public key infrastructure so that only the intended receiver can decrypt and read the messages. 12][15] As a matter of context, an event notification is an electronic communication, within or between organizations, for a transition in care (e.g., admission, discharge, and transfer).Although previous studies have established the benefits of event notifications, [16][17][18][19][20][21][22] the implementation and efficiencies by health care organizations of sending event notifications have been inconsistent.Initiated in July 2020 and adopted by American National Standards Institute in 2022, DirectTrust created the DirectTrust Implementation Guide (IG) for Event Notifications via the Direct Standard, 23 which specifies the standards and metadata elements that enable a receiver of event notifications to be more efficient and effective in handling an incoming event notification. 24Fig. 1 25 summarizes the event notification portion of the May 2020 Centers for Medicare and Medicaid Services' (CMS) Interoperability and Patient Access Final Rule (CMS-9115-F). 26hile CMS spelled out the penalties for noncompliance in the Final Rule, 27 CMS did not specify the mode of electronic transport for event notifications-leading to uncertainty for hospitals on how to implement the requirement.
Department of Veterans Affairs (VA) is not a participant in CMS, so CMS-9115-F is not applicable to VA.As such, VA does not have to send event notifications.However, in testimony to the House Committee on Veterans' Affairs in July of 2022, VA purported that community care included 44% of health care services for enrolled Veterans. 28VA understands the importance of event notifications in ensuring care coordination for Veterans.Since the implementation of CMS Final Rule, there has been only anecdotal data to suggest that more could be done to facilitate event notifications by and between non-VA health systems and VA to ultimately improve care coordination for over nine million enrolled Veterans (of the approximately 19 million living Veterans) 29 that VA serves.

Objectives
This study sought to identify barriers for non-VA senders of event notifications to VA using the Direct Standard.Moreover, this research sought to identify how technical, workflow, standards, and organizational factors contribute to an implementation of event notifications by non-VA senders to VA.Both VA and non-VA interviewees were included to gain the perceived barriers from the perspective of both senders and receivers.The insight gained in this study will help (non-VA) senders of event notifications (using the Direct Standard) to implement effective and efficient processes so that receivers (e.g., VA) can rapidly and more efficiently expand receipt of event notifications at a national level to maximize the coordination of care.

Study Design and Setting
The lead author conducted semistructured interviews with leaders within the DM industry to identify barriers and other factors for community (non-VA) health care systems/facilities to send event notifications to VA using the Direct Standard.Veterans Health Administration, part of VA, serves over nine million enrolled Veterans (of the approximately 19 million living Veterans) and is the largest integrated health care system in the nation. 29An enrolled Veteran is a VA patient who is eligible to receive health care from the VA's almost 1,300 medical centers and outpatient clinics. 29,30o improve Veteran access to care, several initiatives have greatly expanded community care (i.e., health care received by enrolled Veterans outside of VA [i.e., by non-VA organizations]) options for Veterans.To improve interoperability between VA and non-VA health care organizations, VA implemented a DM infrastructure: VA Direct (Legacy) in 2013 and VA Direct (Electronic Health Record (EHR) Modernization) in 2020. 31There may be interoperability or other challenges between VA and non-VA organizations that were out of scope for this study.

Participants and Recruitment
Purposive (n ¼ 17 interviewees) sampling was used to recruit participants.We targeted participants across the DM industry with minimum 4 years' experience either developing or deploying (or both) the Direct Standard in their respective organizations as well as those who were mostly outside of VA to concentrate on senders of event notifications to VA since VA is not required by CMS to send event notifications.Snowball sampling allowed for asking each interviewee if there was someone else who should be interviewed, yielding 14 additional interviewees identified.Additional interviewees were contacted and interviewed until saturation was achieved as evidenced by the lack of new codes during the ongoing rolling coding process occurring at the end of each day of interviews.This study did not focus on participants who are primarily receivers (i.e., administrative staff, clinicians, or other end users) of event notifications.

Data Collection and Analysis
Interviews occurred between March 22 and June 17, 2022 and were conducted by the lead author.The interviews were recorded and transcribed using transcription available on conferencing software (i.e., Zoom, Teams, etc.).Interviews lasted an average of 35 minutes (range of 12-67 minutes), and each interview concluded with member checking to validate key interview takeaways.Information gleaned from previous interviews was not shared during interviews.
The interview guide was developed by members of the research team, based upon knowledge of HIE, DM, event notifications, and the CMS Final Rule.Questions (see ►Supplementary Appendix A: Interview Questions, available in the online version) were asked to better understand perceived barriers and other factors pertaining to event notifications as well as involvement with DM and event notifications with VA.Probing questions were asked if/as warranted to illicit additional information.
The lead author deidentified the interview transcripts from the interviews and checked the transcripts for accuracy by listening to all of the recordings and confirming the transcribed text matched what was said.The lead author used Microsoft Excel to perform thematic, deductive coding by reviewing the interviewees' responses and highlighting categories that emerged and marking each with a 1 to indicate a count of 1 (multiples, such as an interviewee using three identical words in a row to make a point, were counted as 1 and not as 3).The categories were then reviewed to see if any could be combined into major themes.Pivot tables were then used to display counts and concentrations of the comments in each of the final themes.A separate investigator and author reviewed all interview transcripts, confirmed the coding, and assessed for categories and themes (no new categories or themes were identified).These two investigators discussed discrepancies and derived a mutual consensus through discussion.

Results
Of the 34 leaders contacted (see ►Supplementary Appendix B: Interviewee Email Template, available in the online version) by the lead author, 17 were interviewed by the lead author.Of the 17 interviewees (see ►Table 1), 12 (70%) were categorized as participants in care coordination/event notifications encompassing non-VA senders and receivers of event notifications, including EHR vendors, regional and state HIEs, aggregators, HISPs, and non-VA hospital systems.Three (18%) of the interviewees were either from a standards developing organization or were a patient advocate.Lastly, two (12%) of the interviewees were either a national-level VA staff member or a contractor in hopes of identifying barriers to event notifications from a VA perspective.
Coding resulted in six main themes of barriers (see ►Fig. 2) for senders of event notifications using the Direct Standard.Workflow was the primary barrier by an Overall, the data analysis of the interviews also showed that interviewees felt event notifications were beneficial to patients (including Veterans) and their care team members from a coordination of care perspective.Interviewees in leadership positions felt that DM provided the appropriate security and infrastructure to be the preferred transport mechanism for event notifications, but there were various barriers that needed to be overcome for it to sustain and scale.Moreover, even though there continue to be barriers, the industry as a whole was working collaboratively to address and resolve them-but that it would likely take a "multi-pronged approach to get it to work well."One interviewee summed up the words of several, saying: "Veterans are getting more care outside VA.The things that happen outside VA to Veterans in terms of their care, their treatment, their diagnosis, the tests they undergo and so forth is information that they should carry with them back to the VA medical center, where their clinical home is located."Each of the subsections below delve deeper into each of the six barrier-identified themes.

Workflow Barriers: Identifying Provider(s), Identifying Direct Address(es), and Identifying Enrolled Veterans
The primary barrier within the workflow category was identifying a patient's customary provider(s) (n ¼ 10 interviewees).First, interviewees described the inherent flaw of hospitals having to rely on a patient to reliably communicate his/her provider when a patient presents at the hospital.For example, one interviewee commented: "Health care provider organizations generally don't have much infrastructure available to them for locating providers or administrative staff and personnel working with providers' locations and how to reach them.[Hospitals] see themselves as competing, capitalistic, profit-making organizations who have their own vested interest in maintaining their patient populations, their customer base, and the information around that customer base." Another interviewee stated, "You know the fundamental flaw is that you're still assuming that the patient and the registrar get this right." Second, interviewees described that there is a likelihood of hospital registration staff incorrectly capturing information the patient provides (n ¼ 3).This ranged from capturing an incorrect provider, misspelling a provider's name, or selecting the wrong provider with a very common name.Inaccurate or inadequate data capture leads to the inability of, or delays in, identifying providers, provider's Direct addresses, or enrolled Veterans.One interviewee articulated how they were able to proactively curate the information needed to correctly route the event notification: "We've identified [patient information] in advance prior to the event [notification] taking place.We've identified the care team of these patients.When we have the event notification, we send [the event notification] right directly to [the patient's] primary care physician, because we do think it's an opportunity for the primary care physician to actually intervene.And we've added all of those into our electronic health record and made them part of the care team.So, we have some custom code that allows us to send [the event notification] to where we darn well please.But it's not [the EHR's] standard."Third, interviewees said the hospital finding the provider's National Provider Identifier (NPI) number was a barrier (n ¼ 3).This could be as a result of the NPI listed in a directory not being valid or correct, the NPI missing altogether, or only a facility-or group-level NPI included in the directory.This last example would likely prevent a match between an individual provider and the group-or facilitylevel NPI.One interviewee commented: "[I think it would be helpful if] individual NPI numbers exist in the [DirectTrust] Directory.It adds confidence to senders about the recipients' Direct address." Two interviewees described that their respective EHRs only allow Direct addresses to be allocated to users that have a valid NPI.However, the respective EHRs provide the functionality of proxies/authorized delegates for these clinicians so the event notifications can be redirected and processed by nonclinical staff (who do not have a personal NPI).
Within the same category of workflow barriers, the second highest barrier for event notifications was identifying the Direct address of a patient's provider (n ¼ 7) .One interviewee described this barrier as: "It's you trying to match it up to the right provider [  Along the same lines, interviewees described providers' and patients' general lack of awareness of what a provider's Direct address is.
Another barrier identified within the workflow category was the lack of efficiency and accuracy of non-VA hospitals/senders identifying Veterans, and, more specifically, identifying enrolled Veterans during an event notification (n ¼ 2).One non-VA interviewee suggested that non-VA hospitals receive access to "some kind of member roster with VA."However, a VA interviewee expressed concern over the large resource effort, ever-changing status of enrolled Veterans, and complexity of VA enrollment as prohibitive to VA providing a member roster to non-VA facilities.Interview questions did not address Veterans self-identifying when seeking care outside of VA.

Technical Barriers: Payload and Ingestion Variability
Technical barriers, including the lack of a standard payload (i.e., file attachment type such as .PDF) (as dictated by CMS-9115-F), associated with an event notification were identified (n ¼ 3) .
"The biggest challenge is that there has not been until recently [with the creation of the DirectTrust IG] a standard for the payload associated with event notifications… [As a receiver of event notifications, it is about being] able to receive and handle all the various payloads."An interviewee explained further: "Information [in an event notification] may vary from partner to partner depending on if the partner implements the DirectTrust Implementation Guide for Event Notifications, or if they're just sending the minimal necessary information."Another interviewee said there is "not currently any type of flags or data tags that would help to route [Direct] messages."

Content Barrier: Inessential Information
Within the content category, two barriers in particular were highlighted.One was that the receiver only wanted specific event notification content category or categories (n ¼ 2).For example, a cardiologist may only want cardiac-related event notifications.The second barrier within the content category was event notifications not being of value or actionable by the receiver (n ¼ 2).Essentially, receivers of event notifications only want a "gold nugget" of information, not useless or unactionable information."There's a large volume of information coming in and it's difficult to focus what people send to [VA] to be the key event notifications.So, it's just that [with event notifications] the volume is great in many of those [event notifications], are either not needed, go to the wrong facility, or different things.[VA] end users can't distinguish some-times…the value that will come and how it will help them in other parts of their job." Policy/Governance Barriers: Achieving Scalable Governance and Veterans Affairs Non-Participation with CMS Interviewees spoke to needing a scalable governance for event notifications (n ¼ 3).One interviewee described scalable governance as when "event notifications could expand exponentially across different communities, decision-making hierarchies, and methods of communication."Other interviewees spoke to the barrier of VA's non-participation with CMS."There's no requirement that VA receive those [event notification] messages.VA can in effect, block the ability of non-VA facilities to send a completed transaction of an event notification.And you know that's disheartening and difficult.But it also means that VA, in effect, devalues the informational content about the care of those patients that are treated at non-VA facilities."

Cost and Organizational Barriers: Limited Resources and Lack of Awareness and Strong Leadership
One interviewee expressed that a cost barrier of event notifications was limited hospital resources that were already constrained due to EHR implementations/configurations, limited collaboration regarding event notification workflows, and coronavirus disease 2019.Two interviewees suggested strong leadership/a champion within an organization that touts the benefits of, and drives the implementation for, DM and event notifications helps to alleviate these organizational barriers.
"Having a champion is absolutely key [to a successful event notification workflow] and I think that's what probably is the disconnect with most of [the] VA facilities.We haven't been able to find that champion."

Standards Facilitator: DirectTrust Implementation Guide for Event Notifications
While standards can sometimes be construed as a barrier to implementation and participation, interviewees said the creation of the DirectTrust IG for Event Notifications has served as a facilitator of event notifications.In fact, 94% (n ¼ 16) of interviewees were aware of (and some even involved in the creation of) the DirectTrust IG for Event Notifications.Of those 94%, n ¼ 17 were in favor of the DirectTrust IG for Event Notifications.One person summed this up nicely by saying, "We [VA] are too big to have one-off solutions.VA has really gravitated and supported standards and interoperability."

Principal Findings
Overall, interviewees felt event notifications are beneficial to patients and their care team members from a coordination of care perspective.Interviewees felt DM provided the appropriate security and infrastructure to be the preferred transport mechanism for event notifications, but there are still barriers that need to be overcome.Such barriers include: • Identifying the applicable patient's provider, NPI #, Direct address, and receiver of the event notification.• Identifying Veterans, especially Veterans enrolled within VA. • Achieving more consistent payload types and improve ingestion by EHRs.• Communicating the "golden nugget" of information in an event notification.• Reaching a scalable governance that can traverse different modes and communities.
Even though barriers remain, the industry as a whole is working collaboratively to address and resolve them.Moreover, interviewees expressed that there was not a simple answer to addressing all the barriers, but that it would likely take a multipronged approach to getting it to work well.Throughout all the interviews, though, respondents expressed a strong commitment to collaborating with VA so that the millions of enrolled Veterans and their providers can receive the event notifications' benefit of improved care coordination between VA and non-VA providers.
Factors involved in the successful (or, in the worst case scenario, failure of) implementation of HIE, of which event notifications are a subset, include technology, use case/workflow, policy, organizational change management, etc. [32][33][34][35][36] These factors can be further teased apart by barriers/other factors for event notifications identified by receivers (i.e., the provider [or ultimately the patient]) and by senders.In terms of receivers: • Receivers having workflows, resources, and processes in place to handle the event notifications sent (VA and non-VA receivers; multiple transport methods). 18,22,37 Decreasing the negative impact on the receiver's workflow (non-VA; unspecified transport method). 38 Senders including higher quality data in the event notification sent (non-VA; unknown transport method).From this perspective, workflow barriers were also a predominant theme, but the workflow barriers centered around identifying provider(s), identifying Direct address (es), and identifying enrolled Veterans versus prior studies' themes of the presence/absence of the receiver's workflow(s) and negative impact(s) to workflow.Moreover, with the advent of the DirectTrust IG, interviewees' focus in this study was not on deficient information within the event notification.
In terms of senders (e.g., hospitals): • Senders setting up an application programming interface in the sender's EHR to more easily send event notifications to receiving providers (non-VA; not DM as transport method). 39Senders and receivers working collaboratively via master patient index (MPI) matching to target high-utilization patients to prevent costly hospital visits (non-VA; standard Health Level 7 [HL7] admission, discharge, and transfer with MPI matching for transport method). 20• Directing the event notifications to a care manager/care navigator (versus directly to a clinician) and creating groups on the receiving end so multiple recipients receive the event notification (non-VA; unspecified transport method). 20terviewees did not identify these barriers.However, amid the barriers that were identified, it is important to recognize that some of these have been long-standing because they exist due to other governance, technological, or implementation dependencies that are outside the scope of this paper.
Because it shows hope and promise, it is worth noting that during the interviews, interviewees discussed a promising pilot between VA and an aggregator.In the pilot, when a Veteran is admitted to a non-VA hospital, the aggregator uses a state's event notification system to send a patient discovery query through the eHealth Exchange to VA's Joint HIE.This checks to see if the Veteran is an enrolled Veteran.If the Veteran is an enrolled Veteran, then the aggregator is notified that the patient is matched and the aggregator sends a Direct message to VA. VA staff, including the Veteran's VA clinical team (at the Veteran's applicable VA medical center [VAMC]), would receive the event notification thereby improving the Veteran's coordination of care.Today, through eHealth Exchange, an aggregator can determine if an event notification pertains to a VA patient, but cannot identify the applicable VAMC through eHealth Exchange.Interviewees suggested that this pilot shows promise for overcoming many of the event notifications' workflow barriers and represents an area for future research in terms of implementation, evaluation, and associated patient outcomes.Additionally, exploring solutions to the barriers identified, while outside the scope of this paper, is important.Thus, this represents additional research that could contribute to the scalability and sustainability of DM.

Study Limitations
The information gained from this study, while valuable, must be considered among the limitations.First, interviewees included a very limited number of VA staff and non-VA hospital systems.Second, there were no interviewees representing rural hospitals (VA nor non-VA).Third, an overwhelming number of interviewees were aware of, and supportive of, the DirectTrust's IG for Event Notifications, but perhaps hearing the perspective of individuals that are known in the industry as not in favor of DM/DirectTrust's IG for Event Notifications could provide more insight.Fourth, the use of additional investigators involved in the interviews may have led to deeper follow-on questions and richer data.Lastly, this study did not address VA's policy issues or more detailed roles of leadership/organizational management associated with event notifications and represents an area of further research.Additionally, from a more global perspective, this study may have illuminated contributions to the implementation science literature had an implementation science framework been used for implementation and then revisited for the conduct of this study.This study, however, did not take an implementation science perspective and, as such, potentially exposes a limitation.

Conclusion
CMS-9115-F requires hospitals to send event notifications.As highlighted by this study, more work is necessary to develop best practices for workflows surrounding event notifications from non-VA hospitals to VA hospitals.Moreover, technical details lacking in the CMS Final Rule have been provided in the DirectTrust Implementation Guide for Event Notifications and perhaps could be provided in the DirectTrust Directory.As hospitals use the DirectTrust Implementation Guide to implement event notifications, future research is necessary to understand and guide the adoption and use of event notifications as well as make it efficient in hopes of achieving better health outcomes for Veterans.

Clinical Relevance Statement
Overcoming barriers to senders of event notifications, VA could more rapidly and more efficiently expand receiving event notifications at a national level to maximize Veterans' coordination of care with community health care providers.

Protection of Human and Animal Subjects
The study was reviewed by the University of Alabama at Birmingham Institutional Review Board.

Fig. 1
Fig.1Summary of CMS Final Rule for event notifications.25

Table 1
Interviewee categories and count of interviewees National-level VA staff member/contractor 2 approximate 4 to 1 margin (32:7) of total comments by interviewees (12) than the next barrier, which was technical.
as] step one.Secondly, finding the Direct address if it's in [a directory].And then, thirdly, is [the Direct address in a directory] a valid Direct address?The hardest thing is to maintain those directories after you get them started.Directory roster, maintenance, availability and discoverability [are] hard to navigate." 18• Senders including more information in the event notification sent (e.g., next steps and diagnosis; VA and non-VA; DM and other transport methods). 18,22,38 Receiver's EHRs better integrating event notifications into the receiver's workflow (VA; DM as transport method).18