The Effect of Electronic Health Record and Immunization Information System Interoperability on Medical Practice Vaccination Workflow

Keywords

vaccination - workflow - electronic health records - registries - health information exchange

Background and Significance

A dramatic increase in the adoption of electronic health record (EHR) systems has taken place over the past decade,[1] leading to integration throughout patient workflows and substantial levels of electronic health information exchange between health care providers.[2] [3] [4] An excellent example of this is illustrated by medical practices' activities surrounding the administration of vaccinations, an essential function of primary care and public health practice. Electronic data exchange (also known as interoperability) between EHRs and immunization information systems (IIS) has been demonstrated to have positive impacts on data quality such as the timeliness, completeness, and accuracy of these data.[5] [6] [7] [8] These benefits became especially important during the coronavirus disease 2019 (COVID-19) pandemic response in primary care and public health settings.[7] [9] [10] [11] [12] [13]

Potential benefits of EHR/IIS interoperability may be realized by either the sending or receiving organization or both, depending on whether unidirectional or bidirectional exchanges are taking place. EHR/IIS interoperability is accomplished through messages that conform to the HL7 messaging standard.[14] In unidirectional exchange, data are shared in only one direction; an immunization provider's EHR electronically transmits data on vaccine doses administered to the IIS, often in real time. The EHR sends messages to report dose-level details to an IIS for each vaccine administered along with current patient (or responsible party) contact information from patient demographic and administrative records. In bidirectional immunization messaging, data are shared in both directions, from an EHR to the IIS, and vice versa. In addition to electronic transfer of vaccine administered data from the EHR to the IIS, an EHR can “query” the IIS in real time. A query is sent by the EHR to the IIS, and the IIS returns the patient's full vaccination history to the EHR, including data for vaccines administered at other sites. Bidirectional interoperability allows practices' EHRs to receive all available dose-level information from an IIS for a given patient, as well as the patient's vaccination status (up to date, eligible, overdue), and recommended vaccines. These queries may be automatically triggered as patients schedule visits, check in at a clinic, or as staff review clinic vaccination records.[15] Immunization providers whose EHR is not interoperable with the IIS through the HL7 standard (i.e., non-HL7 practices) report vaccine doses administered either through periodic transfers of an electronic file from the EHR to the IIS or direct data entry in the IIS's web portal. The latter method typically entails double data entry, once into the EHR and again in the IIS.

EHR/IIS interoperability is well established in many jurisdictions, which was largely initiated in response to the incentives established under the federal Meaningful Use program in 2009.[16] In Michigan, 48% of active vaccination sites currently report to the statewide IIS through either HL7 unidirectional (9%) or bidirectional (39%) interoperability (unpublished data, 2024). Expansion of interoperability continues to be a work in progress, and onboarding of new HL7 connections continues nationally.[17] [18] An important consideration for practices adopting HL7 messaging with an IIS is the extent to which this interoperability may influence the day-to-day vaccination workflow and related recordkeeping tasks performed by clinical and administrative staff. Vaccination-related information may be available to staff throughout the workflow from either the practice's EHR system or the jurisdiction's IIS, or both; it is not clear which mechanism staff may use, when they do so, and why. Although EHR/IIS interoperability may be anticipated to positively impact data quality, it is unclear whether it may also impact the vaccination workflow of practice staff.

With that in mind, the objective of this study was to evaluate how medical practices use their EHR and IIS to support vaccination-related patient encounters and how EHR/IIS interoperability may impact clinical and administrative functions throughout the vaccination workflow. To our knowledge, this information has not been previously available and is relevant for practices seeking to establish or improve their processes related to interoperability, as well as for IIS staff that work with immunization providers to onboard and maintain interoperable connections.

Methods

To gain a more comprehensive understanding of the potential impacts of EHR/IIS interoperability on the vaccination workflow at the practice level, we conducted semi-structured interviews with medical practices reporting to the Michigan Care Improvement Registry (MCIR), the statewide IIS.[19] [20] Vaccination providers are required to report all doses administered to persons <20 years of age to MCIR; although not required, Michigan law also authorizes the reporting of doses administered to adults.[18] All births are automatically reported to MCIR from state records; very few parents elect to opt children (<0.3%) out of participation. This study was conducted September 1, 2018 to August 31, 2022 and was determined to not be regulated by the University of Michigan Institutional Review Board (HUM00157276). [Fig. 1] provides a schematic of a generalized vaccination workflow model, which we used as a framework for data collection, coding, and reporting of findings; the model built upon an existing framework, and other models have been more recently developed.[2] [21] [22] [23]

Fig. 1 Vaccination workflow overview. EHR, electronic health record; IIS, immunization information systems.

It should be noted that this study was initially designed prior to the COVID-19 pandemic to explore potential benefits and costs of interoperability to practices through onsite observations. However, with the onset of the COVID-19 pandemic, limitations on physical access to medical practices made such observations infeasible and required an alternate approach to evaluate interoperability impacts on vaccination practices. We had conducted an initial semi-structured interview with practices before the onset of COVID-19 (Phase 1) to gauge aspects of EHR/IIS use relative to vaccination workflow with the ultimate intent of conducting onsite observations of the staff time required to perform these tasks. Given constraints precluding onsite observations at practices during the pandemic, we elected to focus the remainder of the study on characterizing vaccination workflow at practices and how staff used and recorded both EHR and IIS information. We employed a subsequent interview (Phase 2) following the initial acute phase of the pandemic to explore any changes to the vaccination workflow related to EHR/IIS interoperability among the practices who participated in the initial phase of the study.

Sampling Frame

Construction of the sampling frame began with identifying all sites designated in MCIR as being “active” in 2018, which resulted in over 4,500 unique practice locations. This list was then refined to focus on the practice types of interest: family medicine, pediatrics, internal medicine, and local health departments. Retail pharmacies and hospitals were excluded, as were practices that reported fewer than 120 doses during 2018, resulting in a sampling frame of 1,892 practice sites.

Given the project resources available, we identified an interview completion target of approximately 50 practices. Based on review of the sampling frame, augmented by recommendations from coordinators for each of the state's six MCIR regions, we sought to define a quasi-purposeful, convenience sample of potential interview participants. We aimed to have representation across the practice types of interest, interoperability levels, practice size (using the number of vaccine doses administered annually), and geographic regions. Overall, we identified a total of 111 candidate practices for recruitment as interview participants.

Sample Outreach

The 111 candidate practices were contacted by the project lead (K.J.D.) via email to request their participation in this study with subsequent contacts by telephone, as necessary. Those that agreed to participate identified the best person(s) at the practice site for addressing questions related to the vaccination workflow, related recordkeeping, and MCIR reporting. The interviewer (A.E.C.) contacted the designated practice contact(s) by email to schedule an initial (Phase 1) telephone interview. For Phase 2, a similar process was followed for requesting participation and scheduling the interview among the subset of practices that participated in Phase 1. To assist with retention of Phase 1 practice sites, a $25 gift card was offered as an incentive for completing the Phase 2 interview. When an interview was scheduled, a list of discussion topics was sent via email to the interviewee(s).

Interview Guides

Based on the extensive IIS experience of the project lead, we developed questions to capture information for each stage of our vaccination workflow model ([Fig. 1]), with probes to explore how each step is done and whether any relevant barriers exist. The Phase 1 interview guide was pilot tested with a convenience sample of five practices; Phase 2 interview questions were based on the Phase 1 interview guide and focused on identifying any changes made to the vaccine recordkeeping workflow since the Phase 1 interview (see [Supplementary Appendix 1], Phase 1 and Phase 2 interview guides [available in the online version]).

Interview Administration and Processing

The interviews, which were typically about 30 minutes in duration, were conducted March 4 to December 2, 2019 (Phase 1) and September 29, 2020 to December 22, 2021 (Phase 2) by telephone (Phase 1, Phase 2) or Zoom audio call (Phase 2). Interviewees were typically office/practice managers or the lead clinical staff person for vaccine-related processes, including MCIR reporting. At the start of the interview, the interviewer obtained permission from all interviewees to record the interview and enable subsequent transcription.

Interview recordings were transcribed verbatim. In preparation for coding the Phase 1 interview transcripts, we developed an initial codebook a priori to capture the main themes for each of the vaccination recordkeeping workflow categories. Codes were assigned by reading through the interview transcripts multiple times and relevant text excerpts were recorded in Excel; all transcripts and coding were also reviewed by the interviewer (A.E.C.). The codebook was iteratively revised during the coding process to include additional themes. For Phase 2, the Phase 1 codebook was adapted by adding categories to reflect new topics. Questions regarding coding were reviewed among the project team to reach consensus. Based on the assigned coded themes, a series of dichotomous indicators were developed in each workflow category to indicate the presence or absence of specific practice characteristics or workflow activities pertaining to EHR and MCIR use. Indicators were summed across responses to derive counts of practices citing each attribute. Practices did not necessarily discuss each question from the interview guide and consequently, so our reported results are based on varying numbers of respondents.

Results

Among the candidate sites identified (n = 111), 45 practices completed interviews during Phase 1, 42 of which also completed a Phase 2 interview. The remaining results presented here reflect the 42 practices that participated in both interviews. For 36 of the 42 sites (86%), the same person(s) was interviewed in both phases.

Participating practices were primarily family medicine or pediatrics ([Table 1]). All practices (n = 42) reported using MCIR throughout the vaccination workflow process for reviewing and recording patient- and vaccine-related information. During Phase 1, most participating practices were either unidirectional (n = 28) or bidirectional HL7 (n = 10) with MCIR; the remaining practices (n = 4) were not HL7-interoperable and only accessed MCIR manually through the web user interface. HL7 connectivity to MCIR expanded among participating practices during the study period such that by Phase 2, all practices were connected to MCIR either by unidirectional HL7 (n = 26) or bidirectional HL7 (n = 16) messaging. Practices ranged in size by annual vaccination doses administered and were located in counties that included metropolitan, nonmetropolitan, and rural areas. We identified 13 various EHR system vendors used by practices, with the most common being Epic, eClinicalWorks, Allscripts, Athenahealth, and Patagonia Health.

Table 1 Characteristics of practices interviewed (n = 42)

Practice specialty
Pediatrics	18 (43%)
Family medicine	16 (38%)
Local health department	6 (14%)
Internal medicine	2 (5%)
Volume of vaccine doses administered (2018)
≥10,000	14 (33%)
5,000–9,999	10 (24%)
<5,000	18 (43%)
Urban/rural classification of county[a]
Metropolitan (>50,000 persons)	26 (62%)
Micropolitan (10,000–50,000)	12 (28%)
Rural (<10,000)	4 (10%)
Phase 1 HL7 interoperability
Unidirectional	28 (67%)
Bidirectional	10 (24%)
No HL7 interoperability	4 (9%)
Phase 2 HL7 interoperability
Unidirectional	26 (62%)
Bidirectional	16 (38%)
No HL7 interoperability	0 (0%)
Electronic health record (EHR) system vendor[b]
Epic	9 (21%)
eClinicalWorks	7 (17%)
Allscripts	4 (10%)
Athenahealth	4 (10%)
Patagonia Health	4 (10%)
NextGen	3 (7%)
Office Practicum	3 (7%)
GE Centricity	2 (5%)
McKesson	2 (5%)
AdvancedMD	1 (2%)
eMDs	1 (2%)
PCC	1 (2%)
Practice fusion	1 (2%)

^a NCHS Urban–Rural Classification Scheme for Counties, 2013; https://www.cdc.gov/nchs/data/series/sr_02/sr02_166.pdf.

^b EHR in use during Phase 2.

All the subsections of section Results provide detailed results following the workflow model provided in [Fig. 1]. As noted previously, these responses were captured through semi-structured interviews and as such, not all respondents necessarily addressed each topic, leading to varying number of responses for each item.

Patient Information Collection

Most practices (n = 24 of 40) noted that patient information, such as contact and insurance information, is collected in advance of a scheduled appointment versus at the appointment time ([Table 2]). Nearly all practices (n = 37 of 40) update patient information in the EHR at the time the information is collected; in addition, other sites update the EHR either during the visit (n = 1) or before patient checkout (n = 1). Several practices (n = 13 of 35) noted that they verify patient insurance information via either a third-party system or a specialized department within the clinic; a few (n = 7) noted including a scanned copy of the patient's insurance card in their EHR. Several practices (n = 13 of 37) mentioned using a paper registration form during the process of obtaining patient information in addition to entering patient information into their EHR. Other practices described a variety of electronic methods to assist with collecting this information, such as tablets for onsite patient registration or check-in, and patient portals.

Tables 2 Vaccination workflow data collection, review, and reconciliation

a. Patient information	Uni	Bi	Total
Collected in advance of appointment (vs. at appointment) (n = 40)	12	12	24
Entered into electronic health record (EHR) system (n = 40)	23	16	39
Insurance information verified (n = 35)	5	8	13
Insurance information scanned (n = 35)	5	2	7
Use paper registration form (n = 37)	9	4	13
Use tablet or other electronic check-in devices (n = 22)	8	4	12
Use patient portal to collect information (n = 40)	1	5	6
b. Vaccination needs assessment	Uni	Bi	Total
Primary vaccine history source (n = 42)
MCIR	23	14	37
EHR system	2	1	3
Both MCIR and EHR	1	1	2
Difficulty finding patient records noted (n = 39)	21	16	37
Dose reconciliation (n = 42)
Reconcile between MCIR and EHR	20	14	34
Difficulties noted with point of care reconciliation (n = 16)	5	9	14
Assessment timing (n = 42)
Before appointment	18	9	27
At appointment	8	7	15
For all visits (vs. only well visits) (n = 27)	10	10	20
Use of paper MCIR
Print paper report from MCIR (n = 42)	25	14	39
Mark paper MCIR report for doses to administer (n = 38)	21	8	29
c. Vaccine administration	Uni	Bi	Total
Preparing vaccines for administration (n = 38)
Indicate vaccine orders in EHR	15	11	26
Indicate orders on paper forms	10	5	15
Quality check for vaccine orders
Orders reviewed, self-check or with others (n = 39)	10	7	17
Checked against paper forms (n = 40)	15	4	19
Timing of orders (n = 35)
Standing orders for some vaccines (n = 38)	10	3	13
Enter pending orders prior to discussion with patient	3	4	7
Enter orders after discussion with patient	19	10	29
Method of data entry into EHR for doses administered (n = 33)
Manual (keyboard)	17	11	28
Barcode scanner	2	4	6
Vaccine lot number autopopulates	5	4	9
Timing of data entry for doses administered (n = 42)
Prior to administration	5	7	12
Following administration	21	9	30
Refusals
Enter into EHR (n = 39)	23	14	37
Scan signed refusal form in EHR (n = 39)	13	6	19
Manually enter into MCIR (n = 25)	6	5	11
Automatically transfers to MCIR (n = 25)	3	4	7
d. Data transfer to IIS	Uni	Bi	Total
Timing (n = 40)
Immediately following close of EHR record	21	13	34
Scheduled transfer	2	1	3
Review of transferred doses
Use MCIR transfer report (n = 39)	23	15	38
Use EHR report (n = 38)	8	4	12
Use other data, such as logs and/or inventory counts (n = 39)	9	5	14
Frequency (n = 41)
Daily	14	8	22
Weekly	12	6	18
Monthly	0	1	1
e. Inventory management	Uni	Bi	Total
Has vaccines for children (VFC) public stock (n = 42)	22	15	37
Enter public stock information into EHR (n = 37)	17	10	27
Use only MCIR for public stock inventory tasks (n = 37)	7	9	16
Use MCIR and EHR for public stock inventory tasks (n = 37)	15	6	21
Has private stock (n = 42)	26	16	42
Enter private stock information into MCIR	24	15	39
Use only MCIR for private stock inventory tasks	8	9	17
Use only EHR for private stock inventory tasks	2	0	2
Use MCIR and EHR for private stock inventory tasks	16	6	22
Neither EHR nor MCIR used for private stock inventory tasks	0	1	1

Abbreviations: Bi, bidirectional; EHR, electronic health record; MCIR, Michigan Care Improvement Registry; Uni, unidirectional.

Vaccine Needs Assessment

In the first step of the vaccine needs assessment process, staff pull and review a patient's vaccine history ([Table 2]). Most practices (n = 37 of 42) said they rely upon MCIR as their primary source of accurate and reliable vaccine history information; others stated they either use their EHR as the primary source (n = 3) or used both systems (n = 2). Among practices using MCIR (n = 39), almost all reported having occasional difficulty finding patients in MCIR (e.g., name discrepancies).

Once vaccine history information is pulled, practices typically compare data between MCIR and the practice's EHR (n = 34 of 42), and any necessary adjustments are made so that information agrees between the two systems (i.e., dose reconciliation). Practices using bidirectional HL7 data exchange can electronically query MCIR via their EHR in real-time, review the doses from each system, discard duplicates, and import those found solely in MCIR. Some practices (n = 14 of 16) noted difficulties with staff reconciling doses at the point of care, including the time-consuming nature of this task, distinguishing the sequence of doses in a vaccine series between the two systems, and dealing with the volume of duplicate doses; the other two practices reported having no difficulties.

About two-thirds of practices (n = 27 of 42) reported doing vaccine needs assessment prior to the scheduled appointment (typically the morning of or the night before), with the balance (n = 15 of 42) completing it at each patient's appointment. Many practices (n = 20 of 27) reported conducting these assessments for all visits, not just well/preventive care visits.

Almost all practices (n = 39 of 42) print paper MCIR forms for all patients identified in their EHR as having upcoming appointments. This process typically involves manually querying MCIR for an individual patient, or for batches of patients with upcoming appointments, and then printing the individual MCIR summary record. Many practices (n = 29 of 38) highlight, circle, or otherwise indicate vaccinations for which a patient is due on the hard copy MCIR forms; in some cases, these marked-up forms are given to the patient at the end of the visit. Practices gave several reasons for continued reliance on paper MCIR forms, including: the forms provide a paper trail for the appointment, provider preferences for hard copy rather than the EHR screen, and the clear format of the MCIR forms indicating when each vaccine is due.

Vaccine Administration

The majority of practices (n = 26 of 38) document, or “order,” the specific vaccines to be administered electronically in their EHR ([Table 2]). Several practices described using paper forms for vaccine orders (either exclusively or in combination with EHR orders), including handwriting orders on patients' paper MCIR forms (n = 10) or a different hard copy method (e.g., encounter form, consent form, n = 5). The timing of the vaccine orders varies; most commonly, practices enter vaccine orders after discussing the immunizations with patients (n = 29 of 35). The majority of practices (n = 29 of 41) reported steps related to quality control to ensure accountability and reduce vaccine administration errors. Practices described verifying the orders by comparing vaccines against MCIR or EHR records, either via self-check by clinical staff (n = 8) or with another staff member (n = 8), or both methods (n = 1). Several practices (n = 19) also described documenting vaccine information, such as with handwritten notes or stickers from prefilled syringes, on hardcopy forms such as practice-specific vaccine logs, patients' paper MCIR forms, or signed patient consent forms.

Once administered, details for each vaccine dose (e.g., lot number, manufacturer, site of administration) are entered by all practices into their EHR. Among practices that addressed how information is entered, most (n = 28 of 33) reported manual data entry and a few (n = 6) use barcode scanners to automate data entry, including one practice that uses barcode scanners for some vaccines and manual entry for others. Some practices (n = 9) mentioned that selection or entry of the vaccine lot number will autopopulate other fields associated with the vaccine. Most practices (n = 30 of 42) enter all the vaccination information immediately after administration. Other practices (n = 12 of 42) allow most of the vaccine dose details to be entered in advance as the vaccines are being prepared; data such as the site of administration may be entered following administration. Vaccine refusals, when patients (or parents) decline vaccine doses, are also documented in practices' EHR (n = 37); many practices (n = 19) also scan signed refusal forms into their EHR.

Data Transfer

Almost all practices (n = 34 of 40) indicated that data on vaccines administered are electronically transferred to the IIS once an individual's dose-level data are complete in the EHR ([Table 2]). Other sites (n = 3) scheduled their data transfer at periodic intervals or set times of the day. All practices discussed their process for resolving any issues associated with immunization data transfer between their EHR and MCIR. Most practices (n = 38 of 39) utilize a data transfer report available in MCIR as the main source of information for this task; this report summarizes total immunizations processed, providing details on individual transfer errors and doses that were not successfully deducted from inventory. Several practices supplement the MCIR transfer report with data generated by their EHR (n = 11), whereas one practice uses only data from its EHR. Additional sources of information that practices use in the data transfer review process included paper logs, individual paper MCIR or consent forms, and/or physical inventory counts. Frequency of review was most commonly daily but ranged from daily to monthly. When practices identify issues with data transfer between their EHR and MCIR, they are investigated and resolved. Resolving transfer problems promptly was deemed to be important due to the link between doses administered data and inventory balancing. For the practices that adopted HL7 interoperability with MCIR over the course of the study, a learning curve was reported regarding this review process. One practice noted that although time was saved by eliminating double data entry, more time may be ultimately spent correcting data transfer errors, which had previously been identified during the data entry process.

Inventory Management

Product details for each vaccine (e.g., manufacturer, lot number, expiration date) ordered are entered into the relevant vaccine inventory management system. For public stock, this information is pre-entered into MCIR by Michigan Department of Health and Human Services staff. Consequently, practices participating in the Vaccines for Children (VFC) program (n = 37 of 42) only need to double check that their public vaccine shipments match the shipping invoice and what is already entered in MCIR ([Table 2]). Many of these practices also enter at least some information for the public vaccine that they receive into their EHR (n = 27 of 37). All 42 practices have private stock, and almost all practices enter the vaccine details into MCIR (n = 39 of 42). Most practices manually enter vaccine information (other than public stock doses pre-entered in MCIR), although a few reported using utilize barcode scanners.

The vaccine inventory balancing process involves counting physical vaccine inventory and comparing this information to reports of doses administered. Supporting documentation used for inventory balancing can include MCIR-generated reports, EHR-generated reports, vaccine borrowing or wastage logs, and other hard copy documentation. All VFC practices are required to use MCIR inventory functions to manage both their public and private vaccine stocks to assure borrowing transactions are complete.[24] This process requires reconciling counts of their physical stock with the inventory tracked in MCIR at least monthly prior to placing a new order. Over half of the VFC-participating practices in this study (n = 21 of 37) also use their EHR to help manage their vaccine inventory, while the remaining VFC practices (n = 16) exclusively use MCIR's inventory management functionality. Among the 42 practices with private stock, over half (n = 22) use both their EHR and MCIR to manage vaccine inventory, 17 use only MCIR, 2 use only their EHR, and one doesn't use any electronic system for inventory management.

Discussion

Our findings provide a novel perspective on how providers use EHRs in conjunction with the statewide IIS in Michigan to support vaccination encounters. We found that while practices reported that adopting EHR/IIS interoperability may offer benefits to the vaccination workflow, some challenges may likewise be introduced. To our knowledge, this study is the first assessment of how medical practices employ these resources throughout vaccination encounters and illustrates potential opportunities to streamline vaccination reconciliation processes.

We found that practices employ both their EHRs and MCIR throughout the vaccination workflow and may call upon either to address information and documentation needs. IIS and EHRs have several functions in common to support vaccination workflow. Clinical decision support for immunizations (often referred to as vaccination forecasting) is an important example,[25] [26] [27] where practices may have the option of assessing patients' eligibility for vaccination using either, or both, systems. Our findings indicate that practices typically rely heavily on MCIR for vaccination assessment, with nearly all practices printing MCIR vaccination history and forecast reports prior to patient encounters. While practices reported a strong reliance on the MCIR vaccination forecast, it is not clear whether this preference is due to it being viewed as the authoritative source or being simply a more familiar source. Although some EHRs may not currently support forecasting as robustly as IIS, it is important to note that open-source forecasting software is freely available for use by EHRs and other systems.[26] [28]

Employing an EHR for forecasting is predicated on having a complete vaccination history, which is realized via reconciliation of vaccine doses between the EHR and IIS.[29] The dose reconciliation process is facilitated by interoperability between EHRs and IIS, particularly bidirectional HL7, but still may require considerable manual review. In our study, practices reported the time consuming nature of dose reconciliation, highlighting an opportunity for workflow improvement.[30] Unintended consequences of EHR/IIS interoperability such as duplication of dose records have also been reported in other studies.[6] [22] [31] [32] [33] However, there are indications from other sources, in addition to our study, suggesting that integration of vaccination information from the IIS into the EHR is valued by practices and supports clinical decision-making. For example, high rates of satisfaction have been reported among primary care providers viewing immunization data through their EHR.[34] Other reports demonstrate how EHR/IIS interoperability can play a role in EHR-based patient alerts[35] and positively impact vaccination rates.[36]

Regardless of the level of interoperability, practices reported the use of paper-based information throughout the entirety of the vaccination workflow, as either the primary source of information or a supplement to several electronic mechanisms. Workflows for a vaccination encounter typically begin with verification of key patient information, including the patient's name, address, insurance status, and any other key information required to provide immunizations, identify the correct vaccine stock (public vs. private), and bill for services. Although practices reported the use of electronic means these functions, the collection of this information is often a paper-intensive process. Ultimately, this information will be electronically transferred from the EHR to MCIR, but only if a vaccine is administered to the patient during the encounter. This electronic reporting was a new functionality for practices that established HL7 connectivity during the study; these practices were previously required to document patient information changes in their EHR as well as MCIR, creating the need for duplicate data entry. Practices also commonly used paper MCIR vaccination history and forecast reports as a tracking sheet to record administered dose information and other pertinent information, including dose lot numbers or patient refusals of recommended vaccines.

While the use of MCIR for inventory management among VFC-participating practices is mandatory, we found that over half of those practices also utilized their EHRs for inventory-related tasks. Although seemingly duplicative, this may in part reflect the need for vaccine-specific data (e.g., lot numbers) in EHRs to support related functionality, including facilitation of vaccination orders or data entry for vaccines administered. Some practices reported potentially time-consuming reconciliation that may be necessary when incorrect dose-level information is transferred and inventory balancing is incomplete, suggesting an opportunity for training staff on best practice recommendations.[37] It is important to note that interoperability between EHRs and MCIR does not currently include the ability to electronically communicate vaccine shipment details between MCIR and EHRs, so practices that want private stock information in both systems must double enter it.

There are limitations to our reported findings, which reflect the responses of a convenience sample of primary care and public health practices that administer vaccines in Michigan. It is not known how these practices may differ from those more broadly in Michigan or other states. However, the vaccination workflows reported here are similar to those reported in other settings,[2] [21] [22] [23] and the core functionality for patient identification, vaccination needs assessment/forecasting, inventory management, and interoperability follows national best practices and is consistent with that found in other IIS.[15] [38] Future studies may benefit from offering incentives to encourage participation among a broader sample of practices, including in other states. Because data were obtained through a semi-structured interview, not all questions were addressed by each practice. To build on this exploratory study, future studies could also collect data using a more systematic method and delve deeper into issues that may impact interoperable functionality, such as EHR product used. Finally, these data were collected during the COVID-19 pandemic, which impacted the design of our study and could have impacted its findings. However, we expect the impact to be minimal, as practices did not report any major changes to their vaccination-related workflow processes due to the pandemic between Phases 1 and 2.

Conclusion

Practices employed both their EHRs and IIS throughout the entire vaccination workflow, although the use of each relied heavily on paper-based processes. While benefits of adopting EHR/IIS interoperability were reported by practices, this may require staff to learn and implement new workflow processes that can be time consuming and may introduce new challenges. Strategies to establish and disseminate best practices to sites adopting EHR/IIS interoperability should be explored.

Clinical Relevance

We believe that these findings have potential implications for IIS programs, primary care, and public health practices that administer vaccines:

• Electronic interoperability between practices' EHR systems and an IIS may have potential impacts on data quality in either system, as well as staff time. Best practices are needed to guide the efficient reconciliation of EHR and IIS information for doses queried from IIS by EHRs as well as the preferred source for vaccination forecasts, given that these systems may furnish conflicting information.

• Potentially time-consuming reconciliation may be necessary when vaccination practices attempt to electronically transfer data for administered doses to the IIS containing incorrect vaccine lot numbers or other dose-level information. Opportunities for training staff on best practice recommendations, such as frequent reviews of dose transfer logs, may help alleviate inventory balancing problems and ensure completeness of data transfers.

• While electronic interoperability was ultimately adopted by all vaccination practices, manual procedures and paper-based processes persist throughout the inventory management process. Paper reports, such as shipping summaries of vaccine stock received, and summaries of doses administered are commonly used to reconcile inventory records. Such reports may remain essential resources for staff to balance stocks and resolve discrepancies between electronic systems. Opportunities may exist to reduce manual vaccine stock data entry though expanded adoption of barcode scanners.

Multiple-Choice Questions

1. In unidirectional HL7 vaccination message reporting to an IIS, EHRs send:

   • a record of all prescribed medications

   • a complete patient billing record

   • all appointments currently scheduled

   • dose-level vaccination details

   Correct Answer: The correct answer is option d. This is correct because unidirectional exchanges are where immunization practices send information from their EHR to the immunization registry in the form of an HL7 message that contains dose-level details for each vaccination administered. The other responses are not correct because the other types of patient information referenced are not reported by an EHR to an IIS.

2. In bidirectional HL7 vaccination messaging, a query is initiated by:

   • SMS text messaging

   • the medical practice's EHR

   • the medical practice's inventory management system

   • a barcode scanner

   Correct Answer: The correct answer is option b. This is correct because in bidirectional exchanges a query is sent by the EHR, and the patient's full vaccination history is received from the IIS. The other responses are not correct because the other types of technology are not used in HL7 vaccination messaging.

3. Practices were found to use both EHRs and IIS throughout the vaccination workflow in conjunction with

   • cell phones

   • fax machines

   • paper-based processes

   • electronic pagers

   Correct Answer: The correct answer is option c. This is correct because practices reported using paper registration forms, paper MCIR forms, paper consent forms, and paper documentation. The other responses are not correct because the other types of technology were not reported by practice staff as being used in conjunction with EHRs and IIS throughout the vaccination workflow.

• References

• 1 Office of the National Coordinator for Health Information Technology. Office-Based Physician Electronic Health Record Adoption, Health IT Quick-Stat# 50. Accessed August 8, 2023 at: https://www.healthit.gov/data/quickstats/office-based-physician-electronic-health-record-adoption
• 2 Immunizations and Health Information Technology. About the Workflows. Accessed August 8, 2023 at: http://www.immunizationsandhealthit.org/about-workflows/
• 3 Lowry SZ, Ramaiah M, Patterson ES. et al. Integrating electronic health records into clinical workflow: an application of human factors modeling methods to ambulatory care. Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care 2014; 3 (01) 170-177
• 4 Pylypchuk Y, Jordan Everson DC, Patel V. Interoperability Among Office-Based Physicians in 2019. July 2022. Accessed August 8, 2023 at: https://www.healthit.gov/sites/default/files/2022-07/Interoperability_Among_Office-Based_Physicians_in_2019.pdf
• 5 Birkhead GS, Klompas M, Shah NR. Uses of electronic health records for public health surveillance to advance public health. Annu Rev Public Health 2015; 36: 345-359
• 6 Woinarowicz M, Howell M. The impact of electronic health record (EHR) interoperability on immunization information system (IIS) data quality. Online J Public Health Inform 2016; 8 (02) e184
• 7 Greene K, Huber K, McClellan M. Improving Immunization Information Sharing to Support Targeted COVID-19 Vaccination Outreach. July 2021. Accessed August 8, 2023 at: https://healthpolicy.duke.edu/publications/improving-immunization-information-sharing-support-targeted-covid-19-vaccination
• 8 Centers for Disease Control and Prevention (CDC). Progress in immunization information systems—United States, 2009. MMWR Morb Mortal Wkly Rep 2011; 60 (01) 10-12
• 9 Shen AK, Sobczyk EA, Coyle R, Tirmal A, Hannan C. How ready was the US vaccination infrastructure and network of immunization information systems for COVID-19 vaccination campaigns: Recommendations to strengthen the routine vaccination program and prepare for the next pandemic. Hum Vaccin Immunother 2022; 18 (05) 2088010
• 10 American Immunization Registry Association. IIS Insights Shared at AIRA's August 2021 National Meeting, COVID-19 Interim Debrief Summary. August 2022. Accessed August 8, 2023 at: https://repository.immregistries.org/resource/iis-insights-shared-at-aira-s-august-2021-national-meeting/
• 11 Lenert LA, Ding W, Jacobs J. Informatics for public health and health system collaboration: Applications for the control of the current COVID-19 pandemic and the next one. J Am Med Inform Assoc 2021; 28 (08) 1807-1811
• 12 Benjamin-Chung J, Reingold A. Measuring the Success of the US COVID-19 Vaccine Campaign-It's Time to Invest in and Strengthen Immunization Information Systems. Am J Public Health 2021; 111 (06) 1078-1080
• 13 Madhavan S, Bastarache L, Brown JS. et al. Use of electronic health records to support a public health response to the COVID-19 pandemic in the United States: a perspective from 15 academic medical centers. J Am Med Inform Assoc 2021; 28 (02) 393-401
• 14 Health Level Seven International. Introduction to HL7 Standards. Accessed July 2, 2024 at: https://www.hl7.org/implement/standards/index.cfm
• 15 CDC. IIS Health Level 7 (HL7) Implementation Guides. Accessed August 2, 2023 at: https://www.cdc.gov/vaccines/programs/iis/technical-guidance/hl7.html
• 16 US Department of Health and Human Services. Health information technology: initial set of standards, implementation specifications, and certification criteria for electronic health record technology. Fed Regist 2010; 75 (08) 13
• 17 American Immunization Registry Association. Onboarding Consensus-Based Recommendations. November 2018. Accessed August 8, 2023 at: https://repository.immregistries.org/resource/onboarding-consensus-based-recommendations/
• 18 American Immunization Registry Association. Onboarding Shared Services Program. Accessed August 2, 2023 at: https://www.immregistries.org/onboarding-shared-services-program
• 19 Michigan Care Improvement Registry. Accessed August 21, 2023 at: https://mcir.org/
• 20 Michigan Care Improvement Registry. MCIR Reporting Rules. Accessed August 21, 2023 at: https://mcir.org/2020/05/01/mcir-reporting-rules/
• 21 CDC. Workflow Determination Tool for Vaccine Administration. Accessed August 21, 2023 at: https://www.cdc.gov/vaccines/programs/iis/2d-barcodes/downloads/toolkit-workflow-determination-tool-508.pdf
• 22 CDC. AIRA, HIMSS, Drummond. Immunization Integration Program: Immunization-related Capabilities and Guidance. March 2023. Accessed August 21, 2023 at: https://www.himss.org/sites/hde/files/iip-testing-recognition-capabilities-guidance.pdf
• 23 CDC. Vaccine Administration. Accessed September 8, 2023 at: https://www.cdc.gov/vaccines/hcp/admin/admin-protocols.html
• 24 Michigan Department of Health & Human Services – Division of Immunization. Michigan Vaccines for Children (VFC) Provider Manual. 2023 . Accessed September 8, 2023 at: https://www.michigan.gov/-/media/Project/Websites/mdhhs/Folder2/Folder95/Folder1/Folder195/Michigan_VFC_Provider_Manual.pdf?rev=3fd7dbe53be1498a8f2b8bfa1f875a98
• 25 Rajamani S, Bieringer A, Wallerius S, Jensen D, Winden T, Muscoplat MH. Direct and electronic health record access to the clinical decision support for immunizations in the Minnesota immunization information system. Biomed Inform Insights 2016; 8 (Suppl. 02) 23-29
• 26 Arzt NH. Clinical decision support for immunizations (CDSi): a comprehensive, collaborative strategy. Biomed Inform Insights 2016; 8 (Suppl. 02) 1-13
• 27 CDC. Clinical Decision Support for Immunization (CDSi). Accessed September 8, 2023 at: https://www.cdc.gov/vaccines/programs/iis/downloads/CDSi-miniGuide.pdf
• 28 Arzt NH, Chertcoff D, Nicolary S, Suralik M, Berry M. Immunization calculation engine: an open source immunization evaluation and forecasting system. Learn Health Syst 2021; 6 (01) e10285
• 29 MIROW. Consolidating demographic records and vaccination event records. 2017 . Accessed September 8, 2023 at: https://repository.immregistries.org/files/resources/59d677eb1b908/aira_mirow_consolidating_demographic_rec_updated.pdf
• 30 Zayas-Cabán T, Haque SN, Kemper N. Identifying opportunities for workflow automation in health care: lessons learned from other industries. Appl Clin Inform 2021; 12 (03) 686-697
• 31 Abbott EK, Coyle R, Dayton A, Kurilo MB. Measurement and improvement as a model to strengthen immunization information systems and overcome data gaps. Int J Med Inform 2021; 148: 104412
• 32 Stockwell MS, Natarajan K, Ramakrishnan R. et al. Immunization data exchange with electronic health records. Pediatrics 2016; 137 (06) e20154335
• 33 Pabst LJ, Williams W. Immunization information systems. J Public Health Manag Pract 2015; 21 (03) 225-226
• 34 Richwine C, Strawley C. Electronic Access to Immunization Information among Primary Care Physicians. ONC Data Brief. April 2024
• 35 Stephens AB, Wynn CS, Hofstetter AM. et al. Effect of electronic health record reminders for routine immunizations and immunizations needed for chronic medical conditions. Appl Clin Inform 2021; 12 (05) 1101-1109
• 36 Vinci DM, Ryan J, Howard M. et al. Increasing human papillomavirus vaccination in a federally qualified health center organization using a systems-based intervention integration EHR and statewide immunization information system. J Community Health 2022; 47 (01) 53-62
• 37 American Immunization Registry Association. Decrementing Inventory via Electronic Data Exchange. 2016 . Accessed September 28, 2023 at: https://repository.immregistries.org/files/resources/5835adc2a034b/aira_mirow_decrementing_inventory_via_electronic_data_exchange_mini_guide.pdf
• 38 CDC. Immunization Information System (IIS) Functional Standards, v4.1. Accessed October 23, 2023 at: https://www.cdc.gov/vaccines/programs/iis/functional-standards/func-stds-v4-1.html