Keywords
computerized provider order entry - clinical decision support - electronic health record - renal dose adjustment - medication alert systems
Background and Significance
Background and Significance
Clinical decision support (CDS) tools are decision aides within clinical information systems that provide information and knowledge to optimize the delivery of patient care.[1]
[2]
[3]
[4] Current clinical information systems apply CDS in a variety of formats. CDS can be delivered passively through informational text or a limited selection list. CDS can also be delivered actively, such as through noninterruptive user prompts to be resolved at a time of the user's choice, or through interruptive alerts requiring immediate resolution.[1]
[2]
The “five rights” of CDS states that the right information should be delivered to the right person through the right channel in the right format at the right time.[5] If optimally designed, CDS can augment patient care by providing technical safeguards to clinician-end users as they enter medication orders electronically.[2]
[3] This promise, however, is largely unrealized, as many users respond to poorly designed CDS with high override rates and alert fatigue.[4]
[5]
[6]
[7]
Several individual strategies have been successful in improving user adherence to alerts. A noninterruptive CDS strategy that provided users with preselected defaults and a filtered selection list of context-appropriate order parameters was shown to be an effective means of providing renal dose guidance.[8] A passive CDS strategy of providing in-line support text was shown to be an effective means of increasing adherence to drug–drug interaction alerts.[9] We leveraged both strategies and designed a multistage CDS system to support successive parts of ordering workflow, beginning with order entry, proceeding through order signing, and concluding with medication warning resolution. We targeted antimicrobial medications requiring renal dose adjustment prescribed to pediatric patients. Renal dose adjustments are common clinical decisions that require the user to account for the patient's degree of renal impairment using laboratory markers and clinical calculations (e.g., Schwartz's equation), and then adjust the medication regimen accordingly.[10]
[11] If renal dose adjustments are overlooked, patients may experience drug toxicity due to their reduced capacity to eliminate these compounds and/or their metabolites.[12]
[13]
[14] Antimicrobial medications are commonly administered within inpatient settings, are routinely monitored by clinicians, and have well-defined dosing schedules for varying degrees of renal impairment.[15]
[16] This class of medications is therefore well-suited for trialing a CDS system designed to be transparent, flexible, minimally intrusive, and integrated closely with medication ordering and prescribing.
Objectives
The objective of this study was to investigate the impact of the passive dose-guidance and successive CDS, which employs varying renal dosing assistance strategies at multiple points of the order entry workflow, on adherence of providers to institutional antimicrobial dosing guidelines in hospitalized pediatric patients.
Methods
Study Setting
The study was conducted at a 1,000-bed academic health center with a pediatric hospital containing 276 beds. The electronic health record (EHR) at the time of the study was based on the Epic platform (Epic version 2014, Epic Systems, Verona, WI), and medication warning data were supplied through a third-party data vendor (First Data Bank, San Francisco, CA) through monthly loads.
Antimicrobial Dosing Guidelines
An interdisciplinary committee published pediatric antimicrobial dosing guidelines that provided evidence-based dosing parameters for antibiotic, antifungal, and antiviral drugs for a variety of indications and disease severities. The guidelines also provided dose adjustment recommendations for five discrete categories of renal function: continuous renal replacement therapy (CRRT); intermittent hemodialysis (iHD); creatinine clearance (CRCL) less than 10 mL/min; CRCL between 10 mL/min and less than 30 mL/min; CRCL between 30 mL/min and 50 mL/min; or CRCL greater than 50 mL/min.
CDS Intervention and Implementation
The implementation of our successive CDS system was separated into two phases due to logistical issues arising from the EHR version upgrade occurring concurrently with the project. The phased implementation presented an opportunity to investigate the effects of the different CDS elements individually ([Fig. 1]). End users were informed via a brief system-wide e-mail communication following a regular system enhancement. Each phase is described in the following subsections:
Fig. 1 Implementation timeline and study design. The solid line denotes the implementation timeline, and the dotted line denotes the study design's comparisons.
Noninterruptive Dose Warnings and Interruptive Dose Alerts without Renal Dose Adjustment Recommendations (Historical Control)
Prior to March 2015, passive dose guidance was absent and dose-checking functionality was based on dosing rules provided by the data vendor, which did not provide renal adjustment recommendations. If a provider entered order parameters (dose and frequency) that exceeded preconfigured thresholds, an in-line dose warning appeared directly below order composer's dose field in a noninterruptive manner ([Fig. 2]). If the provider continued through the prescribing workflow and signed the order, the system generated an interruptive dose alert, prompting the provider to either change the dose or acknowledge/override the alert ([Fig. 3]). Orders placed during this time period served as the historical controls.
Fig. 2 Noninterruptive in-line dose warning. The warning shown contains an actionable dose button that redirects the user to the maximum dose allowed for the patient (not pictured).
Fig. 3 Interruptive dose alert.
Addition of Passive Dose-Guidance CDS (Phase I)
During phase I, we implemented passive dose guidance to display within the ordering instructions of the order composer ([Fig. 4]). If a provider selected an antimicrobial for order entry, the system displayed appropriate dosing recommendations using embedded rules and logic based on the calculated CRCL and selected drug. [Table 1] provides the detailed order instructions displayed for cefepime as an illustrative example.
Table 1
Criteria for passive dose-guidance CDS recommendations. Details for cefepime shown
|
Criterion
|
Passive CDS displayed text
|
|
No HD or CRRT order, CRCL incalculable
|
Creatinine clearance is not available because the patient may not have height or weight documented or a creatinine level is not available from the last 14 days. Please consult the antibiotic guidelines or pharmacy for assistance.
|
|
CRRT order
|
Patient has an active CRRT order. Estimated creatinine clearance: ___ mL/min/1.73 m2 (by Schwartz formula based on Cr of __)
Recommended doses are:
Mild/mod infection: 50 mg/kg/dose every 12 hours
Severe infection/febrile neutropenia: 50 mg/kg/dose every 12 hours
CF: 50 mg/kg/dose every 12 hours
|
|
HD order
|
Patient has an active hemodialysis order. Estimated creatinine clearance: ___ mL/min/1.73 m2 (by Schwartz formula based on Cr of __)
Recommended doses are:
Mild/mod infection: 25 mg/kg/dose every 48 hours
Severe infection/febrile neutropenia: 25–50 mg/kg/dose every 48 hours
CF: 25–50 mg/kg/dose every 48 hours
Schedule dose after hemodialysis
|
|
CRCL < 10 mL/min
|
Estimated creatinine clearance: ___ mL/min/1.73 m2 (by Schwartz formula based on Cr of __)
Recommended doses are:
Mild/mod infection: 25 mg/kg/dose every 48 hours
Severe infection/febrile neutropenia: 25–50 mg/kg/dose every 48 hours
CF: 25–50 mg/kg/dose every 48 hours
|
|
CRCL 10–29 mL/min
|
Estimated creatinine clearance: ___ mL/min/1.73 m2 (by Schwartz formula based on Cr of __)
Recommended doses are:
Mild/mod infection: 50 mg/kg/dose every 24 hours
Severe infection/febrile neutropenia: 50 mg/kg/dose every 24 hours
CF: 50 mg/kg/dose every 24 hours
|
|
CRCL 30–50 mL/min
|
Estimated creatinine clearance: ___ mL/min/1.73 m2 (by Schwartz formula based on Cr of __)
Recommended doses are:
Mild/mod infection: 50 mg/kg/dose every 24 hours
Severe infection/febrile neutropenia: 50 mg/kg/dose every 12–24 hours
CF: 50 mg/kg/dose every 12–24 hours
|
|
CRCL > 50 mL/min
|
Estimated creatinine clearance: ___ mL/min/1.73 m2 (by Schwartz formula based on Cr of __)
Recommended doses are:
Mild/mod infection: 50 mg/kg/dose every 12 hours
Severe infection/febrile neutropenia: 50 mg/kg/dose every 8 hours
CF: 50 mg/kg/dose every 8 hours
|
Abbreviations: CF, cefepime; CRCL, creatinine clearance; CRRT, continuous renal replacement therapy; HD, hemodialysis.
Fig. 4 Passive dose-guidance CDS.
Noninterruptive Dose Warnings and Interruptive Dose Alerts with Renal Dose Adjustment Recommendations (Phase II)
In phase II, the data vendor updated the dose checking rules with renal dose adjustment recommendations. We reconfigured the rules to match dosing recommendations and renal adjustment parameters specified by the institution's guidelines. Upon fully implementing the successive CDS system in phase II, providers may receive CDS at multiple points during ordering workflow: (1) passive dose guidance, matching the institutional guidelines and adjusted for renal function, immediately after selecting an antimicrobial medication to order, (2) noninterruptive dose checking, matching the institutional guidelines and adjusted for renal function, during order parameter selection, and (3) interruptive dose alerting, matching the institutional guidelines and adjusted for renal function, after order signing.
Renal Function and Creatinine Clearance Calculation Assumptions
The system calculated renal function for pediatric patients using a modified Schwartz equation,[10] where K = 0.413 for all patients meeting the system's pediatric definition (age greater than 1 year but less than 14 years, or age 14–18 years and weighing less than 50 kg). CRCL was calculated using the most recent serum creatinine result within a 19-day lookback period. HD- or CRRT-specific recommendations overrode CRCL-based recommendations when discrete orders for HD or CRRT existed. If no CRCL was calculable, the system stated that renal function was incalculable and instructed the user to refer to the institutional guidelines for further assistance.
Study Design
The study was a single-center, observational, retrospective cohort study. The study compared provider adherence to institutional guidelines across three time points: a historical control period prior to implementation, phase I of CDS implementation (March 2015 through May 2015), and phase II of CDS implementation (May 2015 through October 2015).
We assessed all antimicrobial orders within this time frame. We included antimicrobial orders for admitted patients who fit the system's definition of a pediatric patient. We excluded orders for patients with a principal diagnosis or problem list indication of cystic fibrosis; without a documented creatinine or height; with “once,” “user-specified,” “as needed,” or “preoperative” frequencies; with a free-text antimicrobial selected; or with antimicrobials without CDS configured. These antimicrobials either lacked renal dose adjustment recommendations or were subject to therapeutic drug monitoring by clinical pharmacists.
Outcomes and Analysis
The primary outcome of the study was defined as the proportion of orders adherent to institutional guidelines with regard to both dose and frequency. The secondary outcomes of the study were defined as the proportion of orders adherent to institutional guidelines with regard to dose alone, frequency alone, or total daily dose alone. We also assessed the proportion of adherent orders stratified by user roles and by presence of renal dysfunction. User roles were defined as physicians, pharmacists, registered nurses, and advanced practice practitioners (e.g., nurse practitioners and physician assistants).
To conduct the analysis, we compared the order parameters between the provider's initial order and the parameters recommended by the CDS system. Because the CDS system suggested a range of appropriate doses, we defined the reference range as the range of values between the lowest possible value and the highest possible value as suggested by the institutional dosing guidelines. A 10% allowance was incorporated to account for the EHR's threshold for triggering a maximum dose-checking alert.
Data and Statistical Analysis
Order parameters were collected from EHR reports. For each order placed, the system logged the initial order parameters selected by the ordering user (e.g., drug, dose, frequency) in addition to final order parameters filed after the pharmacist verified the order. Our analysis focused on the initial order parameters, rather than the final order parameters. These data were then joined to patient-specific information extracted from other EHR reports. Mathematical calculations and initial comparisons/analysis were conducted in Microsoft Excel (Microsoft, Redmond, WA) spreadsheet software.
Student's t-test was used to compare baseline age and CRCL. χ
2 test was used to compare baseline gender frequency. Primary and secondary outcomes were recorded as categorical variables and reported as frequencies and percentages, with odds ratios (ORs) and p-values describing the magnitude of effect. We used a logistical regression with a mixed model analysis (PROC GLIMMIX) to assess statistical significance. We used the mixed model to account for the interrelatedness of groups of orders clustering within individual providers. Statistical tests were performed in SAS (SAS Institute, Cary, NC).
Results
After excluding all ineligible orders, a total of 1,588 orders remained for further analysis, ordered by 488 different unique providers, 618 orders in the control period, 234 orders in the phase I, and 736 orders in phase II ([Table 2]). The mean patient age was 7.2, 6.7, and 6.8 years in the control period, phase I, and phase II, respectively. Although the mean CRCL was different between the time periods, the mean CRCL was well above the institutional guideline's minimum threshold for normal renal function (CRCL of 50 mL/min) for all time periods.
Table 2
Baseline information
|
Control
|
Phase I
|
Phase II
|
|
Orders written, n (%)
|
618 (39)
|
234 (15)
|
736 (46)
|
|
Age (y), mean ± SD (p)[a]
|
7.3 ± 4.5
|
6.7 ± 4.8 (0.07)
|
6.8 ± 4.5 (0.04)[*]
|
|
Female, n (%, p)[a]
|
269 (44)
|
114 (49,0.17)
|
371 (50,0.01)
[*]
|
|
CRCL, mean ± SD (p)[a]
|
182.5 ± 92.9
|
159.8 ± 79.3 (0.001)[*]
|
152.1 ± 75 (<0.001)
[*]
|
|
Orders written by CRCL category
|
|
CRRT, n (%)
|
40 (6)
|
27 (12)
|
36 (4.9)
|
|
<10 or HD, n (%)
|
30 (5)
|
5 (0.3)
|
4 (0.5)
|
|
10–29, n (%)
|
6 (1)
|
7 (3)
|
9 (1.2)
|
|
30–50, n (%)
|
17 (3)
|
4 (1.7)
|
10 (1.4)
|
|
>50, n (%)
|
525 (85)
|
191 (82)
|
677 (92)
|
|
Orders written by provider type
|
|
Physician, n (%)
|
413 (67)
|
151 (65)
|
472 (64)
|
|
APP, n (%)
|
82 (13)
|
43 (18)
|
143 (19)
|
|
Registered nurse, n (%)
|
60 (10)
|
28 (12)
|
86 (12)
|
|
Pharmacist, n (%)
|
63 (10)
|
12 (5)
|
35 (5)
|
Abbreviations: APP, advanced practice practitioner (nurse practitioner and physician assistant); CRCL, creatinine clearance; CRRT, continuous renal replacement therapy; HD, hemodialysis.
a Student's t-test to compare mean age and creatinine clearance, χ
2 test for gender comparison between phase I vs. control period, and between phase II vs. control period.
*
p-value < 0.05.
Primary Outcome
The proportion of adherent orders with respect to both dose and frequency increased from 74% in the control period to 81% in phase II of the study (OR = 1.54; 95% confidence interval = 1.16–2.03; p = 0.003). There was no statistically significant difference in the primary outcome in phase I of the study compared with the control period ([Table 3], [Fig. 5], [Fig. 6]).
Fig. 5 Percentages of antimicrobial orders within recommended range specified by institutional guidelines. Control: noninterruptive dose warnings and interruptive dose alerts without renal dose adjustment recommendations. Phase I: addition of passive dose-guidance CDS. Phase II: noninterruptive dose warnings and interruptive dose alerts with renal dose adjustment recommendations.
Fig. 6 Odds ratio of adherent orders compared versus historical control. Control: noninterruptive dose warnings and interruptive dose alerts without renal dose adjustment recommendations. Phase I: addition of passive dose-guidance CDS. Phase II: noninterruptive dose warnings and interruptive dose alerts with renal dose adjustment recommendations.
Table 3
Results of the successive CDS system across the phased implementation
|
Adherence
|
Control, n (%)
|
Phase I
|
Phase II
|
|
n (%)
|
OR (95% CI, p)
|
n (%)
|
OR (95% CI, p)
|
|
Dose and frequency
|
|
Overall
|
456 (74)
|
178 (76)
|
1.11 (0.76–1.16, 0.6)
|
596 (81)
|
1.54 (1.16–2.03, 0.003)[*]
|
|
Physician
|
297 (72)
|
117 (77)
|
1.31 (0.82–2.1, 0.26)
|
380 (81)
|
1.66 (1.18–2.35, 0.004)
[*]
|
|
APP
|
63 (77)
|
31 (72)
|
0.78 (0.3–2.03, 0.61)
|
118 (83)
|
1.47 (0.7–3.09, 0.31)
|
|
Registered nurse
|
55 (92)
|
23 (82)
|
0.42 (<0.001 to >999, 0.42)
|
72 (84)
|
0.47 (<0.001–512.68, 0.4)
|
|
Pharmacist
|
41 (65)
|
7 (58)
|
0.8 (0.2–3.15, 0.74)
|
26 (74)
|
1.69 (0.58–4.9, 0.33)
|
|
CRCL > 50 mL/min
|
410 (78)
|
152 (80)
|
1.01 (0.65–1.58,0.96)
|
560 (83)
|
1.28 (0.93–1.76, 0.13)
|
|
Abnormal renal function
|
46 (49)
|
26 (60)
|
1.69 (0.72–4)
|
36 (61)
|
1.92 (0.92–4, 0.08)
|
|
Dose
|
517 (84)
|
212 (91)
|
1.74 (1.04–2.93, 0.04)
[*]
|
662 (90)
|
1.69 (1.19–2.4, 0.004)
[*]
|
|
Frequency
|
527 (85)
|
192 (82)
|
0.78 (0.51–1.21, 0.27)
|
644 (88)
|
1.2 (0.85–1.68, 0.3)
|
|
Total daily dose
|
491 (79)
|
197 (84)
|
1.31 (0.86–2.01, 0.21)
|
629 (85)
|
1.52 (1.12–2.06, 0.01)[*]
|
Abbreviations: APP, advanced practice practitioner (nurse practitioner and physician assistant); CI, confidence interval; CRCL, creatinine clearance; OR, odds ratio.
Note: n is the number of orders within guidelines-recommended range; % is the percentage of orders per total number of orders for the study phase.
*
p-value < 0.05.
Secondary Outcomes
The CDS system showed a statistically significant improvement in the proportion of orders with a dose within the recommended range in both phase I and phase II compared with the control period, and in the proportion of orders with a total daily dose within the recommended range in phase II of the study compared with the control period. There was no statistically significant difference in the proportion of orders with a frequency within the recommended range in any of the study phases. A subgroup analysis of adherent orders with regard to dose and frequency stratified by provider type and by renal function category is shown in [Table 3]. We observed a statistically significant improvement in the proportion of adherent orders among physicians from the control period to phase II of the study. The increase in the proportion of adherent orders with respect to both dose and frequency was more apparent in patients with reduced renal function when compared with patients with normal renal function (CRCL > 50 mL/min) but this difference was not statistically significant ([Table 3], [Fig. 5], [Fig. 6]).
Discussion
This study found that provider adherence to guideline-recommended pediatric antimicrobial regimens improved following the implementation of a successive CDS system that incorporated passive and noninterruptive/interruptive CDS elements during phase II of the study. This study also investigated the impact of a passive dose-guidance CDS during phase I of the study, but no statistical significance difference was found during this phase when compared with the control period.
Physicians placed majority of the orders (65%). In total, 95% of orders placed by physicians were by resident physicians in training. The subgroup analysis based on provider type showed even a higher improvement in adherence among physicians in relation to other provider types. The difference in adherence for orders placed by nonphysician providers between study phases did not reach statistical significance because the number of orders placed by them was relatively small. The subgroup analysis based on presence of renal impairment showed a trend for larger improvement in adherence in orders for patients with impaired renal function compared with those with normal renal function. The odds of adherence within the renal impairment subgroup were 40 and 33% higher than the normal renal function subgroup in phases I and II of the study, respectively. This trend was not statistically significant and will require a study with larger sample size to confirm.
Our CDS system provided a novel approach for guiding renal dose adjustments that adhered to CDS best practices.[3]
[4] The CDS system provided real-time guidance at multiple stages of ordering workflow: immediately after medication selection, then during order parameter selection, and, lastly, following order signing. The CDS system incorporated successive safeguards that prevented the user from ordering an inappropriate regimen. Additionally, the CDS system was designed to minimize intrusiveness and utilized noninterruptive, in-line warnings to reduce downstream interruptive alerting, thereby fitting closely with a user's natural ordering workflow. Also, the CDS system provided multiple acceptable, patient-specific recommendations, thereby giving users' flexibility when choosing a clinically appropriate regimen. Lastly, the CDS system was transparent to its users. The passive CDS displayed informational text directly replicated from the institutional guidelines, and users could view criterion triggering dose warnings through a “view details …” hyperlink embedded within the order composer.
Our CDS system was able to provide transparent CDS. The passive CDS informational text began with a statement describing the renal function criterion (e.g., “Estimated CRCL: 24.8 mL/min/1.73 m2 [by Schwartz's formula based on Cr of 1.5]”) followed by the corresponding dosing recommendations. By using this strategy, the system was effectively delivering drug and patient-specific snippets of the guidelines directly to the user at the time of ordering. When in-line dose warnings were triggered, users could also reference dose-checking criteria through a “view details …” hyperlink, which displayed the systems dose-checking logic based on age, weight, and renal function. The desirability of transparency of CDS rules has been discussed in the literature. An evaluation study of a computer-assisted management program for antibiotics reported significant reduction of inappropriate antibiotic regimens, length of stay, and drug costs. In that study, ordering users could view the “Explain Logic ...” for a detailed description of the source of recommendations. The study authors discussed that an important strength of their system was the transparency by which the system displayed its recommended antimicrobial regimen.[17]
Our successive CDS system has several advantages to other published strategies. We attributed these advantages to our strategy of using passive dose guidance instead of other noninterruptive strategies such as preselecting order defaults. Chertow et al's CDS system improved appropriate medication ordering in a general hospitalized patient population functioned by preselecting dose and frequency fields and filtering inappropriate choices from selection lists based on calculated renal function.[8] However, their system was limited in that dose and frequency selection logic depended on calculated CRCL. Therefore, the system had the potential to populate orders with clinically inappropriate doses and/or frequencies for patients undergoing CRRT in which a calculated CRCL based on serum creatinine may falsely represent renal function. Our CDS system contained logic that recognized the presence of HD or CRRT and therefore provided the appropriate CDS regardless of the calculated CRCL. Killelea et al's CDS system in a pediatric population preselected default dose and frequency values based on medication form, patient age, patient weight, and most common indication. Recommendations for alternative regimens (e.g., renal impairment or other indications) were displayed as text in a separate window. Provider adherence to the preselected order defaults was low (30%) and users of this CDS system frequently deviated from order defaults because of the system's limited logic, which could only account for a single indication at a time.[18] A distinct advantage in our CDS system was its capacity to recommend multiple dosing regimens for different indications and disease severities at one time. Our findings in phase I demonstrate that passive CDS improved adherence to the recommended regimens in regard to dose selection, although its effect on both dose and frequency showed a positive trend that did not reach statistical significance.
Other institutions have utilized novel dose-checking CDS strategies by generating interruptive alerts.[19]
[20]
[21] Literature also exists showing that provider adherence to interruptive alerts improves if they are exposed to fewer interruptive alerts overall.[9] We did not implement a purely interruptive strategy as that would have added to the number of interruptive alerts in our EHR system. The goal of our strategy was to reduce the number of interruptive alerts by incorporating upstream passive and noninterruptive CDS, and improve the quality of interruptive alerts by making them specific to patient's renal function.
Limitations
The study was a retrospective, observational study design at a single academic medical center. Most orders in this setting are placed by practitioners in training, and results may not be generalizable to other nonacademic settings where most orders are placed by more experienced providers.
The CDS system's design, implementation, and investigation were constrained to antimicrobial medications within a specific patient demographic. However, the CDS system's design is easily replicable and can be readily expanded to include other types of medications and broader patient populations.
The study design assumes that institutional clinical guidelines are optimal for every patient, and that increased provider adherence to these guidelines is a desired behavior. However, these recommendations may be subject to change as the medical community discovers more knowledge about antimicrobial medications, infectious disease states, or the affected patient populations. Given this fluidity, 100% provider adherence to dose suggestions provided by our CDS system may not be a realistic target.
Our study assessed provider adherence to the institutional dosing guidelines but was not powered to assess clinical and economic patient outcomes such as length of stay, quality of life, and avoidance of adverse drug events. A much larger study is needed to understand the impact of novel CDS tools on patient outcomes.
We were constrained by the implementation timeline of our study. This timeline limited the number of orders available for study, especially in phase I. We were also unable to conduct a direct comparison between phase I and phase II of the study, and were therefore unable to separate the cumulative effect of the CDS system in phase I compared with phase II. Future work further investigating the difference in effect between noninterruptive and interruptive CDS strategies as they pertain to dose guidance and renal function would be beneficial.
Conclusion
Provider adherence to the institutional antimicrobial dosing guidelines significantly improved following the implementation of a successive CDS system combining passive and noninterruptive/interruptive strategies. We describe the value of a passive dose-guidance CDS system that functions in real time and provides multiple acceptable, patient-specific recommendations concurrently. We also describe the value of designing a successive CDS system that supports users through multiple stages of ordering workflow and minimizes intrusiveness. The CDS system improved provider adherence to institutional dosing guidelines using a blend of CDS strategies and can be considered for implementation by other institutions using similar EHR systems.
