Keywords
predictive analytics - decision support - dashboard - heart failure - population metrics
Background and Significance
Background and Significance
Heart failure (HF), a prevalent and costly condition, is one of the leading causes
for hospitalization in the United States among adults[1]
[2] and is the most common Medicare inpatient discharge diagnosis.[3] Within the Veterans Affairs (VA) healthcare system, HF is the second most common
diagnosis as well as one of the most costly diagnoses to treat annually.[4]
[5]
[6]
[7] The relatively higher prevalence of HF[8]
[9] among veterans may be secondary to their elevated risk of poor physical and mental
health.[10]
[11]
[12]
[13] The Veterans Integrated Service Network (VISN 1) has eight medical centers located
in six New England states that deliver care to approximately 4,200 veterans with HF.
This subset of the population serviced by VISN 1 has serious cardiopulmonary disease,
as well as the highest readmission rates.[14] Studies investigating reasons for these high readmission rates identified medication
discrepancies and cognitive impairment as likely contributors.[15]
[16] One study noted that 30-day readmission rates were higher in HF patients who were
not on a target dose of β blockers or vasodilators.[9]
However, early electronic identification of patients at the highest risk for admission
presents a challenge.[17] The data needed to identify HF patients are located in a variety of areas in the
electronic medical record (EMR) and in different formats (free text notes, images,
and coded data). Dashboards for population health management are powerful tools that
can be used to identify subsets of patients to improve their care and track their
progress toward performance goals.[18]
[19] The use of dashboards within the VA has enabled the ability to implement system-wide
processes for both population management and quality improvement.[18]
[20]
[21]
Dashboards that utilize both structured and unstructured data can produce patient-specific
risk assessments and support guideline-directed medical therapy (GDMT) recommendations
leading to greater evidence-based care.[22]
[23]
[24] Previous studies have demonstrated the efficacy of using multiple identifiers in
algorithms or from notes using natural language processing (NLP) for identifying patient
phenotype for early diagnosis of HF.[25]
[26]
[27]
[28] We therefore endeavored to develop and implement a dashboard that tracks clinical
treatment. We hypothesized that the dashboard would help to identify all patients
with HF that are at the highest risk for readmission across the medical center with
the hopes of supporting better patient care and improving the cost-effectiveness of
clinical practice, which is a major goal of the VA.[29]
Objective
The purpose of this paper is to describe the development and data validation of a
HF dashboard that monitors the overall metrics of outcomes and treatments of the veteran
patient population with HF while providing guidance to clinicians on mainstay of pharmacologic
therapies.
Methods
The process improvement project took place at VA Boston Medical Center, a 150-bed
tertiary care hospital that provides care to veterans from four of the six New England
states within VISN 1. The initial project team of a business analyst, database programmer,
and a cardiologist with expertise in HF met to discuss the dashboard's development
and determined that an Agile software development approach[30] would be most feasible. The team collaborated with the Director of Primary Care,
Chief Medical Officer, and Primary Care Patient Aligned Care Teams (PACTs) who would
be the primary users of the dashboard. PACTs consist of providers, nurse care managers,
clinical associates, pharmacists, and ancillary staff that manage Veteran patients'
overall health care.[31] Key Performance Indicators (KPIs) and data points used to develop the dashboard
were determined by the cardiologist and refined with input from the PACTs, Director
of Primary Care, and Chief Medical Officer. The KPIs evaluate effectiveness of HF
management across VISN 1: admissions per 100 patients by fiscal year; admissions for
HF per 100 patients by fiscal year; bed day of care per 100 patients by fiscal year;
and ER visit per 100 patients by fiscal year. These KPIs were based on the quality
metrics for HF care that now exist based on process of care and outcomes.[1]
[32]
Data points
Data points were obtained from three different databases and included: care assessment
need (CAN) score,[33]
[34] EF, medication history, specific laboratory testing (creatinine, B-type natriuretic
peptide, hemoglobin, and sodium), specific comorbidities (chronic kidney disease,
chronic obstructive pulmonary disease, diabetes, and coronary artery disease), primary
care, and cardiology appointments. The CAN score is calculated by the Veterans Health
Administration Support Service Center. It reflects the estimated probability of three
outcomes for an individual veteran patient: (1) hospitalization, (2) death, and (3)
hospitalization or death. The percentile of these probabilities ranges from 0 (lowest
risk of hospitalization or death) to 99 (highest risk hospitalization or death), and
it gives a perspective on how the patient compares with other veterans in terms of
their likelihood of a given event over a 90-day period.[33] NLP and information extraction techniques were used to extract EF data from free
text notes. Based on the literature, four distinct EF cohorts were formed: HF with
preserved EF (HFpEF) >50%, HF with reduced EF (HFrEF) ≤40%, HF with recovered EF (HFrecEF),
patients who had an EF at one time of <40%, and is now >50%, and HF with intermediate
or mid-range EF (HFmrEF) 41 to 49%.[1] Querying of structured data was done by the database programmer who first confirmed
a HF diagnosis using International Classification of Disease (ICD) 9 code of 428.x
or an ICD 10 code of 150.x. An algorithm was used to determine patients' prescription
concordance which is the agreement between the provider's treatment plan and current
GDMT.[1]
[35] The algorithm searched the records to see if any of the target medications ([Table 1]) were ordered in the past 5 years. In addition to prescription concordance, medication
concordance was evaluated. Medication concordance provides information on whether
patients are on target doses of GDMT. The algorithm determined patients' medication
concordance for each class of medication listed in [Table 1] and noted if there was full concordance, partial concordance, partially concordant
but not at target dose or nonconcordant. Fully concordant are those on target doses
of medications noted in [Table 1]. Partially concordant are those on at least one class of medication at target dose
while partially concordant but not at target dose(s) are those on all classes of medications,
but none are at target dose. Nonconcordant refers to those not taking any recommended
medications.
Table 1
Prescriptions and respective target dose
|
Medication class
|
Prescription and target dose
|
|
β blockers
|
• Metoprolol succinate target dose 200 mg daily
• Carvedilol target dose 25 mg twice a day
• Bisoprolol target dose 10 mg daily
|
|
Vasodilators
|
• Lisinopril, or fosinopril, or enalapril target dose 20 mg daily
• Captopril target dose 50 mg three times a day
• Valsartan target dose 160 mg twice a day
• Candesartan target dose 32 mg daily
• Losartan 150 mg daily
• Hydralazine 300 mg daily and isosorbide 160 mg daily (both prescriptions)
|
|
Aldosterone blockers
|
• Spironolactone 25 mg daily
• Epleronone 25 mg daily
|
|
Diuretics (no specific dosage or frequency)
|
• Furosemide
• Bumetanide
• Torsemide
• Metolazone
• Chlorthalidone
|
Data Validation
Members of the PACTs conducted data validation testing at three different points during
the dashboard development. They were given a test guide and scripts which were used
to validate information from excel spreadsheets containing graphs, dashboard table,
and patient detail reporting for a specific site and clinician. Validation testers
were directed to indicate “P” for pass if the test case presented the expected result
in each field or “F” for fail if that expectation was not met.
Results
We developed the dashboard and completed validation testing in March of 2019. The
landing page of the HF dashboard has four graphs see [Fig. 1]. All graphs display outcomes for each medical center within VISN 1. The landing
page also has four tables that show the patient base for each of the four defined
cohorts by EF level [Fig. 2]. Note that medication concordance is not displayed for patients in the HFpEF and
HF with intermediate EF cohort as no current guidelines exists for patients with these
EFs. The medication concordance algorithm will re-run on a nightly basis to capture
changes in patient's medications as patients have the potential to transition to another
cohort if they experience a major cardiovascular event. A drill down report of individual
patients is available to clinicians by clicking on their respective home Veteran Affairs
Medical Center from one of the four cohort-based tables noted in [Fig. 2]. [Fig. 3] is an example of a drill down report that is populated with fictitious data. The
report displays in a left to right with color coded columns to enhance the ease of
reading and includes a hyperlink to GDMT for providers to review if necessary.
Fig. 1 Graphs on dashboard home screen.
Fig. 2 Tables on dashboard home screen.
Fig. 3 Example of a drill down reduced EF cohort report (fictional patient data). EF, ejection
fraction.
Eleven clinicians in total, seven physicians, two nurse practitioners, and two nurses
validated dashboard data by comparing it with patient EMRs. A total of 43 medical
records were reviewed and 66 HF dashboard data discrepancies or issues were identified
[Table 2]. In addition to the discrepancies, users also provided suggestions on how to increase
usability of the dashboard. Suggestions included the flow of information within the
dashboard, the use of color to differentiate columns in the drill-down reports page
as well as changes in wording, for example, compliance was replaced by concordance
and we modified cohort definitions to make them clearer.
Table 2
Results of testing over time
|
Time I
charts reviewed (n = 18)
Number of providers (n = 5)
|
Time II
charts reviewed (n = 11)
Number of providers (n = 4)
|
Time III
charts reviewed (n = 14)
Number of providers (n = 2)
|
|
Issue/discrepancy
|
Number of frequencies of issue/discrepancy over time
|
|
Missing Hgb value
|
6
|
3
|
|
|
Inaccuracy of last EF note date
|
7
|
3
|
10
|
|
CKD, CAD, and DM diagnosis not captured
|
14
|
5
|
|
|
BNP value missing
|
|
1
|
|
|
Patients concordant for medications in chart, but it did not show up on dashboard
|
7
|
1
|
|
|
Cardiology was not noted as the specialty care
|
1
|
|
|
|
Furosemide issue date was incorrect
|
3
|
|
|
|
Metoprolol issue date was incorrect
|
1
|
|
|
|
Spironolactone issue date was incorrect
|
1
|
|
|
|
Cardio Appt within last 1 year was incorrect
|
|
|
2
|
|
Deceased patient included in the dashboard population
|
|
|
1
|
Abbreviations: BNP, brain natriuretic peptide; CAD, coronary artery disease; CKD,
chronic kidney disease; DM, diabetes mellitus; EF, ejection fraction; Hgb, hemoglobin.
Discussion
For this process improvement project, an Agile software development approach was used
to carefully explore and integrate the perspectives of key stakeholders into the dashboard.
Dashboards developed with input from end-users, leadership, and subject matter experts
have a greater chance of being adopted and have higher user acceptance.[21]
[36]
[37] The project team used structured and unstructured VA data to develop the dashboard.
The dashboard presents data via a clear mechanistic interface and allows users to
see comparisons between hospitals. The HF dashboard is suitable for the use case because
it will allow for collaborative population care among members of primary care PACTs.
It can also be used by pharmacy to review prescriptions concordance and collaborate
with primary care/cardiology for appropriate follow-up.
During dashboard development, we encountered several challenges, duplicate data, missing
data, inaccuracy in the last EF note date (the date when the note was added to EMR,
not necessarily the actual day the echocardiogram was performed), and multiple EFs
noted on the same patient in a single date. This occurred because the dashboard reports
the date of the note containing an EF value and not the date when the echocardiogram
was performed. It should be noted that although EF data are initially important to
first categorize patients into a cohort, once patients are placed in a cohort the
exact dates of the EF data update and last EF note date are unlikely to affect the
HF cohort designation. Multiple EFs in a single note were encountered by the NLP algorithm
which led to several EFs being generated for a few patients. The issue was discovered
and adjustments were made to the algorithm used to cleanse the data to ensure that
the algorithm would handle similar future cases appropriately by choosing a truly
representative EF value. Furthermore, re-running of the algorithm will occur approximately
every 6 months to capture EF changes over that time period. This will ensure that
the newest EF is represented in the dashboard. Another issue involved deceased patients
populating in the dashboard. Data are uploaded nightly to the VA's corporate data
warehouse and therefore if a patient's status is listed as alive that person will
populate in the dashboard but will automatically be removed from the dashboard, once
the system refreshes at the next upload. With the validation of the data elements,
we see great potential for the dashboard to enhance HF care at VA Boston. Overall,
evidence indicates that implementing healthcare dashboards can improve clinician adherence
to quality guidelines and more consistently provide GDMT[38] and may play an important role in decreasing readmission rates.[39] Furthermore, if used consistently the HF dashboard could encourage discussions between
providers and patients regarding appropriate HF treatment goals. However, it is recognized
that introducing dashboards can impact workflow.[40] Therefore, the goal is to continue to seek feedback from users on how to improve
the dashboard's usefulness, information quality and efficiency[40] to ensure that the dashboard leverages data that informs clinicians on how best
to manage HF patients.
Limitations
One major drawback is that the current structure of the EMR does not support either
embedding or placing a direct link to the dashboard into the EMR. However, primary
care clinicians agreed to add the dashboard to their hub which can be accessed via
a link from the EMR. The data hub is a location where common data tools, reports,
and dashboards are stored for use by primary care staff to perform their duties. Another
limitation is that the dashboard does not capture prescriptions written and filled
by providers outside of the VA. To capture this information, both cardiology and primary
care providers collect this information from patients and place it in the EMR.
Conclusion
This paper presents a HF dashboard providing real-time information to support better
patient care and thereby improve population metrics. Healthcare dashboards that utilize
both structured and unstructured data, such as the HF Dashboard, can provide cohort-based
groupings as well as individual patient-based risk assessments to help primary care
providers identify and appropriately treat HF patients according to GDMT. Overall,
there are several challenges and opportunities that come with using the HF dashboard.
The collaboration between primary care and cardiology to accomplish the shared goal
of increasing access to quality HF healthcare is central to the success of this project.
Future work with this dashboard will involve follow-up user testing to evaluate the
tool's usefulness, usability, and effectiveness by a group of primary care providers
and their PACTs. In addition, process of care and outcomes will be reassessed at the
1-year mark to determine the quality of HF care within VISN 1.
Clinical Relevance Statement
Clinical Relevance Statement
This case report has clinical relevance for clinicians, programmers, and quality improvement
staff because it adds to the evidence that developing dashboards with input from end-users
enhances the willingness to adopt new software.
Multiple Choice Questions
Multiple Choice Questions
-
Which of the following are best practices of data validation testing?
-
Compare the output result with the expected.
-
Test on full complete data instead of sample data.
-
No need to have a detailed plan as things will change along the way.
-
Handle bad data incorrectly.
Correct Answer: The correct answer is a, compare the output result with the expected. One challenge
in our improvement process project was dealing with the inaccurate outputs over the
development process and determining the root cause. Because validation testers alerted
us to the incorrect output, we were able to remedy the issues.
-
When developing dashboards to support a process improvement project, besides an SQL
database programmer who should be a partner in the design?
-
A postdoctoral medical informatics fellow who is completing the last month of their
fellowship.
-
The chief informatics officer.
-
The end user.
-
Another database programmer.
Correct Answer: The correct answer is option c, the end user. This is crucial because they are the
people who will use the software daily. Our project used an Agile approach to ensure
that the stakeholders who were the end users not only gave input to the dashboard
during development but also validated the data used in the dashboard.
