Download PDF
Appl Clin Inform 2023; 14(05): 822-832
DOI: 10.1055/s-0043-1775566
Research Article

Sustained Effect of Clinical Decision Support for Heart Failure: A Natural Experiment Using Implementation Science

Katy E. Trinkley
1   Department of Family Medicine, University of Colorado Anschutz Medical Campus, School of Medicine, Aurora, Colorado, United States
2   UCHealth, Aurora, Colorado, United States
3   Department of Clinical Pharmacy, University of Colorado Anschutz Medical Campus Skaggs School of Pharmacy and Pharmaceutical Sciences, Aurora, Colorado, United States
,
Garth Wright
3   Department of Clinical Pharmacy, University of Colorado Anschutz Medical Campus Skaggs School of Pharmacy and Pharmaceutical Sciences, Aurora, Colorado, United States
,
Larry A. Allen
4   Adult and Child Center for Outcomes Research and Delivery Science, Aurora, Colorado, United States
5   Division of Cardiology, University of Colorado Anschutz Medical Campus, School of Medicine, Aurora, Colorado, United States
,
Tellen D. Bennett
4   Adult and Child Center for Outcomes Research and Delivery Science, Aurora, Colorado, United States
6   Department of Biomedical Informatics, University of Colorado School of Medicine, Aurora, Colorado, United States
,
Russell E. Glasgow
1   Department of Family Medicine, University of Colorado Anschutz Medical Campus, School of Medicine, Aurora, Colorado, United States
4   Adult and Child Center for Outcomes Research and Delivery Science, Aurora, Colorado, United States
7   Veterans Affairs Eastern Colorado Geriatric Research Education and Clinical Center, Aurora, Colorado, United States
,
Gary Hale
2   UCHealth, Aurora, Colorado, United States
,
Simeon Heckman
2   UCHealth, Aurora, Colorado, United States
,
Amy G. Huebschmann
4   Adult and Child Center for Outcomes Research and Delivery Science, Aurora, Colorado, United States
8   Division of Internal Medicine, University of Colorado Anschutz Medical Campus, School of Medicine, Aurora, Colorado, United States
9   University of Colorado Anschutz Medical Campus Ludeman Family Center for Women's Health Research, Aurora, Colorado, United States
,
Michael G. Kahn
6   Department of Biomedical Informatics, University of Colorado School of Medicine, Aurora, Colorado, United States
,
David P. Kao
2   UCHealth, Aurora, Colorado, United States
3   Department of Clinical Pharmacy, University of Colorado Anschutz Medical Campus Skaggs School of Pharmacy and Pharmaceutical Sciences, Aurora, Colorado, United States
,
Chen-Tan Lin
2   UCHealth, Aurora, Colorado, United States
8   Division of Internal Medicine, University of Colorado Anschutz Medical Campus, School of Medicine, Aurora, Colorado, United States
,
Daniel C. Malone
10   Department of Pharmacotherapy, University of Utah Skaggs College of Pharmacy, Salt Lake City, Utah, United States
,
Daniel D. Matlock
4   Adult and Child Center for Outcomes Research and Delivery Science, Aurora, Colorado, United States
7   Veterans Affairs Eastern Colorado Geriatric Research Education and Clinical Center, Aurora, Colorado, United States
8   Division of Internal Medicine, University of Colorado Anschutz Medical Campus, School of Medicine, Aurora, Colorado, United States
11   Division of Geriatrics, University of Colorado Anschutz Medical Campus, School of Medicine, Aurora, Colorado, United States
,
Lauren Wells
3   Department of Clinical Pharmacy, University of Colorado Anschutz Medical Campus Skaggs School of Pharmacy and Pharmaceutical Sciences, Aurora, Colorado, United States
,
Vincent Wysocki
3   Department of Clinical Pharmacy, University of Colorado Anschutz Medical Campus Skaggs School of Pharmacy and Pharmaceutical Sciences, Aurora, Colorado, United States
,
Shelley Zhang
1   Department of Family Medicine, University of Colorado Anschutz Medical Campus, School of Medicine, Aurora, Colorado, United States
,
Krithika Suresh
4   Adult and Child Center for Outcomes Research and Delivery Science, Aurora, Colorado, United States
12   Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, Colorado, United States
› Author Affiliations Funding K.E.T. was supported in part by the National Heart, Lung, and Blood Institute (grant nos.: K12HL137862 and 1K23HL161352).
› Further Information
    Permissions and Reprints

Abstract

Objectives In a randomized controlled trial, we found that applying implementation science (IS) methods and best practices in clinical decision support (CDS) design to create a locally customized, “enhanced” CDS significantly improved evidence-based prescribing of β blockers (BB) for heart failure compared with an unmodified commercially available CDS. At trial conclusion, the enhanced CDS was expanded to all sites. The purpose of this study was to evaluate the real-world sustained effect of the enhanced CDS compared with the commercial CDS.

Methods In this natural experiment of 28 primary care clinics, we compared clinics exposed to the commercial CDS (preperiod) to clinics exposed to the enhanced CDS (both periods). The primary effectiveness outcome was the proportion of alerts resulting in a BB prescription. Secondary outcomes included patient reach and clinician adoption (dismissals).

Results There were 367 alerts for 183 unique patients and 171 unique clinicians (pre: March 2019–August 2019; post: October 2019–March 2020). The enhanced CDS increased prescribing by 26.1% compared with the commercial (95% confidence interval [CI]: 17.0–35.1%), which is consistent with the 24% increase in the previous study. The odds of adopting the enhanced CDS was 81% compared with 29% with the commercial (odds ratio: 4.17, 95% CI: 1.96–8.85). The enhanced CDS adoption and effectiveness rates were 62 and 14% in the preperiod and 92 and 10% in the postperiod.

Conclusion Applying IS methods with CDS best practices was associated with improved and sustained clinician adoption and effectiveness compared with a commercially available CDS tool.

Keywords

clinical decision support systems - heart failure - prescribing - PRISM - RE-AIM - implementation science

Protection of Human and Animal Subjects

The study was reviewed by the Institutional Review Board and deemed exempt and a full waiver of Health Insurance Portability and Accountability Act authorization was approved.




Publication History

Received: 05 May 2023

Accepted: 02 August 2023

Article published online:
18 October 2023

© 2023. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Poole-Wilson PA, Swedberg K, Cleland JGF. et al; Carvedilol Or Metoprolol European Trial Investigators. Comparison of carvedilol and metoprolol on clinical outcomes in patients with chronic heart failure in the Carvedilol Or Metoprolol European Trial (COMET): randomised controlled trial. Lancet 2003; 362 (9377): 7-13
    CrossrefPubMedGoogle Scholar
  • 2 Lechat P. The Cardiac Insufficiency Bisoprolol Study (CIBIS). CIBIS Investigators and Committees. A randomized trial of beta-blockade in heart failure. Circulation 1994; 90 (04) 1765-1773
    CrossrefPubMedGoogle Scholar
  • 3 Goldstein S, Fagerberg B, Hjalmarson A. et al; MERIT-HF Study Group. Metoprolol controlled release/extended release in patients with severe heart failure: analysis of the experience in the MERIT-HF study. J Am Coll Cardiol 2001; 38 (04) 932-938
    CrossrefPubMedGoogle Scholar
  • 4 Packer M, Bristow MR, Cohn JN. et al; U.S. Carvedilol Heart Failure Study Group. The effect of carvedilol on morbidity and mortality in patients with chronic heart failure. N Engl J Med 1996; 334 (21) 1349-1355
    CrossrefPubMedGoogle Scholar
  • 5 Packer M, Coats AJ, Fowler MB. et al; Carvedilol Prospective Randomized Cumulative Survival Study Group. Effect of carvedilol on survival in severe chronic heart failure. N Engl J Med 2001; 344 (22) 1651-1658
    CrossrefPubMedGoogle Scholar
  • 6 Ponikowski P, Voors AA, Anker SD. et al; ESC Scientific Document Group. 2016 ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure: the Task Force for the diagnosis and treatment of acute and chronic heart failure of the European Society of Cardiology (ESC) Developed with the special contribution of the Heart Failure Association (HFA) of the ESC. Eur Heart J 2016; 37 (27) 2129-2200
    CrossrefPubMedGoogle Scholar
  • 7 McDonagh TA, Metra M, Adamo M. et al; ESC Scientific Document Group. 2021 ESC guidelines for the diagnosis and treatment of acute and chronic heart failure. Eur Heart J 2021; 42 (36) 3599-3726
    Google Scholar
  • 8 Yancy CW, Jessup M, Bozkurt B. et al. 2017 ACC/AHA/HFSA focused update of the 2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines and the Heart Failure Society of America. Circulation 2017; 136 (06) e137-e161
    CrossrefPubMedGoogle Scholar
  • 9 Heidenreich PA, Bozkurt B, Aguilar D. et al. 2022 AHA/ACC/HFSA guideline for the management of heart failure: a report of the American College of Cardiology/American Heart Association Joint Committee on Clinical Practice Guidelines. Circulation 2022; 145 (18) e895-e1032
    CrossrefPubMedGoogle Scholar
  • 10 Greene SJ, Butler J, Albert NM. et al. Medical therapy for heart failure with reduced ejection fraction: the CHAMP-HF registry. J Am Coll Cardiol 2018; 72 (04) 351-366
    CrossrefPubMedGoogle Scholar
  • 11 Allen LA, Venechuk G, McIlvennan CK. et al. An electronically delivered patient-activation tool for intensification of medications for chronic heart failure with reduced ejection fraction: the EPIC-HF trial. Circulation 2021; 143 (05) 427-437
    CrossrefPubMedGoogle Scholar
  • 12 Bhatt AS, Varshney AS, Nekoui M. et al. Virtual optimization of guideline-directed medical therapy in hospitalized patients with heart failure with reduced ejection fraction: the IMPLEMENT-HF pilot study. Eur J Heart Fail 2021; 23 (07) 1191-1201
    CrossrefPubMedGoogle Scholar
  • 13 Get With The Guidelines - Heart Failure | American Heart Association. Accessed April 22, 2022 at: https://www.heart.org/en/professional/quality-improvement/get-with-the-guidelines/get-with-the-guidelines-heart-failure
    PubMed
  • 14 Kao DP, Trinkley KE, Lin CT. Heart failure management innovation enabled by electronic health records. JACC Heart Fail 2020; 8 (03) 223-233
    CrossrefPubMedGoogle Scholar
  • 15 Mukhopadhyay A, Reynolds HR, Phillips LM. et al. Cluster-randomized trial comparing ambulatory decision support tools to improve heart failure care. J Am Coll Cardiol 2023; 81 (14) 1303-1316
    CrossrefPubMedGoogle Scholar
  • 16 Ghazi L, Yamamoto Y, Riello RJ. et al. Electronic alerts to improve heart failure therapy in outpatient practice: a cluster randomized trial. J Am Coll Cardiol 2022; 79 (22) 2203-2213
    CrossrefPubMedGoogle Scholar
  • 17 Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005; 330 (7494): 765
    CrossrefPubMedGoogle Scholar
  • 18 Tierney WM, Overhage JM, Murray MD. et al. Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med 2003; 18 (12) 967-976
    CrossrefPubMedGoogle Scholar
  • 19 Blecker S, Austrian JS, Horwitz LI. et al. Interrupting providers with clinical decision support to improve care for heart failure. Int J Med Inform 2019; 131: 103956
    CrossrefPubMedGoogle Scholar
  • 20 Bates DW, Kuperman GJ, Wang S. et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 10 (06) 523-530
    CrossrefPubMedGoogle Scholar
  • 21 Osheroff J, Teich J, Levick D, Saldana L, Velasco F, Sittig D. Improving Outcomes with Clinical Decision Support: An Implementers Guide. 2nd ed.. Healthcare Information Management Systems Society (HIMSS); 2012
    CrossrefGoogle Scholar
  • 22 Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions. J Biomed Inform 2012; 45 (06) 1202-1216
    CrossrefPubMedGoogle Scholar
  • 23 Sittig DF, Wright A, Osheroff JA. et al. Grand challenges in clinical decision support. J Biomed Inform 2008; 41 (02) 387-392
    CrossrefPubMedGoogle Scholar
  • 24 Trinkley KE, Kahn MG, Bennett TD. et al. An implementation science approach to clinical decision support within electronic health records: integrating PRISM with CDS design best practices (preprint). J Med Internet Res 2020; 22 (10) e19676
    CrossrefPubMedGoogle Scholar
  • 25 Brownson R, Colditz G, Proctor E. Dissemination and Implementation Research in Health: Translating Science to Practice. 2nd ed.. Oxford University Press; 2018
    Google Scholar
  • 26 Kwan BM, Brownson RC, Glasgow RE, Morrato EH, Luke DA. Designing for dissemination and sustainability to promote equitable impacts on health. Annu Rev Public Health 2022; 43 (01) 331-353
    CrossrefPubMedGoogle Scholar
  • 27 Shelton RC, Lee M. Sustaining evidence-based interventions and policies: recent innovations and future directions in implementation science. Am J Public Health 2019; 109 (S2): S132-S134
    CrossrefPubMedGoogle Scholar
  • 28 Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to enhance sustainability: addressing dynamic context and promoting health equity over time. Front Public Health 2020; 8: 134
    CrossrefPubMedGoogle Scholar
  • 29 Tabak RG, Khoong EC, Chambers D, Brownson RC. Models in dissemination and implementation research: useful tools in public health services and systems research. 2013. Accessed March 18, 2015 at: http://uknowledge.uky.edu/cgi/viewcontent.cgi?article=1012&context=frontiersinphssr&sei-redir=1&referer=http://www.bing.com/search?q=Models+in+dissemination+and+implementationresearch%3A+useful+tools+in+public+health+services+andsystems+research&form=DLRD
    PubMed
  • 30 Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med 2012; 43 (03) 337-350
    CrossrefPubMedGoogle Scholar
  • 31 Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci 2015; 10: 53
    CrossrefPubMedGoogle Scholar
  • 32 Feldstein AC, Glasgow RE. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf 2008; 34 (04) 228-243
    PubMedGoogle Scholar
  • 33 Rabin BA, Cakici J, Golden CA, Estabrooks PA, Glasgow RE, Gaglio B. A citation analysis and scoping systematic review of the operationalization of the Practical, Robust Implementation and Sustainability Model (PRISM). Implement Sci 2022; 17 (01) 62
    CrossrefPubMedGoogle Scholar
  • 34 Trinkley KE, Kahn MG, Bennett TD. et al. Integrating the practical robust implementation and sustainability model with best practices in clinical decision support design: implementation science approach. J Med Internet Res 2020; 22 (10) e19676
    CrossrefPubMedGoogle Scholar
  • 35 Trinkley KE, Kroehl ME, Kahn MG. et al. Applying clinical decision support design best practices with the practical robust implementation and sustainability model versus reliance on commercially available clinical decision support tools: randomized controlled trial. JMIR Med Inform 2021; 9 (03) e24359
    CrossrefPubMedGoogle Scholar
  • 36 Glasgow RE, Harden SM, Gaglio B. et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Health 2019; 7 (MAR): 64
    CrossrefPubMedGoogle Scholar
  • 37 Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999; 89 (09) 1322-1327
    CrossrefPubMedGoogle Scholar
  • 38 Lewinski AA, Crowley MJ, Miller C. et al. Applied rapid qualitative analysis to develop a contextually appropriate intervention and increase the likelihood of uptake. Med Care 2021; 59 (Suppl. 03) S242-S251
    CrossrefPubMedGoogle Scholar
  • 39 Desai S, Roberts E. Leveraging natural experiments to evaluate interventions in learning health systems. BMJ Qual Saf 2021; 30 (03) 183-185
    CrossrefPubMedGoogle Scholar
  • 40 Nanji KC, Slight SP, Seger DL. et al. Overrides of medication-related clinical decision support alerts in outpatients. J Am Med Inform Assoc 2014; 21 (03) 487-491
    CrossrefPubMedGoogle Scholar
  • 41 Poly TN, Islam MM, Yang HC, Li YJ. Appropriateness of overridden alerts in computerized physician order entry: systematic review. JMIR Med Inform 2020; 8 (07) e15653
    CrossrefPubMedGoogle Scholar
  • 42 Stettner S, Adie S, Hanigan S, Thomas M, Pogue K, Zimmerman C. Effect of replacing vendor QTc alerts with a custom QTc risk alert in inpatients. Appl Clin Inform 2022; 13 (01) 19-29
    Article in Thieme ConnectPubMedGoogle Scholar
  • 43 Green LA, Nease Jr D, Klinkman MS. Clinical reminders designed and implemented using cognitive and organizational science principles decrease reminder fatigue. J Am Board Fam Med 2015; 28 (03) 351-359
    CrossrefPubMedGoogle Scholar
  • 44 Shakowski C, Page Ii RL, Wright G. et al. Comparative effectiveness of generic commercial versus locally customized clinical decision support tools to reduce prescription of nonsteroidal anti-inflammatory drugs for patients with heart failure. J Am Med Inform Assoc 2023; ocad109
    PubMedGoogle Scholar
  • 45 Rabin B, Brownson R. Developing terminology for dissemination and implementation research. In: Brownson R, Colditz G, Proctor EK. eds. Dissemination and Implementation Research in Health. 2nd ed.. Oxford University Press; 2018: 19-45
    Google Scholar
  • 46 Craig P, Cooper C, Gunnell D. et al. Using natural experiments to evaluate population health interventions: new Medical Research Council guidance. J Epidemiol Community Health 2012; 66 (12) 1182-1186
    CrossrefPubMedGoogle Scholar
  • 47 Nadkarni PM. Drug safety surveillance using de-identified EMR and claims data: issues and challenges. J Am Med Inform Assoc 2010; 17 (06) 671-674
    CrossrefPubMedGoogle Scholar
  • 48 Carrell DS, Schoen RE, Leffler DA. et al. Challenges in adapting existing clinical natural language processing systems to multiple, diverse health care settings. J Am Med Inform Assoc 2017; 24 (05) 986-991
    CrossrefPubMedGoogle Scholar
  • 49 Huebschmann AG, Trinkley KE, Gritz M, Glasgow RE. Pragmatic considerations and approaches for measuring staff time as an implementation cost in health systems and clinics: key issues and applied examples. Implement Sci Commun 2022; 3 (01) 44
    CrossrefPubMedGoogle Scholar
  • 50 Sanyal J, Rubin D, Banerjee I. A weakly supervised model for the automated detection of adverse events using clinical notes. J Biomed Inform 2022; 126: 103969
    CrossrefPubMedGoogle Scholar
  • 51 Dev S, Hoffman TK, Kavalieratos D. et al. Barriers to adoption of mineralocorticoid receptor antagonists in patients with heart failure: a mixed-methods study. J Am Heart Assoc 2016; 5 (03) e002493
    CrossrefPubMedGoogle Scholar