Download PDF
CC BY-NC-ND 4.0 · Appl Clin Inform 2024; 15(03): 637-649
DOI: 10.1055/s-0044-1787184
Research Article

Interprofessional Evaluation of a Medication Clinical Decision Support System Prior to Implementation

Jacqueline Bauer
1   Pharmacy Department, Universitätsklinikum Erlangen and Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
,
Marika Busse
1   Pharmacy Department, Universitätsklinikum Erlangen and Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
,
Tanja Kopetzky
2   Medical Center for Information and Communication Technology (MIK), Universitätsklinikum Erlangen and Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
,
Christof Seggewies
2   Medical Center for Information and Communication Technology (MIK), Universitätsklinikum Erlangen and Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
,
Martin F. Fromm
3   Institute of Experimental and Clinical Pharmacology and Toxicology, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
4   FAU NeW—Research Center New Bioactive Compounds, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
,
Frank Dörje
1   Pharmacy Department, Universitätsklinikum Erlangen and Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
4   FAU NeW—Research Center New Bioactive Compounds, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany
› Author Affiliations Funding None.
› Further Information
Also available at
   Permissions and Reprints

Abstract

Background Computerized physician order entry (CPOE) and clinical decision support systems (CDSS) are widespread due to increasing digitalization of hospitals. They can be associated with reduced medication errors and improved patient safety, but also with well-known risks (e.g., overalerting, nonadoption).

Objectives Therefore, we aimed to evaluate a commonly used CDSS containing Medication-Safety-Validators (e.g., drug–drug interactions), which can be locally activated or deactivated, to identify limitations and thereby potentially optimize the use of the CDSS in clinical routine.

Methods Within the implementation process of Meona (commercial CPOE/CDSS) at a German University hospital, we conducted an interprofessional evaluation of the CDSS and its included Medication-Safety-Validators following a defined algorithm: (1) general evaluation, (2) systematic technical and content-related validation, (3) decision of activation or deactivation, and possibly (4) choosing the activation mode (interruptive or passive). We completed the in-depth evaluation for exemplarily chosen Medication-Safety-Validators. Moreover, we performed a survey among 12 German University hospitals using Meona to compare their configurations.

Results Based on the evaluation, we deactivated 3 of 10 Medication-Safety-Validators due to technical or content-related limitations. For the seven activated Medication-Safety-Validators, we chose the interruptive option [“PUSH-(&PULL)-modus”] four times (4/7), and a new, on-demand option [“only-PULL-modus”] three times (3/7). The site-specific configuration (activation or deactivation) differed across all participating hospitals in the survey and led to varying medication safety alerts for identical patient cases.

Conclusion An interprofessional evaluation of CPOE and CDSS prior to implementation in clinical routine is crucial to detect limitations. This can contribute to a sustainable utilization and thereby possibly increase medication safety.

Keywords

clinical decision support system - clinical pharmacists - evaluation - medication safety - alert fatigue

Note

This study was conducted in collaboration with members of the working group “Medication-Safety-Validators” of the Medicines Management Board of the Erlangen University Hospital.


Protection of Human and Animal Subjects

In this project no human and/or animal subjects were included.


Authors' Contributions

J.B., F.D., and M.F.F. designed the research.


All authors are responsible for all aspects of the presented work.


J.B. analyzed all data; M.B. and T.K. verified the data analyses.


J.B. and F.D. wrote the first version of the manuscript; M.F.F, M.B., T.K., and, C.S. revised the manuscript for important content.


All authors finally approved the manuscript.


Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.


Supplementary Material



Publication History

Received: 15 January 2024

Accepted: 01 April 2024

Article published online:
31 July 2024

© 2024. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Institute of Medicine (US) Committee on Quality of Health Care in America. To Err is Human: Building a Safer Health System. Washington, DC: National Academies Press (US); 2000
    Search in Google Scholar
  • 2 Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med 2020; 3: 17
    CrossrefPubMedSearch in Google Scholar
  • 3 Pedersen CA, Schneider PJ, Scheckelhoff DJ. ASHP national survey of pharmacy practice in hospital settings: prescribing and transcribing-2016. Am J Health Syst Pharm 2017; 74 (17) 1336-1352
    CrossrefPubMedSearch in Google Scholar
  • 4 Agency for Healthcare Research and Quality. Clinical Decision Support Systems. Updated 07.09.2019. Accessed August 11, 2023 at: https://psnet.ahrq.gov/primer/clinical-decision-support-systems
    PubMed
  • 5 Office of the National Coordinator for Health Information Technology. National Trends in Hospital and Physician Adoption of Electronic Health Records. 06.04.2022. Accessed August 11, 2023 at: https://www.healthit.gov/data/quickstats/national-trends-hospital-and-physician-adoption-electronic-health-records
    PubMed
  • 6 Blum DK, Löffert DS, Offermanns DM, Steffen DP. Krankenhaus Barometer 2016. 19.12.2016. Accessed August 11, 2023 at: https://www.dki.de/sites/default/files/2019-01/2016_12_19_kh_barometer_final.pdf
    PubMed
  • 7 Blum DK, Löffert DS, Offermanns DM, Steffen DP. Krankenhaus Barometer. 2018. 12.2018. Accessed May 26, 2024 at: https://www.dki.de/fileadmin/forschungsberichte/2018_11_kh_barometer_final.pdf
    PubMed
  • 8 Agency for Healthcare Research and Quality. Computerized provider order entry. Updated 07.09.2019. Accessed August 11, 2023 at: https://psnet.ahrq.gov/primer/computerized-provider-order-entry
    PubMed
  • 9 Hak F, Guimarães T, Santos M. Towards effective clinical decision support systems: A systematic review. PLoS One 2022; 17 (08) e0272846
    CrossrefPubMedSearch in Google Scholar
  • 10 Wasylewicz ATM, Scheepers-Hoeks A. Clinical Decision Support Systems. In: Kubben P, Dumontier M, Dekker A. eds. Fundamentals of Clinical Data Science. Cham: Springer; 2019: 153-169
    Search in Google Scholar
  • 11 Sim I, Gorman P, Greenes RA. et al. Clinical decision support systems for the practice of evidence-based medicine. J Am Med Inform Assoc 2001; 8 (06) 527-534
    CrossrefPubMedSearch in Google Scholar
  • 12 Bates DW, Teich JM, Lee J. et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999; 6 (04) 313-321
    CrossrefPubMedSearch in Google Scholar
  • 13 Vélez-Díaz-Pallarés M, Pérez-Menéndez-Conde C, Bermejo-Vicedo T. Systematic review of computerized prescriber order entry and clinical decision support. Am J Health Syst Pharm 2018; 75 (23) 1909-1921
    CrossrefPubMedSearch in Google Scholar
  • 14 Garg AX, Adhikari NK, McDonald H. et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005; 293 (10) 1223-1238
    CrossrefPubMedSearch in Google Scholar
  • 15 Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assoc 2008; 15 (05) 585-600
    CrossrefPubMedSearch in Google Scholar
  • 16 Wolfstadt JI, Gurwitz JH, Field TS. et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008; 23 (04) 451-458
    CrossrefPubMedSearch in Google Scholar
  • 17 Nuckols TK, Smith-Spangler C, Morton SC. et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 2014; 3: 56
    CrossrefPubMedSearch in Google Scholar
  • 18 Jia P, Zhang L, Chen J, Zhao P, Zhang M. The effects of clinical decision support systems on medication safety: an overview. PLoS One 2016; 11 (12) e0167683
    CrossrefPubMedSearch in Google Scholar
  • 19 Zaal RJ, Jansen MM, Duisenberg-van Essenberg M, Tijssen CC, Roukema JA, van den Bemt PM. Identification of drug-related problems by a clinical pharmacist in addition to computerized alerts. Int J Clin Pharm 2013; 35 (05) 753-762
    CrossrefPubMedSearch in Google Scholar
  • 20 Nanji KC, Seger DL, Slight SP. et al. Medication-related clinical decision support alert overrides in inpatients. J Am Med Inform Assoc 2018; 25 (05) 476-481
    CrossrefPubMedSearch in Google Scholar
  • 21 Ranji SR, Rennke S, Wachter RM. Computerised provider order entry combined with clinical decision support systems to improve medication safety: a narrative review. BMJ Qual Saf 2014; 23 (09) 773-780
    CrossrefPubMedSearch in Google Scholar
  • 22 van der Sijs H, Mulder A, van Gelder T, Aarts J, Berg M, Vulto A. Drug safety alert generation and overriding in a large Dutch university medical centre. Pharmacoepidemiol Drug Saf 2009; 18 (10) 941-947
    CrossrefPubMedSearch in Google Scholar
  • 23 Abell B, Naicker S, Rodwell D. et al. Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review. Implement Sci 2023; 18 (01) 32
    CrossrefPubMedSearch in Google Scholar
  • 24 Ammenwerth E, Aly AF, Bürkle T. et al. Memorandum on the use of information technology to improve medication safety. Methods Inf Med 2014; 53 (05) 336-343
    Thieme ConnectPubMedSearch in Google Scholar
  • 25 The Office of the National Coordinator for Health Information Technology. SAFER Self-Assessment. Computerized Provider Order Entry with Decision Support. Updated 11.2016. Accessed August 12, 2023 at: https://www.healthit.gov/sites/default/files/safer/guides/safer_cpoe.pdf
    PubMed
  • 26 Tse G, Algaze C, Pageler N, Wood M, Chadwick W. Using clinical decision support systems to decrease intravenous acetaminophen use: implementation and lessons learned. Appl Clin Inform 2024; 15 (01) 64-74
    Thieme ConnectPubMedSearch in Google Scholar
  • 27 E Dawson T, Beus J, W Orenstein E, Umontuen U, McNeill D, Kandaswamy S. Reducing therapeutic duplication in inpatient medication orders. Appl Clin Inform 2023; 14 (03) 538-543
    Thieme ConnectPubMedSearch in Google Scholar
  • 28 Knight AM, Maygers J, Foltz KA, John IS, Yeh HC, Brotman DJ. The effect of eliminating intermediate severity drug-drug interaction alerts on overall medication alert burden and acceptance rate. Appl Clin Inform 2019; 10 (05) 927-934
    Thieme ConnectPubMedSearch in Google Scholar
  • 29 Rehr CA, Wong A, Seger DL, Bates DW. Determining inappropriate medication alerts from “inaccurate warning” overrides in the intensive care unit. Appl Clin Inform 2018; 9 (02) 268-274
    Thieme ConnectPubMedSearch in Google Scholar
  • 30 Meslin SMM, Zheng WY, Day RO, Tay EMY, Baysari MT. Evaluation of clinical relevance of drug-drug interaction alerts prior to implementation. Appl Clin Inform 2018; 9 (04) 849-855
    Thieme ConnectPubMedSearch in Google Scholar
  • 31 Mesalvo Freiburg GmbH Accessed August 11, 2023 at: https://www.meona.de/
    PubMed
  • 32 Wright A, Aaron S, Sittig DF. Testing electronic health records in the “production” environment: an essential step in the journey to a safe and effective health care system. J Am Med Inform Assoc 2017; 24 (01) 188-192
    CrossrefPubMedSearch in Google Scholar
  • 33 University Medical Center of Erlangen. Number and facts. Updated 02.01.2023. Accessed August 12, 2023 at: https://www.uk-erlangen.de/presse/zahlen-und-fakten/
    PubMed
  • 34 Van de Velde S, Kunnamo I, Roshanov P. et al; GUIDES expert panel. The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support. Implement Sci 2018; 13 (01) 86
    CrossrefPubMedSearch in Google Scholar
  • 35 Zheng WY, Van Dort B, Marcilly R. et al. A tool for evaluating medication alerting systems: development and initial assessment. JMIR Med Inform 2021; 9 (07) e24022
    CrossrefPubMedSearch in Google Scholar
  • 36 Kilbridge PM, Welebob EM, Classen DC. Development of the Leapfrog methodology for evaluating hospital implemented inpatient computerized physician order entry systems. Qual Saf Health Care 2006; 15 (02) 81-84
    CrossrefPubMedSearch in Google Scholar
  • 37 Zachariah M, Phansalkar S, Seidling HM. et al. Development and preliminary evidence for the validity of an instrument assessing implementation of human-factors principles in medication-related decision-support systems–I-MeDeSA. J Am Med Inform Assoc 2011; 18 (suppl 1): i62-i72
    CrossrefPubMedSearch in Google Scholar
  • 38 Leiner DJ. SoSci Survey [Computer software]. 2019 . Accessed May 9, 2024 at: https://www.soscisurvey.de
    PubMedSearch in Google Scholar
  • 39 Amkreutz J, Koch A, Buendgens L, Muehlfeld A, Trautwein C, Eisert A. Prevalence and nature of potential drug-drug interactions among kidney transplant patients in a German intensive care unit. Int J Clin Pharm 2017; 39 (05) 1128-1139
    CrossrefPubMedSearch in Google Scholar
  • 40 Amkreutz J, Koch A, Buendgens L, Trautwein C, Eisert A. Clinical decision support systems differ in their ability to identify clinically relevant drug interactions of immunosuppressants in kidney transplant patients. J Clin Pharm Ther 2017; 42 (03) 276-285
    CrossrefPubMedSearch in Google Scholar
  • 41 Seiberth S, Mannell H, Birkenmaier C. et al. Benefit of medication reviews by renal pharmacists in the setting of a computerized physician order entry system with clinical decision support. J Clin Pharm Ther 2022; 47 (10) 1531-1538
    CrossrefPubMedSearch in Google Scholar
  • 42 AiDKlinik. Accessed August 20, 2023 at: https://www.dosing-gmbh.de/produktloesungen/aidklinik-2/
    PubMed
  • 43 Bittmann JA, Rein EK, Metzner M, Haefeli WE, Seidling HM. The acceptance of interruptive medication alerts in an electronic decision support system differs between different alert types. Methods Inf Med 2021; 60 (5-06): 180-184
    Thieme ConnectPubMedSearch in Google Scholar
  • 44 Seidling HM, Schmitt SP, Bruckner T. et al. Patient-specific electronic decision support reduces prescription of excessive doses. Qual Saf Health Care 2010; 19 (05) e15
    PubMedSearch in Google Scholar
  • 45 Pauly A, Wolf C, Busse M. et al. Evaluation of eight drug interaction databases commonly used in the German healthcare system. Eur J Hosp Pharm 2015; 22: 165-170
    CrossrefPubMedSearch in Google Scholar
  • 46 McCoy AB, Waitman LR, Lewis JB. et al. A framework for evaluating the appropriateness of clinical decision support alerts and responses. J Am Med Inform Assoc 2012; 19 (03) 346-352
    CrossrefPubMedSearch in Google Scholar
  • 47 Humphrey KE, Mirica M, Phansalkar S, Ozonoff A, Harper MB. Clinician perceptions of timing and presentation of drug-drug interaction alerts. Appl Clin Inform 2020; 11 (03) 487-496
    Thieme ConnectPubMedSearch in Google Scholar
  • 48 Poly TN, Islam MM, Yang HC, Li YJ. Appropriateness of overridden alerts in computerized physician order entry: systematic review. JMIR Med Inform 2020; 8 (07) e15653
    CrossrefPubMedSearch in Google Scholar
  • 49 Bittmann JA, Haefeli WE, Seidling HM. Modulators influencing medication alert acceptance: an explorative review. Appl Clin Inform 2022; 13 (02) 468-485
    Thieme ConnectPubMedSearch in Google Scholar
  • 50 Wright A, McEvoy DS, Aaron S. et al. Structured override reasons for drug-drug interaction alerts in electronic health records. J Am Med Inform Assoc 2019; 26 (10) 934-942
    CrossrefPubMedSearch in Google Scholar
  • 51 Coleman JJ, van der Sijs H, Haefeli WE. et al. On the alert: future priorities for alerts in clinical decision support for computerized physician order entry identified from a European workshop. BMC Med Inform Decis Mak 2013; 13: 111
    CrossrefPubMedSearch in Google Scholar
  • 52 Simpao AF, Ahumada LM, Desai BR. et al. Optimization of drug-drug interaction alert rules in a pediatric hospital's electronic health record system using a visual analytics dashboard. J Am Med Inform Assoc 2015; 22 (02) 361-369
    CrossrefPubMedSearch in Google Scholar
  • 53 Phansalkar S, van der Sijs H, Tucker AD. et al. Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records. J Am Med Inform Assoc 2013; 20 (03) 489-493
    CrossrefPubMedSearch in Google Scholar
  • 54 Wright A, Ash JS, Erickson JL. et al. A qualitative study of the activities performed by people involved in clinical decision support: recommended practices for success. J Am Med Inform Assoc 2014; 21 (03) 464-472
    CrossrefPubMedSearch in Google Scholar
  • 55 Zhang T, Gephart SM, Subbian V. et al. Barriers to adoption of tailored drug-drug interaction clinical decision support. Appl Clin Inform 2023; 14 (04) 779-788
    Thieme ConnectPubMedSearch in Google Scholar
  • 56 Seidling HM, Stützle M, Hoppe-Tichy T. et al. Best practice strategies to safeguard drug prescribing and drug administration: an anthology of expert views and opinions. Int J Clin Pharm 2016; 38 (02) 362-373
    CrossrefPubMedSearch in Google Scholar
  • 57 Co Z, Holmgren AJ, Classen DC. et al. The development and piloting of the ambulatory electronic health record evaluation tool: lessons learned. Appl Clin Inform 2021; 12 (01) 153-163
    Thieme ConnectPubMedSearch in Google Scholar
  • 58 Holmgren AJ, Kuznetsova M, Classen D, Bates DW. Assessing hospital electronic health record vendor performance across publicly reported quality measures. J Am Med Inform Assoc 2021; 28 (10) 2101-2107
    CrossrefPubMedSearch in Google Scholar
  • 59 Aarts J, Berg M. Same systems, different outcomes–comparing the implementation of computerized physician order entry in two Dutch hospitals. Methods Inf Med 2006; 45 (01) 53-61
    Thieme ConnectPubMedSearch in Google Scholar
  • 60 By the 2023 American Geriatrics Society Beers Criteria Update Expert Panel. American Geriatrics Society 2023 updated AGS Beers Criteria for potentially inappropriate medication use in older adults. J Am Geriatr Soc 2023; 71 (07) 2052-2081
    CrossrefPubMedSearch in Google Scholar
  • 61 O'Mahony D, Cherubini A, Guiteras AR. et al. STOPP/START criteria for potentially inappropriate prescribing in older people: version 3. Eur Geriatr Med 2023; 14 (04) 625-632
    CrossrefPubMedSearch in Google Scholar