PDF herunterladen
Appl Clin Inform 2021; 12(01): 182-189
DOI: 10.1055/s-0041-1722918
Research Article

Algorithmic Detection of Boolean Logic Errors in Clinical Decision Support Statements

Adam Wright
1   Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, United States
2   Division of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
3   Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States
4   Partners eCare, Partners HealthCare System, Boston, Massachusetts, United States
,
Skye Aaron
2   Division of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
,
Allison B. McCoy
1   Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, United States
,
Robert El-Kareh
5   Department of Medicine, UC San Diego Health, University of California, San Diego, San Diego, California, United States
,
Daniel Fort
6   Center for Outcomes and Health Services Research, Ochsner Health System, New Orleans, Louisiana, United States
,
Steven Z. Kassakian
7   Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, United States
,
Christopher A. Longhurst
5   Department of Medicine, UC San Diego Health, University of California, San Diego, San Diego, California, United States
,
Sameer Malhotra
8   Department of Healthcare Policy and Research, Weill Cornell Medicine, New York, New York, United States
9   Department of Internal Medicine, NewYork-Presbyterian Hospital, New York, New York, United States
,
Dustin S. McEvoy
4   Partners eCare, Partners HealthCare System, Boston, Massachusetts, United States
,
Craig B. Monsen
10   Center for Informatics, Atrius Health, Boston, Massachusetts, United States
,
Richard Schreiber
11   Physician Informatics and Department of Internal Medicine, Geisinger Holy Spirit, Camp Hill, Pennsylvania, United States
,
Asli O. Weitkamp
1   Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, United States
,
DuWayne L. Willett
12   Department of Internal Medicine, University of Texas Southwestern Medical Center, Dallas, Texas, United States
,
Dean F. Sittig
13   School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, Texas, United States
› Institutsangaben Funding Research reported in this publication was supported by the National Library of Medicine of the National Institutes of Health under Award Number R01LM011966. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
› Weitere Informationen
   Lizenzen und Reprints

Abstract

Objective Clinical decision support (CDS) can contribute to quality and safety. Prior work has shown that errors in CDS systems are common and can lead to unintended consequences. Many CDS systems use Boolean logic, which can be difficult for CDS analysts to specify accurately. We set out to determine the prevalence of certain types of Boolean logic errors in CDS statements.

Methods Nine health care organizations extracted Boolean logic statements from their Epic electronic health record (EHR). We developed an open-source software tool, which implemented the Espresso logic minimization algorithm, to identify three classes of logic errors.

Results Participating organizations submitted 260,698 logic statements, of which 44,890 were minimized by Espresso. We found errors in 209 of them. Every participating organization had at least two errors, and all organizations reported that they would act on the feedback.

Discussion An automated algorithm can readily detect specific categories of Boolean CDS logic errors. These errors represent a minority of CDS errors, but very likely require correction to avoid patient safety issues. This process found only a few errors at each site, but the problem appears to be widespread, affecting all participating organizations.

Conclusion Both CDS implementers and EHR vendors should consider implementing similar algorithms as part of the CDS authoring process to reduce the number of errors in their CDS interventions.

Keywords

clinical decision support - electronic health records and systems - alerting - decision support algorithm - efficiency improvement

Protection of Human and Animal Subjects

Human and/or animal subjects were not included in this project.


Authors' Contributions

A.W. wrote the manuscript and conceived the study design. S.A. wrote the logic minimization program. D.F. acted as a tester for the logic minimization program. All authors provided data. All authors provided critical revisions for important intellectual content.




Publikationsverlauf

Eingereicht: 11. August 2020

Angenommen: 16. Dezember 2020

Artikel online veröffentlicht:
10. März 2021

© 2021. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003; 163 (12) 1409-1416
    CrossrefPubMedGoogle Scholar
  • 2 Garg AX, Adhikari NK, McDonald H. et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005; 293 (10) 1223-1238
    CrossrefPubMedGoogle Scholar
  • 3 Mishuris RG, Linder JA, Bates DW, Bitton A. Using electronic health record clinical decision support is associated with improved quality of care. Am J Manag Care 2014; 20 (10) e445-e452
    PubMedGoogle Scholar
  • 4 Nuckols TK, Smith-Spangler C, Morton SC. et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 2014; 3: 56
    CrossrefPubMedGoogle Scholar
  • 5 Melton BL. Systematic review of medical informatics-supported medication decision making. Biomed Inform Insights 2017; 9: 1178222617697975
    CrossrefPubMedGoogle Scholar
  • 6 Wright A, Ash JS, Aaron S. et al. Best practices for preventing malfunctions in rule-based clinical decision support alerts and reminders: results of a Delphi study. Int J Med Inform 2018; 118: 78-85
    CrossrefPubMedGoogle Scholar
  • 7 Wright A, Wright AP, Aaron S, Sittig DF. Smashing the strict hierarchy: three cases of clinical decision support malfunctions involving carvedilol. J Am Med Inform Assoc 2018; 25 (11) 1552-1555
    CrossrefPubMedGoogle Scholar
  • 8 Wright A, Hickman TT, McEvoy D. et al. Methods for detecting malfunctions in clinical decision support systems. Stud Health Technol Inform 2017; 245: 1385
    PubMedGoogle Scholar
  • 9 Wright A, Ai A, Ash J. et al. Clinical decision support alert malfunctions: analysis and empirically derived taxonomy. J Am Med Inform Assoc 2018; 25 (05) 496-506
    CrossrefPubMedGoogle Scholar
  • 10 Wright A, Hickman TT, McEvoy D. et al. Analysis of clinical decision support system malfunctions: a case series and survey. J Am Med Inform Assoc 2016; 23 (06) 1068-1076
    CrossrefPubMedGoogle Scholar
  • 11 McCoy AB, Wright A, Sittig DF. Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development. J Am Med Inform Assoc 2015; 22 (05) 1081-1088
    CrossrefPubMedGoogle Scholar
  • 12 Kong G, Xu D-L, Yang J-B. Clinical decision support systems: a review on knowledge representation and inference under uncertainties. Intl J Comput Intel Syst 2008; 1 (02) 159-167
    PubMedGoogle Scholar
  • 13 Grohskopf LA, Sokolow LZ, Broder KR, Walter EB, Fry AM, Jernigan DB. Prevention and control of seasonal influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices-United States, 2018-19 Influenza Season. MMWR Recomm Rep 2018; 67 (03) 1-20
    CrossrefPubMedGoogle Scholar
  • 14 Whitesitt JE. Boolean Algebra and Its Applications. Reading, MA: Addison-Wesley Publishing Company, Inc.; 1961
    Google Scholar
  • 15 Herman GL, Loui MC, Kaczmarczyk L, Zilles C. Describing the what and why of students' difficulties in Boolean logic. ACM Trans Comput Edu 2012; 12 (01) 1-28
    CrossrefPubMedGoogle Scholar
  • 16 Pane J, Myers B. eds. Tabular and Textual Methods for Selecting Objects from a Group. IEEE International Symposium on Visual Languages; Seattle, WA: IEEE; 2000
    PubMedGoogle Scholar
  • 17 Almstrum VL. Investigating student difficulties with mathematical logic. In: Hinchey MG, Dean CN. eds. Teaching and Learning Formal Methods. San Diego, CA: Academic Press; 1996: 131-160
    Google Scholar
  • 18 Grätzer G. Lattice Theory: First Concepts and Distributive Lattices. San Francisco: W. H. Freeman; 1971
    Google Scholar
  • 19 McCoy AB, Wright A, Sittig DF. Multi-institutional, large-scale, international applied clinical informatics research through the Clinical Informatics Research Collaborative (CIRCLE). Stud Health Technol Inform 2019; 264: 1730-1731
    PubMedGoogle Scholar
  • 20 Wright A, Aaron S, Sittig DF. Testing electronic health records in the “production” environment: an essential step in the journey to a safe and effective health care system. J Am Med Inform Assoc 2017; 24 (01) 188-192
    CrossrefPubMedGoogle Scholar
  • 21 Drake C. Python Electronic Design Automation. Accessed May 30, 2019 at: https://github.com/cjdrake/pyeda
    PubMedGoogle Scholar
  • 22 Brayton RK, Sangiovanni-Vincentelli AL, McMullen CT, Hachtel GD. Logic Minimization Algorithms for VLSI Synthesis. Norwell, MA: Kluwer Academic Publishers; 1984
    CrossrefGoogle Scholar
  • 23 Rudell RL, Sangiovanni-Vincentelli AL. Multiple-valued minimization for PLA optimization. IEEE Trans Comput Aided Des Integrated Circ Syst 1987; 6 (05) 727-750
    CrossrefPubMedGoogle Scholar
  • 24 Stone EG. Unintended adverse consequences of a clinical decision support system: two cases. J Am Med Inform Assoc 2018; 25 (05) 564-567
    CrossrefPubMedGoogle Scholar
  • 25 Chaparro JD, Hussain C, Lee JA, Hehmeyer J, Nguyen M, Hoffman J. Reducing interruptive alert burden using quality improvement methodology. Appl Clin Inform 2020; 11 (01) 46-58
    Artikel in Thieme ConnectPubMedGoogle Scholar
  • 26 McGreevey III JD, Mallozzi CP, Perkins RM, Shelov E, Schreiber R. Reducing alert burden in electronic health records: state of the art recommendations from four health systems. Appl Clin Inform 2020; 11 (01) 1-12
    Artikel in Thieme ConnectPubMedGoogle Scholar
  • 27 Johnson SC. Lint, a C Program Checker. Computer Science Technical Reports. 1978
    PubMedGoogle Scholar
  • 28 Lomotan EA, Meadows G, Michaels M, Michel JJ, Miller K. To Share is Human! Advancing evidence into practice through a national repository of interoperable clinical decision support. Appl Clin Inform 2020; 11 (01) 112-121
    Artikel in Thieme ConnectPubMedGoogle Scholar
  • 29 Bergeretti J-F, Carre BA. Information-flow and data-flow analysis of while-programs. ACM Trans Prog Lang Syst 1985; 7 (01) 37-61 (TOPLAS)
    CrossrefPubMedGoogle Scholar
  • 30 Li D, Endle CM, Murthy S. et al. Modeling and executing electronic health records driven phenotyping algorithms using the NQF Quality Data Model and JBoss® Drools Engine. AMIA Annu Symp Proc 2012; 2012: 532-541
    PubMedGoogle Scholar
  • 31 Allen FE, Corporation I. eds. Control Flow Analysis. Proceedings of a Symposium on Compiler Optimization; 1970 Urbana-Champaign, IL:
    PubMedGoogle Scholar
  • 32 Van Dort BA, Zheng WY, Baysari MT. Prescriber perceptions of medication-related computerized decision support systems in hospitals: a synthesis of qualitative research. Int J Med Inform 2019; 129: 285-295
    CrossrefPubMedGoogle Scholar