Towards a Maturity Model for Clinical Decision Support Operations

Evan W. Orenstein; Naveen Muthu; Asli O. Weitkamp; Daria F. Ferro; Mike D. Zeidlhack; Jason Slagle; Eric Shelov; Marc C. Tobias

doi:10.1055/s-0039-1697905

Applied Clinical Informatics, Table of Contents

Appl Clin Inform 2019; 10(05): 810-819
DOI: 10.1055/s-0039-1697905

State of the Art/Best Practice Paper

Georg Thieme Verlag KG Stuttgart · New York

Towards a Maturity Model for Clinical Decision Support Operations

Authors

Evan W. Orenstein

¹Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia, United States

²Division of Hospital Medicine, Children's Healthcare of Atlanta, Atlanta, Georgia, United States
Naveen Muthu

³Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

⁴Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
Asli O. Weitkamp

⁵Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee, United States
Daria F. Ferro

³Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

⁴Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
Mike D. Zeidlhack

⁶Phrase Health Inc., Philadelphia, Pennsylvania, United States
Jason Slagle

⁵Department of Biomedical Informatics, Vanderbilt University School of Medicine, Nashville, Tennessee, United States
Eric Shelov

³Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States

⁴Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States
Marc C. Tobias

⁶Phrase Health Inc., Philadelphia, Pennsylvania, United States

Abstract

Clinical decision support (CDS) systems delivered through the electronic health record are an important element of quality and safety initiatives within a health care system. However, managing a large CDS knowledge base can be an overwhelming task for informatics teams. Additionally, it can be difficult for these informatics teams to communicate their goals with external operational stakeholders and define concrete steps for improvement. We aimed to develop a maturity model that describes a roadmap toward organizational functions and processes that help health care systems use CDS more effectively to drive better outcomes. We developed a maturity model for CDS operations through discussions with health care leaders at 80 organizations, iterative model development by four clinical informaticists, and subsequent review with 19 health care organizations. We ceased iterations when feedback from three organizations did not result in any changes to the model. The proposed CDS maturity model includes three main “pillars”: “Content Creation,” “Analytics and Reporting,” and “Governance and Management.” Each pillar contains five levels—advancing along each pillar provides CDS teams a deeper understanding of the processes CDS systems are intended to improve. A “roof” represents the CDS functions that become attainable after advancing along each of the pillars. Organizations are not required to advance in order and can develop in one pillar separately from another. However, we hypothesize that optimal deployment of preceding levels and advancing in tandem along the pillars increase the value of organizational investment in higher levels of CDS maturity. In addition to describing the maturity model and its development, we also provide three case studies of health care organizations using the model for self-assessment and determine next steps in CDS development.

Keywords

decision support - clinical informatics - health information

Full Text

References

References
1 Bright TJ, Wong A, Dhurjati R. , et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012; 157 (01) 29-43
2 Roshanov PS, You JJ, Dhaliwal J. , et al; CCDSS Systematic Review Team. Can computerized clinical decision support systems improve practitioners' diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review. Implement Sci 2011; 6 (88) 88
3 Nieuwlaat R, Connolly SJ, Mackay JA. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for therapeutic drug monitoring and dosing: a decision-maker-researcher partnership systematic review. Implement Sci 2011; 6: 90
4 Souza NM, Sebaldt RJ, Mackay JA. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for primary preventive care: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci 2011; 6 (87) 87
5 Sahota N, Lloyd R, Ramakrishna A. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for acute care management: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci 2011; 6 (91) 91
6 Hemens BJ, Holbrook A, Tonkin M. , et al; CCDSS Systematic Review Team. Computerized clinical decision support systems for drug prescribing and management: a decision-maker-researcher partnership systematic review. Implement Sci 2011; 6 (89) 89
7 Roshanov PS, Fernandes N, Wilczynski JM. , et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ 2013; 346: f657
8 Osheroff JA, Teich JM, Levick D. , et al. Improving Outcomes with Clinical Decision Support: an Implementer's Guide. 2nd ed. Chicago, IL: HIMSS; 2012
9 Van de Velde S, Kunnamo I, Roshanov P. , et al; GUIDES expert panel. The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support. Implement Sci 2018; 13 (01) 86
10 Greenes RA, Bates DW, Kawamoto K, Middleton B, Osheroff J, Shahar Y. Clinical decision support models and frameworks: Seeking to address research issues underlying implementation successes and failures. J Biomed Inform 2018; 78: 134-143
11 Millard Krause T. CMIO/CHIO Summit: managing clinical decision support and improving workflow. Healthcare Informatics Magazine (2016). Available at: https://www.healthcare-informatics.com/article/ehr/cmiochio-summit-managing-clinical-decision-support-and-improving-workflow . Accessed November 9, 2018
12 Carvalho JV, Rocha Á, Abreu A. Maturity models of healthcare information systems and technologies: a literature review. J Med Syst 2016; 40 (06) 131
13 Knosp BM, Barnett WK, Anderson NR, Embi PJ. Research IT maturity models for academic health centers: early development and initial evaluation. J Clin Transl Sci 2018; 2 (05) 289-294
14 HIMSS Analytics - North America. Electronic Medical Record Adoption Model. Available at: https://www.himssanalytics.org/emram . Accessed November 12, 2018
15 Analytics HIMSS. Continuity of Care Maturity Model. Available at: https://www.himssanalytics.org/ccmm . Accessed: November 23, 2018
16 van de Wetering R, Batenburg R. Towards a theory of PACS deployment: an integrative PACS maturity framework. J Digit Imaging 2014; 27 (03) 337-350
17 Corrigan D, McDonnell R, Zarabzadeh A, Fahey T. A multistep maturity model for the implementation of electronic and computable diagnostic clinical prediction rules (eCPRs). EGEMS (Wash DC) 2015; 3 (02) 1153
18 Staggers N, Rodney M, Alafaireet P. , et al. Promoting Usability in Health Organizations: Initial Steps and Progress Toward a Healthcare Usability Maturity Model. Chicago, IL: Health Information Management Systems Society; 2011: 1-55
19 Innovation Pathways Maturity Model HIMSS. Available at: https://www.himss.org/library/innovation/pathways-maturity-model . Accessed November 23, 2018
20 Lipsitz LA. Understanding health care as a complex system: the foundation for unintended consequences. JAMA 2012; 308 (03) 243-244
21 Ash JS, Smith III AC, Stavri PZ. Performing subjectivist studies in the qualitative traditions responsive to users. In: Friedman CP, Wyatt JC. , eds. Evaluation Methods in Medical Informatics. 2nd ed. New York, NY: Springer-Verlag; 2006: 267-300
22 Marshall MN. Sampling for qualitative research. Fam Pract 1996; 13 (06) 522-525
23 Tobias MC, Orenstein EW, Muthu N. Decision support for decision support: a novel system to prioritize improvement efforts, identify safety hazards, and measure improvement. AMIA Annu Symp Proc 2018; 2018: 2246
24 Orenstein EW, Weitkamp AO, Rosenau P, Mallozzi C, Tobias MC. Towards a maturity model for clinical decision support operations: an interactive panel. In: American Medical Informatics Association Clinical Informatics Conference; April 30–May 2, 2019; Atlanta, GA
25 Carayon P, Schoofs Hundt A, Karsh BT. , et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care 2006; 15 (Suppl. 01) i50-i58
26 Holden RJ, Carayon P, Gurses AP. , et al. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics 2013; 56 (11) 1669-1686
27 Jones BE, Collingridge DS, Vines CG. , et al. CDS in a learning health care system: identifying physicians' reasons for rejection of best-practice recommendations in pneumonia through computerized clinical decision support. Appl Clin Inform 2019; 10 (01) 1-9
28 Aaron S, McEvoy DS, Ray S, Hickman TTT, Wright A. Cranky comments: detecting clinical decision support malfunctions through free-text override reasons. J Am Med Inform Assoc 2019; 26 (01) 37-43
29 Chan J, Shojania KG, Easty AC, Etchells EE. Does user-centred design affect the efficiency, usability and safety of CPOE order sets?. J Am Med Inform Assoc 2011; 18 (03) 276-281
30 Wright A, Hickman TT, McEvoy D. , et al. Analysis of clinical decision support system malfunctions: a case series and survey. J Am Med Inform Assoc 2016; 23 (06) 1068-1076
31 Yoshida E, Fei S, Bavuso K, Lagor C, Maviglia S. The value of monitoring clinical decision support interventions. Appl Clin Inform 2018; 9 (01) 163-173
32 Kassakian SZ, Yackel TR, Gorman PN, Dorr DA. Clinical decisions support malfunctions in a commercial electronic health record. Appl Clin Inform 2017; 8 (03) 910-923
33 Thayer JG, Miller JM, Fiks AG, Tague L, Grundmeier RW. Assessing the safety of custom web-based clinical decision support systems in electronic health records: a case study. Appl Clin Inform 2019; 10 (02) 237-246
34 Institute of Medicine. The Learning Healthcare System: Workshop Summary. Washington, DC: National Academies Press; 2007
35 Lindgren H, Lu MH, Hong Y, Yan C. Applying the zone of proximal development when evaluating clinical decision support systems: a case study. Stud Health Technol Inform 2018; 247: 131-135
36 Wright A, Ai A, Ash J. , et al. Clinical decision support alert malfunctions: analysis and empirically derived taxonomy. J Am Med Inform Assoc 2018; 25 (05) 496-506
37 Nielsen J. Corporate UX maturity: stages 5–8. 2006. Available at: https://www.nngroup.com/articles/ux-maturity-stages-5-8/ . Accessed November 23, 2018
38 Wright A, Feblowitz JC, Pang JE. , et al. Use of order sets in inpatient computerized provider order entry systems: a comparative analysis of usage patterns at seven sites. Int J Med Inform 2012; 81 (11) 733-745
39 Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev 2009; 8 (03) CD001096
40 Mutafelija B, Stromberg H. Systematic Process Improvement Using ISO 9001:2000 and CMMI. Norwood, MA: Artech House; 2003
41 Sanders D, Burton DA, Prott D. The healthcare analytics adoption model: a framework and roadmap. 2016 . Available at: https://downloads.healthcatalyst.com/wp-content/uploads/2013/11/analytics-adoption-model-Nov-2013.pdf . Accessed September 2, 2019
42 Tarakci M, Ates NY, Porck JP, van Knippenberg D, Groenen PJF, de Haas M. Strategic consensus mapping: a new method for testing and visualizing strategic consensus within and between teams. Strateg Manage J 2014; 35: 1053-1069
43 Okoli C, Pawlowski S. The Delphi method as a research tool: an example, design considerations and applications. Inf Manage 2004; 42: 15-29