Open Access
CC BY 4.0 · Appl Clin Inform 2025; 16(04): 1067-1076
DOI: 10.1055/a-2595-0317
Special Issue on CDS Failures

A Measurement Science Framework to Optimize CDS for Opioid Use Disorder Treatment in the ED

Mark S. Iscoe
1   Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut, United States
2   Department of Biomedical Informatics and Data Sciences, Yale University School of Medicine, New Haven, Connecticut, United States
,
Carolina Diniz Hooper
1   Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut, United States
,
Deborah R. Levy
2   Department of Biomedical Informatics and Data Sciences, Yale University School of Medicine, New Haven, Connecticut, United States
3   Department of Veterans Affairs, Amarillo VA Healthcare System, Amarillo, Texas, United States
,
John Lutz
4   Yale University School of Medicine, New Haven, Connecticut, United States
,
Hyung Paek
5   Department of Biostatistics (Health Informatics), Yale School of Public Health, New Haven, Connecticut, United States
,
Christian Rose
6   Department of Emergency Medicine, Stanford University School of Medicine, Stanford, California, United States
,
Thomas Kannampallil
7   Institute for Informatics, Data Science, and Biostatistics Washington University School of Medicine, St. Louis, Missouri, United States
,
Daniella Meeker
2   Department of Biomedical Informatics and Data Sciences, Yale University School of Medicine, New Haven, Connecticut, United States
,
James D. Dziura
1   Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut, United States
5   Department of Biostatistics (Health Informatics), Yale School of Public Health, New Haven, Connecticut, United States
,
Edward R. Melnick
1   Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut, United States
2   Department of Biomedical Informatics and Data Sciences, Yale University School of Medicine, New Haven, Connecticut, United States
5   Department of Biostatistics (Health Informatics), Yale School of Public Health, New Haven, Connecticut, United States
› Author Affiliations

Funding Research reported in this study was supported by the National Institute On Drug Abuse of the National Institutes of Health under Award Number R33DA059884. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. This publication was also made possible by CTSA grant number UL1 TR001863 from the National Center for Advancing Translational Science (NCATS), a component of the National Institutes of Health (NIH). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of NIH. The contents of this manuscript represent the view of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs or the United States Government.
Preview

Abstract

Objectives

In the emergency department-initiated buprenorphine for opioid use disorder (EMBED) trial, a clinical decision support (CDS) tool had no effect on rates of buprenorphine initiation in emergency department (ED) patients with opioid use disorder. The Agency for Healthcare Research and Quality (AHRQ) recently released a CDS Performance Measure Inventory to guide data-driven CDS development and evaluation. Through partner co-design, we tailored AHRQ inventory measures to evaluate EMBED CDS performance and drive improvements.

Methods

Relevant AHRQ inventory measures were selected and adapted using a partner co-design approach grounded in consensus methodology, with three iterative, multidisciplinary partner working group sessions involving stakeholders from various roles and institutions; meetings were followed by postmeeting surveys. The co-design process was divided into conceptualization, specification, and evaluation phases building on the Centers for Medicare and Medicaid Services' measure life cycle framework. Final measures were evaluated in three EDs in a single health system from January 1, 2023, to December 31, 2024.

Results

The partner working group included 25 members. During conceptualization, 13 initial candidate metrics were narrowed to 6 priority categories. These were further specified and validated as the following measures, presented with preliminary values based on the use of the current (i.e., preoptimization) EMBED CDS: eligible encounters with CDS engagement, 5.0% (95% confidence interval: 4.3–5.8%); teamwork on ED initiation of buprenorphine, 39.9% (32.5–47.3%); proportion of eligible users who used EMBED, 58.3% (50.9–65.8%); time spent on EMBED, 29.0 seconds (20.4–37.7 seconds); proportion of buprenorphine orders placed through EMBED, 6.5% (3.4–9.6%); and task completion, 13.8% (8.9–18.7%) for buprenorphine order/prescription.

Conclusion

A measurement science framework informed by partner co-design was a feasible approach to develop measures to guide CDS improvement. Subsequent research could adapt this approach to evaluate other CDS applications.

Note

Its contents are solely the responsibility of the authors and do not necessarily represent the official views of NIH.


Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was deemed exempt by the Yale School of Medicine Institutional Review Board (protocol #2000037541).




Publication History

Received: 18 February 2025

Accepted: 21 April 2025

Accepted Manuscript online:
20 August 2025

Article published online:
12 September 2025

© 2025. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)

Georg Thieme Verlag KG
Oswald-Hesse-Straße 50, 70469 Stuttgart, Germany