Quality Initiative Using Theory of Change and Visual Analytics to Improve Controlled Substance Documentation Discrepancies in the Operating Room
15 April 2019
13 June 2019
31 July 2019 (online)
Background Discrepancies in controlled substance documentation are common and can lead to legal and regulatory repercussions. We introduced a visual analytics dashboard to assist in a quality improvement project to reduce the discrepancies in controlled substance documentation in the operating room (OR) of our free-standing pediatric hospital.
Methods Visual analytics were applied to collected documentation discrepancy audit data and were used to track progress of the project, to motivate the OR team, and in analyzing where further improvements could be made. This was part of a seven-step improvement plan based on the Theory of Change with a logic model framework approach.
Results The introduction of the visual analytics dashboard contributed a 24% improvement in controlled substance documentation discrepancy. The project overall reduced documentation errors by 71% over the studied period.
Conclusion We used visual analytics to simultaneously analyze, monitor, and interpret vast amounts of data and present them in an appealing format. In conjunction with quality-improvement principles, this led to a significant improvement in controlled substance documentation discrepancies.
Keywordsquality improvement - visual analytics - clinical informatics - anesthesia - medication documentation discrepancies
Protection of Human and Animal Subjects
The Johns Hopkins Medicine All Children's Hospital Institutional Review Board has determined that this research does not constitute human subjects research under the DHHS or FDA regulations. IRB reference number: 00209372.
- 1 Lesselroth BJ, Adams K, Church VL. , et al. Evaluation of multimedia medication reconciliation software: a randomized controlled, single-blind trial to measure diagnostic accuracy for discrepancy detection. Appl Clin Inform 2018; 9 (02) 285-301
- 2 Anyanwu C, Egwim O. The prevalence and determinants of controlled substance discrepancies in a level I trauma hospital. Am Health Drug Benefits 2016; 9 (03) 128-133
- 3 Vigoda MM, Gencorelli FJ, Lubarsky DA. Discrepancies in medication entries between anesthetic and pharmacy records using electronic databases. Anesth Analg 2007; 105 (04) 1061-1065
- 4 McClure SR, O'Neal BC, Grauer D, Couldry RJ, King AR. Compliance with recommendations for prevention and detection of controlled-substance diversion in hospitals. Am J Health Syst Pharm 2011; 68 (08) 689-694
- 5 Trinkoff AM, Storr CL. Substance use among nurses: differences between specialties. Am J Public Health 1998; 88 (04) 581-585
- 6 Controlled Substances Act, US, 1971. Statutes at Large, 1970–1971, vol. 84, part 1. Government Printing Office, Washington, DC
- 7 Settlement agreement [United States of America and Massachusetts General Hospital]. Available at: https://www.justice.gov/usao-ma/file/778651/download . Accessed July 2, 2019
- 8 Brummond PW, Chen DF, Churchill WW. , et al. ASHP guidelines on preventing diversion of controlled substances. Am J Health Syst Pharm 2017; 74 (05) 325-348
- 9 Epstein RH, Dexter F, Gratch DM, Perino M, Magrann J. Controlled substance reconciliation accuracy improvement using near real-time drug transaction capture from automated dispensing cabinets. Anesth Analg 2016; 122 (06) 1841-1855
- 10 Simpao AF, Ahumada LM, Larru Martinez B. , et al. Design and implementation of a visual analytics electronic antibiogram within an electronic health record system at a tertiary pediatric hospital. Appl Clin Inform 2018; 9 (01) 37-45
- 11 Wanderer JP, Gruss CL, Ehrenfeld JM. Using visual analytics to determine the utilization of preoperative anesthesia assessments. Appl Clin Inform 2015; 6 (04) 629-637
- 12 Sorbello A, Ripple A, Tonning J. , et al. Harnessing scientific literature reports for pharmacovigilance. Prototype software analytical tool development and usability testing. Appl Clin Inform 2017; 8 (01) 291-305
- 13 Brenn BR, Kim MA, Hilmas E. Development of a computerized monitoring program to identify narcotic diversion in a pediatric anesthesia practice. Am J Health Syst Pharm 2015; 72 (16) 1365-1372
- 14 Dowding D, Merrill JA. The development of heuristics for evaluation of dashboard visualizations. Appl Clin Inform 2018; 9 (03) 511-518
- 15 Reed JE, Card AJ. The problem with Plan-Do-Study-Act cycles. BMJ Qual Saf 2016; 25 (03) 147-152
- 16 Breuer E, Lee L, De Silva M, Lund C. Using theory of change to design and evaluate public health interventions: a systematic review. Implement Sci 2016; 11: 63
- 17 Anderson AA. The Community Builder's Approach to Theory of Change: A Practical Guide to Theory Development. New York, NY: Aspen Institute Roundtable on Community Change; 2006
- 18 McLaughlin JA, Jordan GB. Logic models: a tool for telling your programs performance story. Eval Program Plann 1999; 22 (01) 65-72
- 19 Epstein RH, Dexter F, Gratch DM, Lubarsky DA. Intraoperative handoffs among anesthesia providers increase the incidence of documentation errors for controlled drugs. Jt Comm J Qual Patient Saf 2017; 43 (08) 396-402