Feasibility and Reliability Testing of Manual Electronic Health Record Reviews as a Tool for Timely Identification of Diagnostic Error in Patients at Risk

Abstract

Background Although diagnostic error (DE) is a significant problem, it remains challenging for clinicians to identify it reliably and to recognize its contribution to the clinical trajectory of their patients. The purpose of this work was to evaluate the reliability of real-time electronic health record (EHR) reviews using a search strategy for the identification of DE as a contributor to the rapid response team (RRT) activation.

Objectives Early and accurate recognition of critical illness is of paramount importance. The objective of this study was to test the feasibility and reliability of prospective, real-time EHR reviews as a means of identification of DE.

Methods We conducted this prospective observational study in June 2019 and included consecutive adult patients experiencing their first RRT activation. An EHR search strategy and a standard operating procedure were refined based on the literature and expert clinician inputs. Two physician-investigators independently reviewed eligible patient EHRs for the evidence of DE within 24 hours after RRT activation. In cases of disagreement, a secondary review of the EHR using a taxonomy approach was applied. The reviewers categorized patient experience of DE as Yes/No/Uncertain.

Results We reviewed 112 patient records. DE was identified in 15% of cases by both reviewers. Kappa agreement with the initial review was 0.23 and with the secondary review 0.65. No evidence of DE was detected in 60% of patients. In 25% of cases, the reviewers could not determine whether DE was present or absent.

Conclusion EHR review is of limited value in the real-time identification of DE in hospitalized patients. Alternative approaches are needed for research and quality improvement efforts in this field.

Keywords

testing - evaluation - electronic health record - communication - new diagnosis - diagnostic error

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was reviewed by Mayo Clinic Institutional Review Board.

Note

This work was performed at Mayo Clinic, Rochester, Minnesota, United States. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or the Society of Critical Care Medicine.

• References

• 1 National Academies of Sciences, Engineering, and Medicine. Improving Diagnosis in Health Care. Washington, DC: The National Academies Press; 2015
• 2 Goldman L, Sayson R, Robbins S, Cohn LH, Bettmann M, Weisberg M. The value of the autopsy in three medical eras. N Engl J Med 1983; 308 (17) 1000-1005
• 3 Aalten CM, Samson MM, Jansen PA. Diagnostic errors; the need to have autopsies. Neth J Med 2006; 64 (06) 186-190
• 4 Winters B, Custer J, Galvagno Jr SM. , et al. Diagnostic errors in the intensive care unit: a systematic review of autopsy studies. BMJ Qual Saf 2012; 21 (11) 894-902
• 5 Murphy DR, Singh H, Berlin L. Communication breakdowns and diagnostic errors: a radiology perspective. Diagnosis (Berl) 2014; 1 (04) 253-261
• 6 Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med 2003; 78 (08) 775-780
• 7 Graber ML, Franklin N, Gordon R. Diagnostic error in internal medicine. Arch Intern Med 2005; 165 (13) 1493-1499
• 8 Schiff GD, Kim S, Abrams R. et al. Diagnosing diagnosis errors: lessons from a multi-institutional collaborative project. In: Henriksen K, Battles JB, Marks ES. , et al., eds. Advances in Patient Safety: From Research to Implementation. Vol 2: Concepts and Methodology. Rockville, MD: Agency For Healthcare Research and Quality; 2005: 255-278
• 9 Schattner A. Researching and preventing diagnostic errors: chasing patient safety from a different angle. QJM 2016; 109 (05) 293-294
• 10 Reilly JB, Myers JS, Salvador D, Trowbridge RL. Use of a novel, modified fishbone diagram to analyze diagnostic errors. Diagnosis (Berl) 2014; 1 (02) 167-171
• 11 Tobin MJ. Why physiology is critical to the practice of medicine: a 40-year personal perspective. Clin Chest Med 2019; 40 (02) 243-257
• 12 Weingart SN, Yaghi O, Barnhart L. , et al. Preventing diagnostic errors in ambulatory care: an electronic notification tool for incomplete radiology tests. Appl Clin Inform 2020; 11 (02) 276-285
• 13 Sabov M, Jayaprakash N, Chae J, Samavedam S, Gajic O, Pickering B. 1259: Reliability of identifying diagnostic error and delay in critically ill patients. Crit Care Med 2018; 46 (01) 613
• 14 Bergl PA, Nanchal RS, Singh H. Diagnostic error in the critically ill: defining the problem and exploring next steps to advance intensive care unit safety. Ann Am Thorac Soc 2018; 15 (08) 903-907
• 15 Saber Tehrani AS, Lee H, Mathews SC. , et al. 25-Year summary of US malpractice claims for diagnostic errors 1986-2010: an analysis from the National Practitioner Data Bank. BMJ Qual Saf 2013; 22 (08) 672-680
• 16 Kachalia A, Gandhi TK, Puopolo AL. , et al. Missed and delayed diagnoses in the emergency department: a study of closed malpractice claims from 4 liability insurers. Ann Emerg Med 2007; 49 (02) 196-205
• 17 Graber ML. The incidence of diagnostic error in medicine. BMJ Qual Saf 2013; 22 (Suppl. 02) ii21-ii27
• 18 Zwaan L, Singh H. The challenges in defining and measuring diagnostic error. Diagnosis (Berl) 2015; 2 (02) 97-103
• 19 Graber ML, Kissam S, Payne VL. , et al. Cognitive interventions to reduce diagnostic error: a narrative review. BMJ Qual Saf 2012; 21 (07) 535-557
• 20 Chalwin R, Giles L, Salter A, Eaton V, Kapitola K, Karnon J. Reasons for repeat rapid response team calls, and associations with in-hospital mortality. Jt Comm J Qual Patient Saf 2019; 45 (04) 268-275
• 21 Braithwaite RS, DeVita MA, Mahidhara R, Simmons RL, Stuart S, Foraida M. ; Medical Emergency Response Improvement Team (MERIT). Use of medical emergency team (MET) responses to detect medical errors. Qual Saf Health Care 2004; 13 (04) 255-259
• 22 Kim HI, Park S. Sepsis: early recognition and optimized treatment. Tuberc Respir Dis (Seoul) 2019; 82 (01) 6-14
• 23 Diaz JV, Riviello ED, Papali A, Adhikari NKJ, Ferreira JC. Global critical care: moving forward in resource-limited settings. Ann Glob Health 2019; 85 (01) 3
• 24 Mayo Clinic Intranet. Quality data and analysis. Available at: http://intranet.mayo.edu/charlie/quality/quality-data-and-analysis/ . Accessed August 22, 2019
• 25 Jayaprakash N, Chae J, Sabov M, Samavedam S, Gajic O, Pickering BW. Improving diagnostic fidelity: an approach to standardizing the process in patients with emerging critical illness. Mayo Clin Proc Innov Qual Outcomes 2019; 3 (03) 327-334
• 26 Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Fam Med 2005; 37 (05) 360-363
• 27 Dubosh NM, Edlow JA, Lefton M, Pope JV. Types of diagnostic errors in neurological emergencies in the emergency department. Diagnosis (Berl) 2015; 2 (01) 21-28
• 28 Chellis M, Olson J, Augustine J, Hamilton G. Evaluation of missed diagnoses for patients admitted from the emergency department. Acad Emerg Med 2001; 8 (02) 125-130
• 29 O'Connor PM, Dowey KE, Bell PM, Irwin ST, Dearden CH. Unnecessary delays in accident and emergency departments: do medical and surgical senior house officers need to vet admissions?. J Accid Emerg Med 1995; 12 (04) 251-254
• 30 Fowles JB, Fowler EJ, Craft C. Validation of claims diagnoses and self-reported conditions compared with medical records for selected chronic diseases. J Ambul Care Manage 1998; 21 (01) 24-34
• 31 Kaye W, Mancini ME, Truitt TL. When minutes count--the fallacy of accurate time documentation during in-hospital resuscitation. Resuscitation 2005; 65 (03) 285-290
• 32 Pickering BW, Dong Y, Ahmed A. , et al. The implementation of clinician designed, human-centered electronic medical record viewer in the intensive care unit: a pilot step-wedge cluster randomized trial. Int J Med Inform 2015; 84 (05) 299-307
• 33 Hudspeth J, El-Kareh R, Schiff G. Use of an expedited review tool to screen for prior diagnostic error in emergency department patients. Appl Clin Inform 2015; 6 (04) 619-628
• 34 Smith M, Murphy D, Laxmisan A. , et al. Developing software to “track and catch” missed follow-up of abnormal test results in a complex sociotechnical environment. Appl Clin Inform 2013; 4 (03) 359-375
• 35 Aaronson E, Jansson P, Wittbold K, Flavin S, Borczuk P. Unscheduled return visits to the emergency department with ICU admission: A trigger tool for diagnostic error. Am J Emerg Med 2019;S0735-6757(19)30579-0. Doi: 10.1016/j.ajem.2019.158430
• 36 Barwise A, Thongprayoon C, Gajic O, Jensen J, Herasevich V, Pickering BW. Delayed rapid response team activation is associated with increased hospital mortality, morbidity, and length of stay in a tertiary care institution. Crit Care Med 2016; 44 (01) 54-63