Developing Software to “Track and Catch” Missed Follow-up of Abnormal Test Results in a Complex Sociotechnical Environment

M. Smith; D. Murphy; A. Laxmisan; D. Sittig; B. Reis; A. Esquivel; H. Singh

doi:10.4338/ACI-2013-04-RA-0019

Applied Clinical Informatics, Table of Contents

Appl Clin Inform 2013; 04(03): 359-375
DOI: 10.4338/ACI-2013-04-RA-0019

Research Article

Schattauer GmbH

Developing Software to “Track and Catch” Missed Follow-up of Abnormal Test Results in a Complex Sociotechnical Environment

M. Smith

¹Houston VA HSR&D Center of Excellence and The Center of Inquiry to Improve Outpatient Safety Through Effective Electronic Communication, Michael E. DeBakey Veterans Affairs Medical Center and the Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, Texas

,

D. Murphy

¹Houston VA HSR&D Center of Excellence and The Center of Inquiry to Improve Outpatient Safety Through Effective Electronic Communication, Michael E. DeBakey Veterans Affairs Medical Center and the Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, Texas

,

A. Laxmisan

¹Houston VA HSR&D Center of Excellence and The Center of Inquiry to Improve Outpatient Safety Through Effective Electronic Communication, Michael E. DeBakey Veterans Affairs Medical Center and the Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, Texas

,

D. Sittig

²University of Texas School of Biomedical Informatics and the UT-Memorial Hermann Center for Healthcare Quality & Safety, Houston, Texas

,

B. Reis

¹Houston VA HSR&D Center of Excellence and The Center of Inquiry to Improve Outpatient Safety Through Effective Electronic Communication, Michael E. DeBakey Veterans Affairs Medical Center and the Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, Texas

,

A. Esquivel

³Department of Clinical Effectiveness and Performance Measurement, St. Luke’s Episcopal Health System, Houston, Texas

,

H. Singh

¹Houston VA HSR&D Center of Excellence and The Center of Inquiry to Improve Outpatient Safety Through Effective Electronic Communication, Michael E. DeBakey Veterans Affairs Medical Center and the Section of Health Services Research, Department of Medicine, Baylor College of Medicine, Houston, Texas

› Author Affiliations

Abstract

Summary

Background: Abnormal test results do not always receive timely follow-up, even when providers are notified through electronic health record (EHR)-based alerts. High workload, alert fatigue, and other demands on attention disrupt a provider’s prospective memory for tasks required to initiate follow-up. Thus, EHR-based tracking and reminding functionalities are needed to improve follow-up.

Objectives: The purpose of this study was to develop a decision-support software prototype enabling individual and system-wide tracking of abnormal test result alerts lacking follow-up, and to conduct formative evaluations, including usability testing.

Methods: We developed a working prototype software system, the Alert Watch And Response Engine (AWARE), to detect abnormal test result alerts lacking documented follow-up, and to present context-specific reminders to providers. Development and testing took place within the VA’s EHR and focused on four cancer-related abnormal test results. Design concepts emphasized mitigating the effects of high workload and alert fatigue while being minimally intrusive. We conducted a multifaceted formative evaluation of the software, addressing fit within the larger socio-technical system. Evaluations included usability testing with the prototype and interview questions about organizational and workflow factors. Participants included 23 physicians, 9 clinical information technology specialists, and 8 quality/safety managers.

Results: Evaluation results indicated that our software prototype fit within the technical environment and clinical workflow, and physicians were able to use it successfully. Quality/safety managers reported that the tool would be useful in future quality assurance activities to detect patients who lack documented follow-up. Additionally, we successfully installed the software on the local facility’s “test” EHR system, thus demonstrating technical compatibility.

Conclusion: To address the factors involved in missed test results, we developed a software prototype to account for technical, usability, organizational, and workflow needs. Our evaluation has shown the feasibility of the prototype as a means of facilitating better follow-up for cancer-related abnormal test results.

Keywords

Diagnostic errors - patient safety - test results - care delays - health information technology

Full Text

References

References
1 Singh H, Naik A, Rao R, Petersen L. Reducing Diagnostic Errors Through Effective Communication: Harnessing the Power of Information Technology.. Journal of General Internal Medicine 2008; 23 (04) 489-494.
2 Singh H, Graber M. Reducing diagnostic error through medical home-based primary care reform.. JAMA 2010; 304 (04) 463-464.
3 Singh H, Thomas EJ, Mani S, Sittig DF, Arora H, Espadas D, Khan MM, Petersen LA. Timely follow-up of abnormal diagnostic imaging test results in an outpatient setting: are electronic medical records achieving their potential?. Arch Intern Med 2009; 169 (17) 1578-1586.
4 Singh H, Thomas EJ, Sittig DF, Wilson L, Espadas D, Khan MM, Petersen LA. Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain?. Am J Med 2010; 123 (03) 238-244.
5 Poon E, Gandhi T, Sequist T, Murff H, Karson A, Bates D. ,,I wish I had seen this test result earlier!“: Dissatisfaction with test result management systems in primary care.. Archives of internal medicine 2004; 164 (20) 2223-2228.
6 Wahls T. Diagnostic errors and abnormal diagnostic tests lost to follow-up: a source of needless waste and delay to treatment.. The Journal of ambulatory care management 2007; 30 (04) 338-343.
7 Singh H, Sethi S, Raber M, Petersen LA. Errors in cancer diagnosis: current understanding and future directions.. J Clin Oncol 2007; 25 (31) 5009-5018.
8 Nemeth CP, Cook RI, Woods DD. The Messy Details: Insights From the Study of Technical Work in Healthcare.. IEEE Transactions on Systems, Man, and Cybernetics –Part A: Systems and Humans 2004; 34 (06) 689-692.
9 Carayon P, Schoofs Hundt A, Karsh BT, Gurses AP, Alvarado CJ, Smith M, Flatley BP. Work system design for patient safety: the SEIPS model.. Quality & safety in health care 2006; 15 Suppl 1 (Suppl. 01) i50-i58.
10 Wears R, Berg M. Computer technology and clinical work: still waiting for Godot.. JAMA 2005; 293 (10) 1261-1263.
11 Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems.. Quality and Safety in Health Care 2010; 19 (Suppl. 03) i68-i74.
12 Carayon P, Bass E, Bellandi T, Gurses A, Hallbeck S, Mollo V. Socio-Technical Systems Analysis in Health Care: A Research Agenda.. IIE transactions on healthcare systems engineering 2011; 1 (01) 145-60.
13 Graber M, Franklin N, Gordon R. Diagnostic error in internal medicine.. Archives of internal medicine 2005; 165 (13) 1493-1499.
14 Raab S, Grzybicki D. Quality in cancer diagnosis.. CA: a cancer journal for clinicians 2010; 60 (03) 139-65.
15 Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF. Information Overload and Missed Test Results in Electronic Health Record Based Settings.. JAMA Internal Medicine 2013; 1-3.
16 Wears R, Perry S, Patterson E. Handoffs and Transitions of Care.. In P Carayon (ed) Handbook of Human Factors and Ergonomics in Healthcare and Patient Safety. L. Erlbaum Associates Inc.;. 2006: 163-163.
17 Pennathur P, Bass E, Rayo M, Perry S, Rosen M, Gurses A. Handoff Communication: Implications For Design.. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2012; 56 (01) 863-866.
18 Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry.. J Am Med Inform Assoc 2007; 14 (04) 415-423.
19 Lawler E, Hedge A, Pavlovic-Veselinovic S. Cognitive ergonomics, socio-technical systems, and the impact of healthcare information technologies.. International Journal of Industrial Ergonomics 2011 Apr.
20 Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Smith MW, Murphy DR, Espadas D, Laxmisan A, Sittig DF. Primary care practitioners views on test result management in EHR-enabled health systems: a national survey.. J Am Med Inform Assoc. 2012
21 Dismukes K. Remembrance of Things Future: Prospective Memory in Laboratory, Workplace, and Everyday Settings. In Reviews of human factors and ergonomics.. Human Factors and Ergonomics Society 2010: 79-122.
22 Westbrook J, Coiera E, Dunsmuir W, Brown B, Kelk N, Paoloni R, Tran C. The impact of interruptions on clinical task completion.. Quality and Safety in Health Care 2010; 19 (04) 284-289.
23 Laxmisan A, Hakimzada F, Sayan O, Green R, Zhang J, Patel V. The multitasking clinician: decision-making and cognitive demand during and after team handoffs in emergency care.. International Journal of Medical Informatics 2007; 76 11–12 801-811.
24 Reason J. Combating omission errors through task analysis and good reminders.. Qual Saf Health Care 2002; 11 (01) 40-44.
25 Saleem JJ, Russ AL, Sanderson P, Johnson TR, Zhang J, Sittig DF. Current challenges and opportunities for better integration of human factors research with development of clinical information systems.. Yearb Med Inform 2009; 48-58.
26 Hutchins E. How a cockpit remembers its speeds.. Cognitive Science 1995; 19 (03) 265-288.
27 Murphy D, Reis B, Sittig DF, Singh H. Notifications Received by Primary Care Practitioners in Electronic Health Records: A Taxonomy and Time Analysis.. American Journal of Medicine 2012; 125 (02) 209-el.
28 Hysong S, Sawhney M, Wilson L, Sittig D, Esquivel A, Singh S, Singh H. Understanding the Management of Electronic Test Result Notifications in the Outpatient Setting.. BMC Med Inform Decis Mak 2011; 11 (01) 22.
29 Murphy D, Reis B, Kadiyala H, Hirani K, Sittig D, Khan M, Singh H. Electronic Health Record-Based Messages to Primary Care Providers: Valuable Information or Just Noise?. Arch Intern Med 2012; 172 (03) 283-285.
30 van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry.. Journal of the American Medical Informatics Association : JAMIA 2006; 13 (02) 138-147.
31 Sorkin R, Woods D. Systems with Human Monitors: A Signal Detection Analysis.. Human–Computer Interaction 1985; 1 (01) 49-75.
32 Singh H, vis Giardina T, Petersen L, Smith M, Paul L, Dismukes K, Bhagwath G, Thomas E. Exploring situational awareness in diagnostic errors in primary care.. BMJ Quality & Safety 2011 Sep 2.
33 Singh H, Arora H, Vij M, Rao R, Khan MM, Petersen L. Communication outcomes of critical imaging results in a computerized notification system.. J Am Med Inform Assoc 2007; 14 (04) 459-466.
34 Singh H, Wilson L, Petersen L, Sawhney MK, Reis B, Espadas D, Sittig DF. Improving follow-up of abnormal cancer screens using electronic health records: trust but verify test result communication.. BMC Medical Informatics and Decision Making. 2009 9. (49).
35 Singh H, Vij MS. Eight recommendations for policies for communicating abnormal test results.. Jt Comm J Qual Patient Saf 2010; 36 (05) 226-232.
36 Hysong SJ, Sawhney MK, Wilson L, Sittig DF, Espadas D, Davis T, Singh H. Provider management strategies of abnormal test result alerts: a cognitive task analysis.. J Am Med Inform Assoc 2010; 17 (01) 71-77.
37 Singh H, Wilson L, Reis B, Sawhney MK, Espadas D, Sittig DF. Ten Strategies to Improve Management of Abnormal Test Result Alerts in the Electronic Health Record.. Journal of Patient Safety 2010; 6 (02) 121-123.
38 Gandhi TK, Kachalia A, Thomas EJ, Puopolo A L., Yoon C, Brennan TA, Studdert DM. Missed and delayed diagnoses in the ambulatory setting: A study of closed malpractice claims.. Ann Intern Med 2006; 145 (07) 488-496.
39 Nepple KG, Joudi FN, Hillis SL, Wahls TL. Prevalence of delayed clinician response to elevated prostate-specific antigen values.. Mayo Clin Proc 2008; 83 (04) 439-448.
40 Singh H, Kadiyala H, Bhagwath G, Shethia A, El-Serag H, Walder A, Velez ME, Petersen LA. Using a multifaceted approach to improve the follow-up of positive fecal occult blood test results.. Am J Gastroenterol 2009; 104 (04) 942-952.
41 Singh H, Hirani K, Kadiyala H, Rudomiotov O, Davis T, Khan MM, Wahls TL. Characteristics and predictors of missed opportunities in lung cancer diagnosis: an electronic health record-based study.. J Clin Oncol 2010; 28 (20) 3307-3315.
42 Zeliadt SB, Hoffman RM, Etzioni R, Ginger VA, Lin DW. What happens after an elevated PSA test: the experience of 13,591 veterans.. J Gen Intern Med 2010; 25 (11) 1205-1210.
43 Saleem J, Patterson E, Militello L, Render M, Orshansky G, Asch S. Exploring Barriers and Facilitators to the Use of Computerized Clinical Reminders.. J Am Med Inform Assoc 2005; 12 (04) 438-447.
44 Grundgeiger T, Sanderson PM, MacDougall HG, Venkatesh B. Distributed prospective memory: An approach to understanding how nurses remember tasks.. In Proceedings of the Human Factors and Ergo-nomics Society Annual Meeting.. SAGE Publications;; 2009: 759-759.
45 Woods DD, Watts JC. How not to have to navigate through too many displays.. In: Helander MG, Land-auer TK, Prabhu PV. editors. Handbook of Human-Computer Interaction, Second Edition.. North Holland: 1997: 617-617.
46 Jaspers MW. A comparison of usability methods for testing interactive health technologies: Methodological aspects and empirical evidence.. International Journal of Medical Informatics 2009; 78 (05) 340-353.
47 Smith PJ, Stone RB, Spencer AL. Applying cognitive psychology to system development.. In: Marras WS, Karwowski W. editors. Fundamentals and Assessment Tools for Occupational Ergonomics (The Occupational Ergonomics Handbook, Second Edition.. CRC Press; 2006: 24-24.
48 Ericsson A, Simon H. Verbal Reports as Data.. Psychological Review 1980; 87 (03) 215-251.
49 Dumas JS, Salzman MC. Usability Assessment Methods.. Reviews of Human Factors and Ergonomics 2006; 2 (01) 109-140.
50 Nielsen J. Finding usability problems through heuristic evaluation.. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM. 1992: 373-380.
51 Schiff G. Medical Error.. JAMA 2011; 305 (18) 1890-1898.
52 Poon E, Wang S, Gandhi T, Bates D, Kuperman G. Design and implementation of a comprehensive out-patient Results Manager.. Journal of Biomedical Informatics 2003; 36 1–2 80-91.
53 Guerlain S, Smith P, Obradovich J, Rudmann S, Strohm P, Smith J, Svirbely J, Sachs L. Interactive Critiquing as a Form of Decision Support: An Empirical Evaluation.. Human Factors: The Journal of the Human Factors and Ergonomics Society 1999; 72-89.
54 Smith PJ, McCoy CE, Layton C. Brittleness in the design of cooperative problem-solving systems: the effects on user performance.. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on 1997; 27 (03) 360-371.
55 Adamczyk P, Bailey B. If not now, when?: the effects of interruption at different moments within task execution. In Proceedings of the SIGCHI conference on Human factors in computing systems.. Vienna, Austria:: ACM; 2004: 271-271.
56 Grudin J. Groupware and social dynamics: eight challenges for developers.. Communications of the ACM 1994; 37 (01) 92-105.
57 Leveson N, Dulac N, Marais K, Carroll J. Moving Beyond Normal Accidents and High Reliability Organizations: A Systems Approach to Safety in Complex Systems.. Organization Studies 2009; 30 2–3 227-249.
58 Karsh BT, Weinger M, Abbott P, Wears R. Health information technology: fallacies and sober realities.. J Am Med Inform Assoc 2010; 17 (06) 617-623.
59 McDonald C, McDonald M. INVITED COMMENTARY –Electronic Medical Records and Preserving Primary Care Physicians’ Time: Comment on ,,Electronic Health Record-Based Messages to Primary Care Providers“.. Arch Intern Med 2012; 172 (03) 285-287.