Generative Artificial Intelligence Summaries to Facilitate Emergency Department Handoff

Abstract

Background

Emergency department (ED) handoff to inpatient teams is a potential source of error. Generative artificial intelligence (AI) has shown promise in succinctly summarizing large quantities of clinical data and may help improve ED handoff.

Objectives

Our objectives were to: (1) evaluate the accuracy, clinical utility, and safety of AI-generated ED-to-inpatient handoff summaries; (2) identify patient and visit characteristics influencing summary effectiveness; and (3) characterize potential error patterns to inform implementation strategies.

Methods

This exploratory study evaluated AI-generated handoff summaries at an urban academic ED (February–April 2024). A Health Insurance Portability and Accountability Act-compliant GPT-4 model generated summaries aligned with the IPASS framework; ED providers assessed summary accuracy, usefulness, and safety through on-shift surveys.

Results

Among 50 cases, median quality and usefulness scores were 4/5 (standard error = 0.13). Safety concerns arose in 6% of cases, with issues including data omissions and mischaracterizations. Consultation status significantly affected usefulness scores (p < 0.05). Omissions of relevant medications, laboratory results, and other essential details were noted (n = 6), and emergency medicine clinicians disagreed with some AI characterizations of patient stability, vitals, and workup (n = 8). The most common response was positive impressions of the technology incorporated into the handoff process (n = 11).

Conclusion

This exploratory provider-in-the-loop model demonstrated clinical acceptability and highlighted areas for refinement. Future studies should incorporate recipient perspectives and examine clinical outcomes to scale and optimize AI implementation.

Keywords

emergency medicine - communication - artificial intelligence - safety - handoffs - electronic health records

Protection of Human and Animal Subjects

This project was certified as a Quality Improvement project by the NYU Langone Health Human Research Protection Program and is not research involving human subjects

Publikationsverlauf

Eingereicht: 01. Mai 2025

Angenommen: 11. August 2025

Accepted Manuscript online:
12. August 2025

Artikel online veröffentlicht:
26. September 2025

• References

• 1 Riesenberg LA. Shift-to-shift handoff research: where do we go from here?. J Grad Med Educ 2012; 4 (01) 4-8
• 2 Sanchez LD, Chiu DT, Nathanson L. et al. A model for electronic handoff between the emergency department and inpatient units. J Emerg Med 2017; 53 (01) 142-150
• 3 Singleton JM, Sanchez LD, Masser BA, Reich B. Efficiency of electronic signout for ED-to-inpatient admission at a non-teaching hospital. Intern Emerg Med 2018; 13 (07) 1105-1110
• 4 Starmer AJ, Spector ND, Srivastava R. et al; I-PASS Study Group. Changes in medical errors after implementation of a handoff program. N Engl J Med 2014; 371 (19) 1803-1812
• 5 Turner JS, Courtney RD, Sarmiento E, Ellender TJ. Frequency of safety net errors in the emergency department: effect of patient handoffs. Am J Emerg Med 2021; 42: 188-191
• 6 Kachman MM, Brennan I, Oskvarek JJ, Waseem T, Pines JM. How artificial intelligence could transform emergency care. Am J Emerg Med 2024; 81 (C): 40-46
• 7 Preiksaitis C, Ashenburg N, Bunney G. et al. The role of large language models in transforming emergency medicine: scoping review. JMIR Med Inform 2024; 12: e53787
• 8 Hartman V, Zhang X, Poddar R. et al. Developing and evaluating large language model-generated emergency medicine handoff notes. JAMA Netw Open 2024; 7 (12) e2448723
• 9 Bedi S, Liu Y, Orr-Ewing L. et al. Testing and evaluation of health care applications of large language models: a systematic review. JAMA 2025; 333 (04) 319-328
• 10 Baker HP, Dwyer E, Kalidoss S, Hynes K, Wolf J, Strelzow JA. ChatGPT's ability to assist with clinical documentation: a randomized controlled trial. J Am Acad Orthop Surg 2024; 32 (03) 123-129
• 11 Yanni E, Calaman S, Wiener E, Fine JS, Sagalowsky ST. Implementation of ED I-PASS as a standardized handoff tool in the pediatric emergency department. J Healthc Qual 2023; 45 (03) 140-147
• 12 Grabinski Z. Improving the Safety of Pediatric Emergency Department to Inpatient Transfers of Care through Risk Stratification, Electronic Heath Record Visualization, and Interdisciplinary Reassessment Workflows; 2024. Accessed January 13, 2025 at: https://2024.pas-meeting.org/fsPopup.asp?PosterID=649440&mode=posterInfo
• 13 Farmer T, Robinson K, Elliott SJ, Eyles J. Developing and implementing a triangulation protocol for qualitative health research. Qual Health Res 2006; 16 (03) 377-394
• 14 Schwieger A, Angst K, de Bardeci M. et al. Large language models can support generation of standardized discharge summaries - a retrospective study utilizing ChatGPT-4 and electronic health records. Int J Med Inform 2024; 192 (105654): 105654
• 15 Barak-Corren Y, Wolf R, Rozenblum R. et al. Harnessing the power of generative AI for clinical summaries: perspectives from emergency physicians. Ann Emerg Med 2024; 84 (02) 128-138
• 16 Landman AB, Tilak SS, Walker GA. Artificial intelligence-generated emergency department summaries and hospital handoffs. JAMA Netw Open 2024; 7 (12) e2448729
• 17 Ahsan H, McInerney DJ, Kim J. et al. Retrieving evidence from EHRs with LLMs: possibilities and challenges. Proc Mach Learn Res 2024; 248: 489-505
• 18 Tang L, Sun Z, Idnay B. et al. Evaluating large language models on medical evidence summarization. NPJ Digit Med 2023; 6 (01) 158