An AI-Powered Strategy for Managing Patient Messaging Load and Reducing Burnout

Abstract

Objective

This study aims to evaluate the impact of using a large language model (LLM) for generating draft responses to patient messages in the electronic health record (EHR) system on clinicians and support staff workload and efficiency.

Methods

We partnered with Epic Systems to implement OpenAI's ChatGPT 4.0 for responding to patient messages. A pilot study was conducted from August 2023 to July 2024 across 13 ambulatory specialties involving 323 participants, including clinicians and support staff. Data on draft utilization rates and message response times were collected and analyzed using statistical methods.

Results

The overall mean generated draft utilization rate was 38%, with significant differences by role and specialty. Clinicians had a higher utilization rate (43%) than scheduling staff (33%). Draft message usage significantly reduced all users' message response time (13 seconds on average). Support staff experienced a more substantial and statistically significant time saving (23 seconds) compared to negligible time savings seen by clinicians (3 seconds). Variability in utilization rates and time savings was observed across different specialties.

Conclusion

Implementing LLMs for drafting patient message replies can reduce response times and alleviate message burden. However, the effectiveness of artificial intelligence (AI)-generated draft responses varies by clinical role and specialty, indicating the need for tailored implementations. Further investigation into this variability, and development and personalization of AI tools are recommended to maximize their utility and ensure safe and effective use in diverse clinical contexts.

Keywords

artificial intelligence - clinical information systems - electronic health records and systems - clinical information systems - burnout - messaging - clinical informatics

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was exempt from institutional review board–mandated consent.

Note

Data presented in the figure are imaginary.

Publication History

Received: 31 October 2024

Accepted: 04 April 2025

Accepted Manuscript online:
08 April 2025

Article published online:
06 August 2025

• References

• 1 Tai-Seale M, Dillon EC, Yang Y. et al. Physicians' well-being linked to in-basket messages generated by algorithms in electronic health records. Health Aff (Millwood) 2019; 38 (07) 1073-1078
• 2 Akbar F, Mark G, Prausnitz S. et al. Physician stress during electronic health record inbox work: in situ measurement with wearable sensors. JMIR Med Inform 2021; 9 (04) e24014
• 3 Holmgren AJ, Downing NL, Tang M, Sharp C, Longhurst C, Huckman RS. Assessing the impact of the COVID-19 pandemic on clinician ambulatory electronic health record use. J Am Med Inform Assoc 2022; 29 (03) 453-460
• 4 Chen S, Guevara M, Moningi S. et al. The effect of using a large language model to respond to patient messages. Lancet Digit Health 2024; 6 (06) e379-e381
• 5 Garcia P, Ma SP, Shah S. et al. Artificial intelligence-generated draft replies to patient inbox messages. JAMA Netw Open 2024; 7 (03) e243201-e243201
• 6 Hswen Y, Voelker R. Electronic health records failed to make clinicians' lives easier—will AI technology succeed?. JAMA 2023; 330 (16) 1509-1511
• 7 Tai-Seale M, Baxter SL, Vaida F. et al. AI-generated draft replies integrated into health records and physicians' electronic communication. JAMA Netw Open 2024; 7 (04) e246565-e246565
• 8 Cacciaglia A. Gen AI Saves Nurses Time by Drafting Responses to Patient Messages. Updated March 4, 2024. Accessed July 17, 2024 at: https://www.epicshare.org/share-and-learn/mayo-ai-message-responses