The Influence of Artificial Intelligence Scribes on Clinician Experience and Efficiency among Pediatric Subspecialists: A Rapid, Randomized Quality Improvement Trial

Abstract

Background

Artificial intelligence (AI) scribes may reduce the documentation burden and improve clinician experience through generative AI automatically producing provider note sections from recordings of patient–provider encounters.

Objective

We aimed to examine the impact of AI scribes on clinician experience, clinician efficiency, and business efficiency measures among pediatric subspecialty physicians.

Methods

We randomized pediatric subspecialty providers with ≥0.5 clinical full-time equivalent and stable electronic health record (EHR) log metrics to use Microsoft/Nuance Digital Ambient eXperience (DAX) Copilot from May 1, 2024, to July 31, 2024 (intervention group) or controls. Using difference-in-differences, we compared quantitative measures of subjective clinician experience using the KLAS Net EHR Experience survey, objective measures of clinician efficiency from EHR logs (e.g., pajama time), and business efficiency measures. At-the-elbow support checked in with intervention providers approximately weekly, and we assessed the sentiment of qualitative comments.

Results

Twelve providers were randomized to the intervention and 11 to the control group. One intervention provider stopped using DAX due to ineffectiveness. In the intervention group, DAX was used to populate one or more characters in 53% of visit notes (range across providers: 10.6–98.2%). Nine intervention and eight control providers completed pre- and postsurveys. KLAS Net EHR Experience improved among intervention providers from 52.6 (70th percentile) to 75.2 (99th percentile) but dropped from 37.3 (38th percentile) to 30 (14th percentile) among control providers. Experiencing burnout dropped from 8 (89%) to 5 (56%) among intervention providers but remained stable at 3 (38%) in the control group. There was no significant change to pajama time (−9.4 minutes per scheduled day, 95% CI: −41.2 to +22.4), time in notes per encounter (+0.2 minutes per note, 95% CI: −6.6 to +6.9), or work Relative Value Units (wRVUs) per encounter (−0.03, 95% CI: −0.5 to +0.44). Of 48 qualitative comments, 69% had a positive sentiment, 15% neutral, and 17% negative.

Conclusion

Among pediatric subspecialists, AI scribes improved clinician experience and burnout without changing charting time or EHR work outside work hours.

Keywords

electronic health records - burnout - artificial intelligence - quality improvement - randomized controlled trial

Protection of Human and Animal Subjects

The CHOA Institutional Review Board deemed this work non-human subjects research (approval no.: STUDY00001962) as a QI project aimed at reducing physician burnout, rather than a project whose primary goal was to create generalizable knowledge.

Publication History

Received: 30 January 2025

Accepted: 16 July 2025

Accepted Manuscript online:
17 July 2025

Article published online:
11 September 2025

• References

• 1 American Medical Association. . Advocacy in action: Reducing physician burnout. 2024 . Accessed July 18, 2025 at: https://www.ama-assn.org/practice-management/physician-health/advocacy-action-reducing-physician-burnout
• 2 American Medical Association. . Physician burnout rate drops below 50% for first time in 4 years. 2024 . Accessed July 18, 2025 at: https://www.ama-assn.org/practice-management/physician-health/physician-burnout-rate-drops-below-50-first-time-4-years
• 3 Han S, Shanafelt TD, Sinsky CA. et al. Estimating the attributable cost of physician burnout in the United States. Ann Intern Med 2019; 170 (11) 784-790
• 4 Wu Y, Wu M, Wang C, Lin J, Liu J, Liu S. Evaluating the prevalence of burnout among health care professionals related to electronic health record use: Systematic review and meta-analysis. JMIR Med Inform 2024; 12: e54811
• 5 Adler-Milstein J, Zhao W, Willard-Grace R, Knox M, Grumbach K. Electronic health records and burnout: Time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Inform Assoc 2020; 27 (04) 531-538
• 6 Levy DR, Withall JB, Mishuris RG. et al. Defining documentation burden (DocBurden) and excessive DocBurden for all health professionals: A scoping review. Appl Clin Inform 2024; 15 (05) 898-913
• 7 Sloss EA, Abdul S, Aboagyewah MA. et al. Toward alleviating clinician documentation burden: A scoping review of burden reduction efforts. Appl Clin Inform 2024; 15 (03) 446-455
• 8 Corby S, Ash JS, Mohan V. et al. A qualitative study of provider burnout: do medical scribes hinder or help?. JAMIA Open 2021; 4 (03) ooab047
• 9 Micek MA, Arndt B, Baltus JJ. et al. The effect of remote scribes on primary care physicians' wellness, EHR satisfaction, and EHR use. Healthcare (Amst) 2022; 10 (04) 100663
• 10 Mishra P, Kiang JC, Grant RW. Association of medical scribes in primary care with physician workflow and patient experience. JAMA Intern Med 2018; 178 (11) 1467-1472
• 11 Ziemann M, Erikson C, Krips M. The use of medical scribes in primary care settings: A literature synthesis. Med Care 2021; 59 (Suppl. 05) S449-S456
• 12 Abridge. The University of Vermont Health Network Signs Enterprise Deal with AI Leader Abridge To Improve Clinician Wellbeing. Accessed July 18, 2025 at: https://www.abridge.com/press-release/uvm-health-network-announcement
• 13 Nuance. . Northwestern Medicine deploys DAX Copilot embedded in Epic within its enterprise to improve patient and physician experiences. August 15, 2024 . Accessed July 18, 2025 at: https://news.nuance.com/2024-08-15-Northwestern-Medicine-deploys-DAX-Copilot-embedded-in-Epic-within-its-enterprise-to-improve-patient-and-physician-experiences
• 14 Dbouk R, Shanks D, Mishuris RG, Yu FP. . PAC02 Pajama Time(less) Stories - Early Experiences with Ambient AI Documentation. Paper presented at: UGM Conference; August 19–22, 2024 ; Verona, WI
• 15 Patel P, Lu H, Curren M, Ahrensfield K. . UGM126 Listen Up! Two Success Stories with Ambient AI Tools. Paper presented at: UGM Conference; August 19–22, 2024 ; Verona, WI
• 16 Garcia P, Shah S, Ma S, Delahaie C. . PAC18 Ambient Note Generative AI Success Stories_Stanford Health Care. Paper presented at: XGM Conference; May 2024 ; Verona, WI
• 17 Dupre B, Daigrepont T. . PAC18 Ambient Note Generative AI Success Stories_Franciscan Missionaries of Our Lady Health System. Paper presented at: XGM Conference; May 2024 ; Verona, WI
• 18 Misurac J, Knake LA, Blum JM. Impact of ambient artificial intelligence notes on provider burnout. medRXiv 2024.07.18.24310656. Preprint at https://doi.org/10.1101/2024.07.18.24310656
• 19 Shanks D. et al. Enhancing clinical documentation workflow with ambient artificial intelligence: Clinician perspectives on work burden, burnout, and job satisfaction. medRXiv 2024.08.12.24311883. Preprint at https://doi.org/10.1101/2024.08.12.24311883
• 20 Galloway JL, Munroe D, Vohra-Khullar PD. et al. Impact of an artificial intelligence-based solution on clinicians' clinical documentation experience: Initial findings using ambient listening technology. J Gen Intern Med 2024; 39 (13) 2625-2627
• 21 Owens LM, Wilda JJ, Hahn PY, Koehler T, Fletcher JJ. The association between use of ambient voice technology documentation during primary care patient encounters, documentation burden, and provider burnout. Fam Pract 2024; 41 (02) 86-91
• 22 Tierney AA, Gayre G, Hoberman B. et al. Ambient artificial intelligence scribes to alleviate the burden of clinical documentation. NEJM Catal 2024; 5 (03) x
• 23 Owens LM, Wilda JJ, Grifka R, Westendorp J, Fletcher JJ. Effect of ambient voice technology, natural language processing, and artificial intelligence on the patient-physician relationship. Appl Clin Inform 2024; 15 (04) 660-667
• 24 Horwitz LI, Krelle HA. Using rapid randomized trials to improve health care systems. Annu Rev Public Health 2023; 44: 445-457
• 25 El-Kareh R, Brenner DA, Longhurst CA. Developing a highly-reliable learning health system. Learn Health Syst 2022; 7 (03) e10351
• 26 Horwitz LI, Kuznetsova M, Jones SA. Creating a learning health system through rapid-cycle, randomized testing. N Engl J Med 2019; 381 (12) 1175-1179
• 27 Austrian J, Mendoza F, Szerencsy A. et al. Applying A/B testing to clinical decision support: Rapid randomized controlled trials. J Med Internet Res 2021; 23 (04) e16651
• 28 Major VJ, Jones SA, Razavian N. et al. Evaluating the effect of a COVID-19 predictive model to facilitate discharge: A randomized controlled trial. Appl Clin Inform 2022; 13 (03) 632-640
• 29 Rosen K, Krelle H, King WC. et al. Effect of text message reminders to improve paediatric immunisation rates: A randomised controlled quality improvement project. BMJ Qual Saf 2025; 34 (05) 339-348
• 30 Tai-Seale M, May N, Sitapati A, Longhurst CA. A learning health system approach to COVID-19 exposure notification system rollout. Learn Health Syst 2021; 6 (02) e10290
• 31 Wardi G, Owens R, Josef C, Malhotra A, Longhurst C, Nemati S. Bringing the promise of artificial intelligence to critical care: What the experience with sepsis analytics can teach us. Crit Care Med 2023; 51 (08) 985-991
• 32 Brenner AT, Rhode J, Yang JY. et al. Comparative effectiveness of mailed reminders with and without fecal immunochemical tests for Medicaid beneficiaries at a large county health department: A randomized controlled trial. Cancer 2018; 124 (16) 3346-3354
• 33 Epic Systems Corporation. . Signal - Efficiency Portal. Accessed July 18, 2025 at: https://signal.epic.com/Documentation/MetricReference
• 34 KLAS Research. . KLAS Arch Collaborative. Accessed July 18, 2025 at: https://engage.klasresearch.com/klas-arch-collaborative/
• 35 Wing C, Dreyer M. Making sense of the difference-in-difference design. JAMA Intern Med 2024; 184: 1250-1251 . Accessed July 18, 2025 at: https://jamanetwork.com/journals/jamainternalmedicine/article-abstract/2822389
• 36 R Core Team. R: A language and environment for statistical computing. Accessed July 18, 2025 at: https://cran.r-project.org/doc/manuals/r-release/fullrefman.pdf
• 37 Cao DY, Silkey JR, Decker MC, Wanat KA. Artificial intelligence-driven digital scribes in clinical documentation: Pilot study assessing the impact on dermatologist workflow and patient encounters. JAAD Int 2024; 15: 149-151
• 38 Liu T-L, Hetherington TC, Stephens C. et al. AI-powered clinical documentation and clinicians' electronic health record experience: A nonrandomized clinical trial. JAMA Netw Open 2024; 7 (09) e2432460
• 39 Haberle T, Cleveland C, Snow GL. et al. The impact of nuance DAX ambient listening AI documentation: A cohort study. J Am Med Inform Assoc 2024; 31 (04) 975-979
• 40 Shah SJ, Devon-Sand A, Ma SP. et al. Ambient artificial intelligence scribes: physician burnout and perspectives on usability and documentation burden. J Am Med Inform Assoc 2025; 32 (02) 375-380
• 41 Ralston SL, Brady PW, Kemper AR. Do we really need scholarly quality improvement?. JAMA Pediatr 2019; 173 (05) 413-414
• 42 Finkelstein JA, Brickman AL, Capron A. et al. Oversight on the borderline: Quality improvement and pragmatic research. Clin Trials 2015; 12 (05) 457-466