Implementation of a Real-Time Documentation Assistance Tool: Automated Diagnosis (AutoDx)

Abstract

Background Clinical documentation improvement programs are utilized by most health care systems to enhance provider documentation. Suggestions are sent to providers in a variety of ways, and are commonly referred to as coding queries. Responding to these coding queries can require significant provider time and do not often align with workflows. To enhance provider documentation in a more consistent manner without creating undue burden, alternative strategies are required.

Objectives The aim of this study is to evaluate the impact of a real-time documentation assistance tool, named AutoDx, on the volume of coding queries and encounter-level outcome metrics, including case-mix index (CMI).

Methods The AutoDx tool was developed utilizing tools existing within the electronic health record, and is based on the generation of messages when clinical conditions are met. These messages appear within provider notes and required little to no interaction. Initial diagnoses included in the tool were electrolyte deficiencies, obesity, and malnutrition. The tool was piloted in a cohort of Hospital Medicine providers, then expanded to the Neuro Intensive Care Unit (NICU), with addition diagnoses being added.

Results The initial Hospital Medicine implementation evaluation included 590 encounters pre- and 531 post-implementation. The volume of coding queries decreased 57% (p < 0.0001) for the targeted diagnoses compared with 6% (p = 0.77) in other high-volume diagnoses. In the NICU cohort, 829 encounters pre-implementation were compared with 680 post. The proportion of AutoDx coding queries compared with all other coding queries decreased from 54.9 to 37.1% (p < 0.0001). During the same period, CMI demonstrated a significant increase post-implementation (4.00 vs. 4.55, p = 0.02).

Conclusion The real-time documentation assistance tool led to a significant decrease in coding queries for targeted diagnoses in two unique provider cohorts. This improvement was also associated with a significant increase in CMI during the implementation time period.

 

MissingLinkText

Keywords

clinical documentation - automated - AutoDx

Background and Significance

Provider documentation is an integral part of the medical system and historically served to communicate active disease processes, medical decision making, and treatment plans.[1] Identifying appropriate diagnoses for a patient on a specific encounter has impact on a wide range of factors, including billing and reimbursement, severity of illness determination, and patient communication, with the “Open Notes Rule” mandate allowing patients to see patient care notes.[2] [3] Organizations dedicate resources to optimize this process, which include clinical documentation improvement (CDI) specialists and proprietary software.[4] [5] When additional diagnoses are identified by trained personnel, changes to the documentation by the provider are required to finalize the process.[6] However, with the growing concerns of provider burnout, it is difficult to demand providers to find additional time to address these coding queries outside of their regular clinical duties.[7]

Improving documentation within the normal provider workflow is the model solution. The introduction of documentation improvement education and dedicated personnel has been demonstrated to significantly increase timely completion of documentation, compliance with quality measures, more comprehensive risk assessments, and even increases in charges made.[7] [8] [9] Despite these efforts in many fields, additional education related to billing and coding has been identified as a need, especially in residency training.[10] [11] [12] This is difficult given the time constrains of residencies and many competing interests for limited time. Ideally, a process to improve documentation of specific diagnoses could occur in real-time with little to no provider interaction, based on actions already completed in the medical record. This has been used in quality reporting and research data collection.[13] The electronic health record (EHR) can offer opportunities to leverage results, orders, and documented assessments to identify specific diagnoses. An emergency medicine study leveraged prior orders for specific medications that indicated level of complexity, but it still required further documentation from providers.[14] Furthermore, it provides additional areas for clinical decision support that fits within the workflow of clinical care without an additional alert. We developed a novel automated diagnosis documentation tool (AutoDx). We aimed to identify a pathway within the EHR to assist providers with the documentation of specific diagnoses in real-time to decrease the burden time related to documentation overall, as well as time related to other documentation improvement processes, and to improve outcomes related to documentation.

 

MissingLinkText

Methods

Setting

The project was initiated at a large 660-bed urban academic hospital utilizing Epic (Epic Care Systems, Madison, Wisconsin, United States) as its EHR. The study institution sponsors an EHR Physician Builder program that supports physicians to make changes to the EHR, under the supervision of the medical center's medical informatics and information technology (IT) teams.[15] All physician builders were required to complete the formal certification program offered by the EHR vendor before making changes to the EHR system. The AutoDx tool was designed, developed, and implemented mainly by the physician builder group. At the study institution, the standard process for the CDI team to address documentation opportunities is to send queries of potential diagnoses through the EHR. These queries can happen during the inpatient encounter or following discharge, if a secondary review was completed. The initial recipient of the query would be the primary note writer, whether the attending physician on direct care Hospital Medicine services or the resident physician on many of the teaching services in the institution. Medical students did not receive coding queries, even if they were the initial authors of a note. In cases where residents were the primary recipient of the intervention, but declined to answer, were significantly delayed in answering, or answered in a manner that the query sender found to be unfavorable, then the query would be forwarded to the attending of the services for secondary consideration.

 

MissingLinkText

Intervention

Development

To improve documentation, the approach taken to develop the automated documentation tool (AutoDx) was to generate text within a provider note based on specific conditions met within the EHR for each individual patient. AutoDx uses functionality already within the EHR, but with a novel use case. In a simplistic summary, the process starts with the creation of basic logic rules based on data elements within the EHR, such as laboratory values, vital signs, or discrete flowsheet entries. The rules, either as a single condition or combined, can be leveraged to trigger patient-specific messages, which can then be placed within the text of a provider note.

The initial diagnoses targeted were identified based on a review of retrospective data of the most frequently queried diagnoses by the CDI team. The messages were created to align with the formatting being used in the note template. The logic used to generate the message was conceptualized in three distinct archetypes ([Fig. 1]): (1) a single data element that generates a diagnosis message, such as obesity and malnutrition; (2) a single data element plus a provider order that generates a diagnosis message, such as hypokalemia, hypophosphatemia, and hypomagnesemia; (3) insufficient information from a data element or order requiring provider input from a curated list of options, such as brain edema and types of congestive heart failure.

Fig. 1 Diagnosis archetypes.

The tool's use was expanded to other services and additional diagnoses were developed, including those in the third archetype. These new diagnoses required input from the provider to complete the documentation. The latest version of AutoDx contained support for 10 specific diagnoses associated with nine different risk variables, noted in parentheses: cerebral edema (Brain Conditions), heart failure with reduced ejection fraction (Congestive Heart Failure), clinically significant coagulation disorders (Coagulopathy), COVID significant decreased lymphocyte count (COVID Lymphopenia), hypokalemia (Electrolyte Disorders), hypomagnesemia (Mineral Metabolism Disorders), hypophosphatemia (Mineral Metabolism Disorders), obesity (Obesity), consistently significantly elevated systolic blood pressure (Hypertensive Urgency/Emergency), and moderate or severe malnutrition based on assessment by a registered dietician (Malnutrition).

All archetypes of AutoDx were based on logic rules configured in our base EHR functionality. For the first two archetypes, the logic rules searched the chart for a discrete data element (obesity or malnutrition) or a combination of data elements with order(s) (hypomagnesemia or hypokalemia) to display formatted diagnosis. For the complete AutoDx tool, these logic rules additionally searched for narrative result text from imaging reports, such as computed tomography scans, magnetic resonance imaging scans, or echocardiogram, containing key terms and patterns of change in laboratory results or vitals. When the logic rules matched a criterion, a message displayed in the note text alerting the authoring provider that a condition was met. The author then clicks on an embedded hyperlink in the message text which opens a pop-up with a predefined list of possible diagnoses based on the condition met. The author selects an appropriate diagnosis (or declines), which files it as flowsheet data. Upon returning to the note and refreshing the text, an inline diagnosis and brief assessment and plan is documented in the narrative. Hard stops were added to prevent signing and completion of the note until all conditions were addressed.

 

MissingLinkText

Oversight

The project was directly overseen by the medical center's IT team with executive oversight from the institution's chief medical information officer. During the tool's development, the CDI team conducted regular reviews to confirm that the logic elements defining each message aligned with diagnosis criteria, ensuring the generated text accurately reflected these criteria while seamlessly integrating into the provider note's normal context. Providers also evaluated the logic and message to ensure they were clinically appropriate and consistent with standard provider documentation of a similar diagnosis. When the process was in the final phases of testing, the entire procedure was reviewed with leadership from Compliance, Risk Management, and Legal to ensure it was appropriate for the final medical record and to accurately be used for billing purposes. It was then integrated into the note template for the pilot provider group. Utilization of the tool and accurate diagnoses were monitored carefully by the clinical, IT, and CDI teams. With the development of each new diagnosis, Physician Builders, physician champions in the specialty, IT, and CDI evaluated both the logic and associated messages. A summary of the logic of all conditions is summarized in the [Supplementary Appendix].

 

MissingLinkText

Implementation

Pilot

The first pilot phase of the AutoDx tool was with the Hospital Medicine provider group. This group was entirely composed of attendings, whose main responsibility was clinical care on inpatients services by themselves or in conjunction with advance practice providers, who are integrated into the Hospital Medicine practice. Diagnoses included in the initial version were hypokalemia, hypomagnesemia, hypophosphatemia, obesity, and malnutrition as documented by a registered dietician. These diagnoses would populate into the note if the conditions were met ([Fig. 2A]). In situations of electrolyte deficiency without an order for correction, a prompt was placed to point out the diagnosis, but provider input was required to explain why correction was not ordered. In this method, the functionality of the tool would have to be broken to allow for edits or the provider would have to exit the note, order the replacement, then refresh the tool within the note to update the message. During the use of the tool, it was updated to allow providers to provide responses for supplementation not being given, while keeping the tool functional ([Fig. 2B]). The tool was incorporated into the default note templates accessible to all users in the Hospital Medicine group. The templates, including history and physical (H&P) and progress notes, would automatically populate based on the note type selected. When writing the notes, providers were still able to remove the AutoDx messages from the note at their discretion, in a manner similar to deleting regular text or vital signs automatically pulled into the note. A brief informational session about the tool was conducted at a staff meeting prior to its rollout. Providers were encouraged to use the default template both to use the tool and to allow for uniformity of note type within the group to better facilitate transitions of care between providers.

Fig. 2 Screenshots of AutoDx in Notes (data presented in the above are imaginary).

 

MissingLinkText

 

MissingLinkText

Broader Implementation

Based on the initial success of the pilot implementation, the AutoDx tool was augmented to meet the needs of the second implementation group, the Neuro Intensive Care Unit (NICU). The augmentation included development of new diagnoses, including Brain Edema, which would prompt a provider with a potential diagnosis that required more detailed response by the provider ([Fig. 2C]). This user group included residents, primary in Neurology, plus neurointensivist attendings. The AutoDx tool was inserted into standard note templates for NICU. Implementation of the AutoDx tool for the NICU involved a comprehensive approach supported by the quality lead for the unit, including development of new NICU condition-specific note templates and multiple educational sessions with key stakeholders. The educational sessions were tailored to reach different members of the NICU team, including residents from neurology and neurosurgery, faculty attendings, and advanced practice providers (APPs).

Educational sessions were tailored to accommodate the diverse roles, availability, and documentation requirements of different NICU providers. Ensuring inclusivity, sessions were conducted for residents in neurology and neurosurgery, faculty attendings in neurocritical care and in one session all neurology faculty, and APPs. Recognizing the importance of attention span and engagement, sessions for residents and APPs were kept concise, not exceeding 20 minutes each, and conducted multiple times per week throughout the month. This approach aimed to ensure redundancy without overwhelming participants with lengthy didactic sessions. Additionally, sessions were integrated into residency meetings to ensure exposure for all residents, even those not currently rotating in the NICU.

For attending physicians, sessions were integrated into faculty and section meetings. Attendings were provided with insights into the purpose and development of the tool through real-time demonstrations of its functionality, emphasizing its role in resident-driven documentation. Attendings were encouraged to embrace AutoDx by ensuring its presence in their respective notes and to extend this practice to nonneurointensive care faculty, considering patients' continued encounters with AutoDx notes upon transfer to other units.

Educational sessions were not theoretical only, offering practical guidance on navigating the AutoDx tool in clinical practice. Providers received hands-on training and troubleshooting strategies to address implementation challenges effectively. To ensure ongoing support and understanding, the educational sessions were scheduled in a structured manner monthly during the implementation period. Champions from the resident teams were identified to work with this cohort of providers to encourage use. Their involvement also was intended to help identify and address specific concerns or challenges that arose for the residents.

Furthermore, as a part of the implementation strategy, small batch retrospective quality checks were conducted. These checks were designed to verify that the AutoDx tool was being utilized correctly and that the information entered aligned with the intended diagnostic criteria.

 

MissingLinkText

 

MissingLinkText

Study of the Intervention

Measures

The primary outcome measure of the intervention is the volume of coding queries generated for the target diagnoses. The coding query data were provided directly from the CDI team's internal charting which was separate from the EHR. Data on coding queries were made available in a reliable fashion starting in April 2021, for the pilot implementation. Utilization of the AutoDx tool was ascertained through the presence of notes from encounters tagged with a data element distinct to AutoDx within the EHR. Post-implementation data collection was planned for approximately 1 year after the pilot was initiated. The values were aggregated as a proportion relative to the volume of discharges completed by the group over the period of time being evaluated. The primary outcome measure of the intervention was reduction in coding queries. Secondary outcomes included case-mix index (CMI) and observed minus expected length of stay (O-E LOS) accounting for both changes in documentation and actual clinical practice.

 

MissingLinkText

Analysis

Evaluation of the tool's implementation consisted of comparisons of proportions of coding queries per patient encounter volume, as calculated by a Chi-square test, and were evaluated over time using run charts. These changes are compared in a pre–post fashion with the month of implementation being excluded from the analysis from both the initial pilot and the NICU implementation. Conditions with high volumes of queries, but not targeted by the tool, were used as a control. The specific high-volume non-AutoDx diagnoses were unique to each implementation group. Changes in cumulative outcome measures, including CMI and observed to expected length, were evaluated using t-test and run charts.

 

MissingLinkText

 

MissingLinkText

 

MissingLinkText

Results

Pilot

The pilot of AutoDx started in mid-February 2020. Coding query data were captured from January through March 2020. The analysis compared a full month, January 2020, pre-implementation, to a full month, March 2020, post-implementation. The Hospital Medicine service discharged 590 patients in January and 531 in March. The number of average coding queries per patient for AutoDx diagnoses was 0.15 pre versus 0.07 post (p < 0.001). The highest query volume non-AutoDx diagnoses, detailed in [Table 1], averaged 0.12 per patient pre-implementation versus 0.12 post-implementation (p = 0.77). Additional months were not included in the data analysis as the institution started a dedicated unit for the treatment of COVID-19 at the end of March 2020, which was mainly staffed by hospitalist providers initially involved in the AutoDx pilot.[16] During the initial implementation period, no significant concerns were reported regarding the tool, and no significant malfunction-related changes were made to the tool. Due to the small amount of data available and the potential inclusion of COVID-19 patients in the post-implementation dataset, encounter-level outcomes were not evaluated.

Table 1 Hospital medicine AutoDx pilot coding queries per 100 patients pre- and postimplementation

Coding query	Query type	Pre (n = 590)	Post (n = 531)	Relative reduction (%)	p-Value
Electrolyte disturbances	AutoDx	4.6	0.4	−92	<0.0001
Malnutrition	AutoDx	3.9	3.0	−23	0.42
Obesity	AutoDx	3.9	2.4	−37	0.18
Mineral metabolism disorders	AutoDx	2.4	1.1	−52	0.12
Total	AutoDx	14.7	7.0	−53	<0.0001
General query	High Volume	3.1	4.0	30	0.41
CHF	High Volume	2.4	2.1	−13	0.73
Uncertain diagnosis	High Volume	1.4	2.3	67	0.25
CKD stage	High Volume	2.0	0.8	−63	0.07
Clinical validation	High Volume	1.7	1.1	−33	0.43
Signs/symptoms	High Volume	1.4	2.3	67	0.25
Total	High Volume	11.9	12.4	5	0.77

Abbreviations: CHF, congestive heart failure; CKD, chronic kidney disease.

Note: Italics represent statistical significance.

 

MissingLinkText

Neonatal Intensive Care Unit Implementation

The AutoDx tool was implemented for the NICU team in May 2022 for testing and utilized clinically starting in June 2022. In the pre-implementation time frame, April 2021 to May 2022, there were 826 admissions to the NICU. During the implementation month of June 2022, there were 69 NICU admissions. Post-implementation from July 2022 to completion of data collection in April 2023, there were 680 NICU admissions. Following the full implementation of AutoDx, 93.5% (636/680) of NICU admissions utilized the tool at least in one note during an encounter. The utilization of AutoDx by month is summarized in [Fig. 3]. Pre-implementation there were 1,280 coding queries at a rate of 1.55 queries per admission compared with 443 coding queries at a rate of 0.65 queries per admission (p < 0.0001) post-implementation. The proportion of AutoDx coding queries to all other coding queries decreased from 54.9 to 37.1% (p < 0.0001). The change in specific coding query volumes is summarized in [Table 2], which includes all of the AutoDx diagnoses and the top six highest volume non-AutoDx coding queries. Two additional diagnoses, anemia and chronic kidney disease, were added to AutoDx during the post-implementation cycle, and are included in the overall analysis, but are not represented in the table.

Fig. 3 NICU admissions by AutoDx Note template utilization. NICU, Neuro Intensive Care Unit.

Table 2 NICU coding queries per 100 patients pre- and post-implementation

Coding query	Query type	Pre (n = 829)	Post (n = 680)	Relative reduction (%)	p-Value
Obesity—adult	AutoDx	13.3	3.8	−71	<0.0001
Mineral metabolism disorders	AutoDx	9.9	1.8	−82	<0.0001
Brain condition	AutoDx	9.8	2.4	−76	<0.0001
Electrolyte disturbances	AutoDx	6.7	3.7	−45	0.01
CHF	AutoDx	4.7	1.5	−68	0.0005
Coagulopathy	AutoDx	2.9	1.0	−66	0.004
Hypertensive urgency/emergency	AutoDx	1.7	0.0	−100	0.0007
Malnutrition	AutoDx	1.1	0.7	−36	0.48
COVID lymphopenia	AutoDx	0.1	0.0	−100	0.36
Total	AutoDx	50.2	14.9	−70	<0.0001
Abnormal findings (not electrolyte disturbances)	High volume	9.6	4.1	−57	<0.0001
Clinical validation	High volume	8.7	6.3	−28	0.09
Uncertain diagnosis	High volume	7.7	3.2	−58	0.0002
Encephalopathy	High volume	6.2	4.0	−35	0.06
General query	High volume	5.6	3.5	−38	0.06
Present on admission (POA)	High volume	4.8	4.0	−17	0.42
Total	High volume	42.6	25.1	−41	<0.0001

Abbreviations: CHF, congestive heart failure; NICU, Neuro Intensive Care Unit.

Note: Italics represent statistical significance.

The proportion of coding queries over time as a function of patient volume is summarized in [Fig. 4] in u-charts. Both AutoDx and Non-AutoDx queries demonstrated significant shifts pre- and post-implementation (solid ovals). There were significant outliers in the pre-implementation period for both groups (dotted circles), but there were only outlier points post-intervention in the AutoDx group.

Fig. 4 u-Charts of coding query volume for AutoDx and non-AutoDx diagnoses.

The outcome measures are summarized in [Table 3]. CMI demonstrated a significant increase post-implementation (4.00 vs. 4.55, p = 0.02), but there was no significant difference in O-E LOS or in O:E mortality. The total number of encounters with O:E mortality calculated was limited due to the availability of data in the CDI database and not all encounters had primary diagnoses related to a mortality risk score.

Table 3 Outcomes comparison NICU

	Pre (n = 836) Average (SD)	Post (n = 680) Average (SD)	p-Value
CMI	4.00 (4.12)	4.55 (5.08)	0.02
O-E LOS	3.98 (9.45)	3.34 (9.99)	0.21
O:E Mortality[a]	0.152 (0.253)	0.177 (0.275)	0.10

Abbreviations: NICU, neonatal intensive care unit; O:E Mortality, observed to expected mortality rate, measured as a ratio of observed deaths to expected death based on risk model; O-E LOS, observed minus expected length of stay, measured in days; SD, standard deviation.

Note: Italics represent statistical significance.

^a Pre (n = 657); post (n = 645).

 

MissingLinkText

 

MissingLinkText

Discussion

The implementation of the AutoDx tool was associated with a significant decline in coding queries to providers in nearly all of the targeted diagnoses in two separate implementations. In the NICU implementation, there was also a noticeable increase in the CMI for the targeted patient population. Neither implementations had concerns raised regarding the accuracy or safety of the tool.

During the initial pilot implementation, only one type of targeted query demonstrated statistically significant improvement, but all categories had a downward trend. The limited duration of data collection secondary to changes in clinical practice from the COVID-19 pandemic led to the analysis being underpowered. Non-AutoDx high-volume diagnoses showed variable levels of changes, but when congregated there was no significant change. The lack of an evaluation of AutoDx at the encounter outcome level was a limitation to the evaluation of the efficacy of the pilot. Once clinical operations returned to normal in Hospital Medicine, there were changes in service structure and to the software used for creating and tracking coding queries, so a larger pre–post comparison would not have been valid.

In the NICU implementation, two of the AutoDx diagnoses, malnutrition and COVID Lymphopenia, did not show a significant decrease in the rate of coding queries per encounter. The incidence of both was the lowest of all the targeted diagnoses. The baseline rate of coding queries for malnutrition was considerably lower in the NICU than in medicine patients over a similar timeframe, which was approximately 5%.[17] This lower level would have required a larger sample size to show significance at the level of improvement reported. However, a similar EHR-based intervention to improve malnutrition documentation was reported to improve diagnosis capture to 100% of opportunities identified.[18] This study utilized similar functionality in the EHR, but had a significant focus on ensuring appropriate documentation of nutritional status by other care providers, nursing and dietician. On manual review of missed opportunities of AutoDx, aside from not utilizing the tool, cases of coding queries for malnutrition were often seen when the dietician did not document the specific malnutrition diagnosis in the flowsheet to trigger the AutoDx message. This is an opportunity for further improvement of the tool's overall performance.

Two non-AutoDx high-volume diagnoses did demonstrate significant improvement following implementation, Abnormal Findings and Uncertain Diagnosis. These are both purposefully vague categories, but Abnormal Findings specifically does not include any queries related to abnormal electrolytes. Near the time of implementation there was additional emphasis placed on the primary teams within the NICU to be mindful about their documentation and to be responsive to coding queries. The improvement in capturing the nonelectrolyte diagnoses within AutoDx may have been another driver in this reduction of these categories as the standardized wording used for the diagnoses would reduce the variation and was built to be compliant with coding standards for those diagnoses. The high-volume diagnoses, Encephalopathy and Present on Admission, which would not have been affected by the implementation of AutoDx, showed no significant change. This refutes the impact on overall documentation education, as encephalopathy is a frequent diagnosis on the neurology service.[19] [20] [21] Present on Admission is a condition that could connect to any diagnosis, but was not integrated as an additional explicit part of AutoDx. However, the tool was implemented into History and Physical note templates, so the targeted conditions should have been captured more frequently as present on initial documentation.

The method in which education was provided in the implementation cohorts was quite different, with the Hospital Medicine group receiving very little, and the NICU receiving a more comprehensive program. The method in which the tool was implemented within the EHR for both cohorts was the same, through a rapid change to default note templates and auto-population of these note templates. These different aspects of implementation likely were the drivers of the differences in both scales of outcome, but also in the difference in improvement in non-AutoDx diagnoses for the NICU team. As opposed to Hospital Medicine, the NICU team had methods for dissemination of information rapidly and with close follow-up, which lends itself well to overall improvement and not just in the targeted diagnoses. The evaluation of implementation of another cohort at the study institution is underway, which was completed in a similar fashion to Hospital Medicine, and appears to be trending in a similar direction with significant targeted query improvement, but limited change in other diagnoses.

Sustained utilization of the AutoDx tool was supported by the emergence of physician champions within both the Hospital Medicine and NICU cohorts. These champions played a key role in facilitating adoption among their peers. In Hospital Medicine, many champions came from the EHR physician builder group, which formed a strong base of support for implementation. For both groups, the champion had an informal role that was identified by those willing to give feedback on the tool, but also had sufficient understanding to provide at-the-shoulder support to colleagues.

One limitation of the NICU analysis is that it only included encounters of patients admitted directly to the NICU, excluding those transferred from other services. This methodology missed opportunities for additional capture of diagnoses using AutoDx for patients transferred from other services. However, given the planned dissemination of AutoDx eventually to all services, the impact of this specific population was not the intended target of the intervention or evaluation.

Another limitation is that the primary process measure was the volume of coding queries rather than the actual incidence of the underlying conditions and appropriate documentation. A decrease in the frequency of coding queries should correlate with improved documentation overall of the targeted conditions, but we did not specifically look at clinical elements of the encounters to identify opportunities to document each of the targeted diagnoses.

The study did not have a clear quantitative balancing measure for the implementation of AutoDx. The continued high utilization of the tool is a strong indicator of provider satisfaction with its use as demonstrated in [Fig. 3], but this is an indirect indicator. Even with the sustained use of the tool, additional diagnoses were developed after the initial implementation period that could reflect missed diagnoses or redundant documentation. One of these diagnoses is anemia, which in a large cohort of use was either listed as “Already Documented” or only “Other” in less than 9% of all cases, indicating a >90% success rate. Informally, leadership did not receive significant negative feedback to the use of the tool. There could still be factors related to the time spent using the tool that might impact a provider's satisfaction.[22] However, physician documentation completion time has been demonstrated to be highly variable, making it an unreliable balancing measure.[23] Providers who utilized AutoDx deemed the time saved from addressing subsequent coding queries to be far more significant than the time spent using the tool in qualitative feedback. In ongoing evaluations of AutoDx implementation in new provider cohorts, we have completed intake surveys and plan to survey post-implementation to capture provider attitudes and concerns regarding the use of the tool. These results will be disseminated once follow-up surveys are completed.

 

MissingLinkText

Conclusion

Implementation of the documentation improvement tool, AutoDx, was associated with a significant decline in overall queries and queries for nearly all targeted diagnoses, and an increase in CMI over the same period. This highly scalable and customizable tool not only reduces physician burden and enhances documentation quality but also lends itself to adoption in other institutions.

 

MissingLinkText

Clinical Relevance Statement

The use of messages triggered by clinical factors in provider notes to support documentation demonstrated a significant reduction in coding queries sent to providers. The real-time support also yielded a demonstrative improvement in the documentation of illness severity.

 

MissingLinkText

Multiple-Choice Questions

1. The documentation of diagnoses using automated messages based on data elements, such as laboratory results, vital signs, and provider orders without additional provider or consultant input, is appropriate for which of the following diagnoses?

   • Hypokalemia without an order for replacement medications

   • Malnutrition

   • Brain edema

   • COVID Lymphopenia

   Correct Answer: The correct answer is option d.

2. Which encounter level outcome measure was definitively improved by the implementation of a documentation improvement process utilizing automated messages within provider notes?

   • Case-mix index

   • Readmissions

   • Observed mortality rate

   • Hospital-acquired infections

   Correct Answer: The correct answer is option a.

 

MissingLinkText

 

MissingLinkText

Conflict of Interest

None declared.

Acknowledgments

We would like to thank Ben Petro and Robert Strong for their insights and support for the development and growth of this project.

Protection of Human and Animal Subjects

This project received a formal Determination of Quality Improvement status according to University of Chicago Medicine institutional policy. As such, this initiative was deemed not human subjects research and was therefore not reviewed by the Institutional Review Board.

• References

• 1 Lorkowski J, Pokorski M. Medical records: a historical narrative. Biomedicines 2022; 10 (10) 2594
• 2 Division HIP. Individuals' Right under HIPAA to Access their Health Information 45 CFR § 164.524. HHS.gov. Published January 5, 2016. Accessed September 18, 2023 at: https://www.hhs.gov/hipaa/for-professionals/privacy/guidance/access/index.html
• 3 Sanderson AL, Burns JP. Clinical documentation for intensivists: the impact of diagnosis documentation. Crit Care Med 2020; 48 (04) 579-587
• 4 Renée Brown L. The secret life of a clinical documentation improvement specialist. Nursing 2013; 43 (02) 10
• 5 3M 360 Encompass System | 3M Health Information Systems. Accessed September 18, 2023 at: https://www.3m.com/3M/en_US/health-information-systems-us/improve-revenue-cycle/360-encompass-system/
• 6 AHIMA. Guidelines for Achieving a Compliant Query Practice (2019 Update). Guidel Achiev Compliant Query Pract 2019 Update AHIMA Am Health Inf Manag Assoc. Published online February 5, 2019. Accessed October 20, 2023 at: http://bok.ahima.org/doc?oid=302673
• 7 Reyes C, Greenbaum A, Porto C, Russell JC. Implementation of a clinical documentation improvement curriculum improves quality metrics and hospital charges in an academic surgery department. J Am Coll Surg 2017; 224 (03) 301-309
• 8 Esper P, Walker S. Improving documentation of quality measures in the electronic health record. J Am Assoc Nurse Pract 2015; 27 (06) 308-312
• 9 Castaldi M, McNelis J. Introducing a clinical documentation specialist to improve coding and collectability on a surgical service. J Healthc Qual 2019; 41 (03) e21-e29
• 10 Patel A, Ali A, Lutfi F, Nwosu-Lheme A, Markham MJ. An interactive multimodality curriculum teaching medicine residents about oncologic documentation and billing. MedEdPORTAL 2018; 14: 10746
• 11 Paz KB, Halverstam C, Rzepecki AK, McLellan BN. A national survey of medical coding and billing training in United States Dermatology Residency Programs. J Drugs Dermatol 2018; 17 (06) 678-682
• 12 Kapa S, Beckman TJ, Cha SS. et al. A reliable billing method for internal medicine resident clinics: financial implications for an academic medical center. J Grad Med Educ 2010; 2 (02) 181-187
• 13 Hebal F, Nanney E, Stake C, Miller ML, Lales G, Barsness KA. Automated data extraction: merging clinical care with real-time cohort-specific research and quality improvement data. J Pediatr Surg 2017; 52 (01) 149-152
• 14 Furmaga J, Courtney DM, Lehmann CU. et al. Improving emergency department documentation with noninterruptive clinical decision support: An open-label, randomized clinical efficacy trial. Acad Emerg Med 2022; 29 (02) 228-230
• 15 Cerasale M, Nguyen K, Cui M, Cao K. Hospitalist builders: maximizing the physician potential to improve the EHR. In: Journal of Hospital Medicine. Accessed September 20, 2023 at: https://shmabstracts.org/abstract/hospitalist-builders-maximizing-the-physician-potential-to-improve-the-ehr/
• 16 D'Souza FR, Murray JP, Tummala S. et al. Implementation and assessment of a proning protocol for nonintubated patients with COVID-19. J Healthc Qual 2021; 43 (04) 195-203
• 17 Cerasale M, Kao CK. A summary of three years of optimizing documentation. Presented at: Epic Users' Group Meeting: Madison, WI; 2023
• 18 Parkash O, Jamil S, Chudzinski V. et al. Academy for Quality and Safety Improvement (AQSI) project to improve diagnosis and documentation of malnutrition in a community hospital. BMJ Open Qual 2023; 12 (04) e002415
• 19 Radmard S, Epstein SE, Roeder HJ. et al. Inpatient neurology consultations during the onset of the SARS-CoV-2 New York City pandemic: a single center case series. Front Neurol 2020; 11: 805
• 20 Chowdhury RN, Hasan AT, Ur Rahman Y, Khan SI, Hussain AR, Ahsan S. Pattern of neurological disease seen among patients admitted in tertiary care hospital. BMC Res Notes 2014; 7 (01) 202
• 21 Bacellar A, Pedreira BB, Costa G, Assis T. Frequency, associated features, and burden of neurological disorders in older adult inpatients in Brazil: a retrospective cross-sectional study. BMC Health Serv Res 2017; 17 (01) 504
• 22 Budd J. Burnout related to electronic health record use in primary care. J Prim Care Community Health 2023; 14: 21 501319231166921
• 23 Florig ST, Corby S, Rosson NT. et al. Chart completion time of attending physicians while using medical scribes. AMIA Annu Symp Proc 2022; 2021: 457-465

Publication History

Received: 27 February 2024

Accepted: 02 May 2024

Accepted Manuscript online:
03 May 2024

Article published online:
26 June 2024