COVID-19 and the Electronic Health Record: Tool Design and Evolution at the U.S. Pandemic Epicenter

Abstract Objective  We detail inpatient electronic health record (EHR) system tools created at Mount Sinai Health System for the clinical management of patients with coronavirus disease 2019 (COVID-19) during the early pandemic months in the U.S. epicenter, New York City. We discuss how we revised these tools to create a robust Care pathway, unlike other tools reported, that helped providers care for these patients as guidelines evolved. Methods  Mount Sinai Health System launched a Command Center on March 8, 2020. The Chief Medical Information Officer launched a workgroup of clinical informaticists and Epic analysts tasked with rapidly creating COVID-19-related EHR tools for the inpatient setting. Results  Initial EHR tools focused on inpatient order sets for care standardization and resource utilization. In preparation for a fall 2020-winter 2021 surge, we created a clinician-facing, integrated Care pathway incorporating additional Epic System-specific tools: a Care Path, a dedicated Navigator, Summary and Timeline Reports, and SmartTexts. Discussion  Initial tools offered standard functionality but included complex decision-making support to account for the lack of COVID-19 clinical knowledge, operational challenges during a dramatic patient surge, and resource limitations. We revised content and built a more comprehensive Care pathway that provided real-time clinical data along with treatment recommendations as knowledge evolved, e.g., convalescent plasma. Conclusion  We have provided a framework that can inform future informaticists in developing EHR tools during an evolving pandemic.


Introduction
New York City was the earliest U.S. epicenter of the coronavirus disease 2019 (COVID-19) pandemic, with its first case in March 2020 and a surge to more than 5,000 confirmed cases per day. 1 ►Fig. 1 provides context on the initial surge. Others have previously described organizational structures, 1 documentation and order panels, 2,3 data visualization tools, 3,4 reporting tools, 3,5,6 clinical decision support (CDS), 7,8 and operational informatics tools. 9, 10 Care pathways have been described in the informatics literature as "structured multidisciplinary care plans that detail essential steps in the care of patients with a specific clinical problem [and] offer a structured means of developing and implementing local protocols of care based on clinical guidelines." 8 Care pathways, particularly those integrated into the electronic health record (EHR), have been shown to improve in-hospital outcomes, 11,12 increase the detection, diagnosis, and treatment of specified conditions, 13 and improve utilization of resources. [14][15][16] Care pathways represent the evolution of previously described tools that integrated clinical guidelines or care recommendations within the provider workflow. 17 This report outlines the development of order sets, novel data visualization tools, and a novel COVID-19 Care pathway within the context of a rapidly escalating pandemic.

Objective
In this case report, we detail the inpatient EHR clinical tools created at the Mount Sinai Health System (MSHS) for the management of patients with COVID-19 during the early months of the pandemic. We describe how these tools evolved into a novel clinician-facing, integrated Care pathway that aggregates clinical information and COVID-19specific tools into a navigator that interprets the patient's clinical status and provides treatment recommendations and lessons we learned about pandemic realities versus best practices in the literature.

Methods
The Mount Sinai Health System is an academic medical center and health system comprising more than 300 hundred ambulatory practices and eight hospitals in the New York City metropolitan area, six of which currently use the Epic EHR (Epic Systems Corporation, Verona, Wisconsin, United States). MSHS launched a pandemic Command Center (CC) on March 8, 2020, composed of senior leadership across the health system, including the president and chief executive officers of each hospital, the system's chief medical information officer (CMIO), and system chairs for Infectious Disease, Hospital Epidemiology, and Pulmonary/ Critical Care. The CMIO formed a small informatics workgroup (three clinician informaticists and 6-8 Epic analysts) tasked with creating as quickly as possible COVID-19-related EHR tools for the inpatient setting. All requests came from the CC, and all tools developed were reviewed by the appropriate health system clinical department chairs prior to implementation. The workgroup created clinical decision support, computerized provider order entry, and data visualization tools deemed essential for caring for COVID-19 patients. The workgroup presented bi-weekly in meetings with a standing EHR-governance council of about two dozen inpatient clinical informaticists from across the system who provided critical feedback during tool development. ►Fig. 1 highlights the daily case rate in New York City, hospitalization rates within our institution, and provides context against which this build work was occurring. 18 For reporting purposes here, the efforts of this workgroup are divided into two phases, Phase 1 (mid-March to mid-June 2020) and Phase 2 (September to December 2020). The timeline of the efforts of this workgroup is presented in ►Fig. 2 and accompanies ►Table 1, which outlines major milestones for each tool.

Phase 1 Inpatient Tools
There were many unknowns regarding caring for patients with COVID-19 in the inpatient setting during the early phase of EHR tool building. Our informatics workgroup aimed to quickly and efficiently create the inpatient tools needed to disseminate information and standardize care for patients with COVID-19.
The first comprehensive inpatient tool created was the COVID-19 order set. With prior experience in bundling diagnostic and treatment options within one order set (for conditions such as community-acquired pneumonia), this seemed like the natural first step in providing decision support for clinicians. The order set included testing for SARS-CoV-2, appropriate isolation precautions, and preselected laboratory tests. Providers would select clinical severity, then were provided with corresponding medication recommendations, including post-treatment monitoring. A set of cascading questions regarding clinical severity, weight, and creatinine clearance was later added to guide anticoagulation dosing (►Fig. 3). Needs were identified by clinical leadership in response to: updated treatment recommendations; availability of resources (including testing capacity, medication supply, etc.); and unforeseen challenges that required course redirection.
As COVID-19 hospitalization rates skyrocketed, and non-COVID related admissions plummeted, these standalone order sets were integrated into the general medicine admission and intensive care admission order sets to streamline the ordering process. A patient-specific summary report was created to centralize important clinical information including vital signs, oxygen requirement or ventilator settings, inflammatory markers over the last 72 hours, microbiology data, and details about the treatment team.

Phase 2 Inpatient Tools
As COVID-19 cases decreased over the summer, MSHS began preparations to manage a potential fall/winter surge. We created an EHR-integrated clinical Care pathway that provided decision support and treatment guidance and included the following Epic-specific tools: a Care Path, a dedicated Navigator, Order Sets, Summary and Timeline Reports, and SmartTexts.

Care Path
A Care Path is an Epic-specific decision support tool that visually outlines treatment options for a particular condition and updates in real-time based on the documentation in the chart. Our prior experience with Care Paths in the inpatient setting involved management of patients with respiratory failure in the intensive care unit. The benefits of this tool include allowing providers to easily track clinical progress and suggest appropriate interventions at various stages of the patient's clinical course.
In designing a COVID-19 care path (►Fig. 4), oxygen supplementation became the designated milestone to track across the patient's clinical course because disease severity was defined by the degree of respiratory failure. All available oxygen delivery devices were associated with one of the milestones shown. Milestones were "achieved" based on clinical data documented in nursing vitals flowsheets and updated in real time. Each milestone had a corresponding treatment recommendation, including steroid and anticoagulation dosing. The Care Path also differentiated between escalating and de-escalating oxygen requirements, and provided discharge recommendations when oxygen requirements decreased below 3 L of oxygen by nasal cannula. The Care Path was triggered by a specific Care

Navigator
Based on our earlier experience with a COVID-19 Summary Report, we believed it was important to create a centralized location for the management of patients with COVID-19. In Epic, a dedicated Navigator allowed for the various tools we developed to be accessed from the same screen. Importantly, we were able to dynamically display the Navigator more prominently in the patient's chart based on infection status (i.e., person under investigation or positive SARS-CoV-2 PCR result). This addressed a shortcoming of the COVID-19 summary report, which was the inability to automatically display the report upon opening the chart of a COVID-19 patient.
We leveraged the existing COVID-19 summary report in the COVID-19 navigator and made optimizations (►Fig. 5), including a direct link to the COVID-19 Care Path. SmartText was added to the top of the summary report highlighting treatment recommendations based on the patient's documented oxygen therapy. This SmartText directly corresponded with the COVID-19 Care Path.
We also provided a link to a custom-built COVID-19 timeline report which displays COVID-19-specific treatments chronologically along with oxygen requirements or ventilator settings, use of prone positioning, and graphical representation of inflammatory markers and fever curves (►Fig. 6). This timeline view provides another data visualization tool by which the provider can trace the patient's clinical course during the hospitalization. Fig. 3 Screenshot of the COVID-19 order set. Upon clinician selection of a particular severity of disease, the order set provides corresponding medication options. Treatment recommendations were incorporated within the order set. Because changes to recommendations occurred rapidly and frequently as COVID-19 knowledge grew and the complexity of those recommendations, cascading orders were used to ensure appropriate dosing for anti-coagulation. Fig. 4 Pictured is the Clinical Care Path, which clinicians access from the Summary report within the COVID-19 navigator. The Care Path is organized by oxygen delivery device, with corresponding treatment recommendations. Patients proceed along the Care Path based on nursing flowsheet documentation of oxygen supplementation. Each treatment option is associated with a corresponding medication or nursing order. For example, an order for dexamethasone (ordered while the patient is documented to be on nasal cannula) will be displayed as a completed step. Completed steps are time stamped. Dark blue boxes with an icon of a person represent the most recent oxygen requirement or therapeutic intervention.   restrictors with multiple rules were created to check if result and documentation criteria were met; if one of the criteria was not met, the convalescent plasma order would not be available to the provider and would instead display criteria needed for ordering convalescent plasma.

Discussion
Although other institutions described inpatient clinical order sets and CDS tools meant to standardize and simplify care for patients with COVID-19, we were in the unique position of creating EHR-based tools during the first and then-unprecedented surge, and continued to improve our tools for use during the ongoing pandemic. We applied known best practices and learned pragmatic lessons for building EHRbased tools that may help health systems in future pandemics. Our approach may help others move more rapidly to develop more sophisticated tools to aid clinicians during similar crises.
First, during that spring surge, speed and simplicity were keys to creating successful tools. Our early efforts were focused on straightforward tools that could be built quickly and updated relatively easily. CDS ease-of-use, always important, 19 was particularly critical because many providers with little inpatient experience were being deployed to inpatient care. Though technically simple, designing these tools required complicated clinical decision-making. Potential diagnostic and treatment options were being announced at a rapid rate, often outside the standard channel of peerreviewed academic journals. This clinical uncertainty added a degree of caution to what options we made available to providers. Moreover, order set composition had to balance clinical necessity (e.g., trending inflammatory markers), and resource limitations (e.g., the turnaround time for obtaining a laboratory result).
Second, employing best practices to create well-integrated CDS tools made treatment recommendations more readily available to providers within the time constraints of an evolving pandemic. 19 Integrating treatment recommendations within the order set was an essential component to this. Additionally, noting that our initial data visualization tools were not centralized within the EHR, potentially discouraging use, utilization of the navigator aggregated these tools, thereby embedding them within the chart review workflow next to these clinical decision support tools. We chose not to incorporate intrusive Best Practice Advisory pop-up alerts to avoid alert fatigue. 20 The tools also decreased the need to leave the EHR, which would have been the case if we relied on email communication or an intranet site.
Limitations to our clinical tools and our processes offer a learning opportunity. There are limits to standardizing care within the EHR, regardless of EHR system used and the specific functionalities it offers. Our tools relied on clinician judgment, rather than forced functions, to adhere to treatment recommendations (excluding the convalescent plasma workflow). In practice that meant that although our order set included recommended dosing for steroids, providers were still able to order these medications outside of the order set at a dosage of their choosing. Given previously discussed challenges early on, particularly the limited availability of evidence to guide therapy, we sacrificed some forced standardization to allow providers the flexibility to consider recommendations within the context of the patients for which they were caring.
Additionally, the complexity of managing patients with COVID-19 makes it difficult to create a linear or easy to digest treatment algorithm. Though respiratory status is the most important indicator, our tool does not provide guidance on how to manage COVID-19 related complications such as delirium, thromboembolism, or renal failure. These tradeoffs were necessary in the hopes of creating a tool that could be used by all providers, particularly if we again had to rely on those re-deployed from other specialties. Similar tradeoffs should be recognized and accepted in a similar future pandemic.
An additional limitation was our inability to track utilization of all the tools we created. Our primary goal was to provide clinicians with recommendations and tools needed to care for patients with COVID-19, and to ensure that resources were utilized judiciously. Quality or outcome measures were not identified because there was no framework to establish reasonable benchmarks. We therefore focused reporting resources on other measures (e.g., ventilator utilization). Additionally, though we were able to ensure that a large majority of patients were placed on the Care Path, we did not have access to information such as time spent within the summary report or various other activities within the navigator. This meant that we did not have sufficient data to further optimize our tools, such as improving tools providers used the most or removing those they did not use.
Lastly, our experience highlights the limitations of being able to apply previously described frameworks during a pandemic. There is previous literature on user-centered or human-centered design and their effectiveness in creating useful EHR tools. 21,22 Our EHR governance council included multiple board-certified clinical informaticists and a PhDprepared informaticist, all expert members familiar with the CDS, usability, and user-centered design literature, who have previously employed these literature-driven best-practices for EHR build, usability testing, implementation, and usercentered optimization. [23][24][25] Unfortunately, the first surge involved innumerable unknowns, making it difficult to proceed in a deliberative, more structured manner. Shiffman et al have previously described validated frameworks to implement guidelines, yet our tools were created in an environment where guidelines did not exist, and recommendations were ever-changing. 26 Moreover, validated design processes, such as Schiffman's, are time and labor intensive and not practical when all personnel were in crisis-response mode with extended enterprise-designated pandemic duties. Jakob Nielsen's simplified model of usability engineering estimated that parts of the process would take at least 105 hours to complete. 27 This would have been untenable in our situation where decisions such as changes to anticoagulation recommendations were made and released within 48 hours. Lastly, rapidly changing events precluded structured processes for evaluating and updating tools. This is not surprising given that established frameworks to evaluate deployed EHR tools are known to be complex schemes with "multiple dimensions, categories, and measures that can be difficult to understand and apply in practice." 28 Even a simplified, practically focused framework such as SEIPS 101 may be too consuming to apply in another emergent pandemic situation. 29 Future research should focus on ultrarapid processes to develop and evaluate EHR tools. However, a key takeaway is that health systems that have access to a corps of literature-and best practice-informed clinical informatics experts 7 or integrate them into standing EHR-governance processes, 30 as we did, will be best positioned to act decisively in crises when more rigorous, intensive feedback/evaluation processes are impossible.

Conclusion
We detailed inpatient EHR clinical tools we developed for the management of COVID-19 patients, how we have refined these tools since the initial New York City surge, and how the tools became a robust clinician-facing, integrated Care pathway to provide comprehensive care to patients with COVID-19. Our best-practices approach in EHR-tool development to meet immediate clinical needs, then leveraging additional functionalities and evolving knowledge, and lessons learned, may inform and aid future informatics and EHR-build pandemic responses.

Funding
None.