Keywords
electronic health records - intensive care units - data display - user-computer interface
- software design - cognition
Background and Significance
Background and Significance
The volume of clinical information collected by modern electronic medical record (EMR)
systems creates a potential for cognitive overload, especially in data-rich environments
such as intensive care units (ICUs).[1] To draw physicians' attention to high-value information, we are designing a Learning
EMR (LEMR) system that utilizes statistical models of ICU physicians' data access
patterns to identify data that are most likely to be sought for each patient.[2]
[3]
[4] To be successful, this system must combine accurate predictions of data seeking
behavior with information displays that meet physicians' needs. This paper presents
the results of a qualitative investigation of the design of a LEMR system display.
Cognitive Overload in Intensive Care Units: the Performance-Altering Effects of Electronic
Medical Record Systems
Physicians must allocate limited cognitive resources to multiple competing tasks.[2] EMR systems should provide excellent cognitive support, but often fall short of
doing so.[5]
[6]
[7]
[8] Most commercial systems have limited capabilities for visualizing complex data,[9] and fragmentation of information across multiple screens increases the difficulty
of collating data and identifying trends and anomalies.[10] Reductions in efficiency due to overabundant data and suboptimal displays may impact
physicians' decision-making and performance,[11]
[12] potentially leading to medical errors and patient harm.[9]
[13]
[14]
[15]
[16] EMR systems that provide cognitive support may help improve patient outcomes.[17]
Approaches to Enhancing Cognitive Performance in the Intensive Care Unit
Despite several studies investigating ICU physicians' data needs and information-seeking
processes,[18]
[19]
[20]
[21]
[22] a thorough understanding of the cognitive support needs of ICU clinicians is yet
to be achieved.[18]
[21] Proposed designs intended to provide cognitive support to physicians[23] have explored techniques including encoding health information using visual attributes
such as color, position, size, or shape.[24]
[25]
[26] Displays have also used visualizations to summarize patient data and medical histories,[21]
[25]
[26]
[27]
[28]
[29] and to aggregate information from different sources in summary views,[10]
[23]
[30]
[31] organized by organ system[30] or around clinical concepts.[24]
[30]
[32] Other efforts have explored visual highlighting of trends[22] and key data in EMRs[33] or clinical notes,[34] and the adoption of configurable user interfaces.[35]
[36] While many of these approaches have been shown to reduce cognitive load[2]
[21]
[23]
[37] and improve physician decision-making,[30]
[32]
[38] they are not commonly used in healthcare.[7]
A Data-Driven Learning Electronic Medical Record System
We propose a novel data-driven approach that has the potential to enhance ICU physicians'
performance. The LEMR system uses data access patterns from past patient cases to
train models capable of predicting which data items are important in understanding
a patient's condition and how to treat it.[2] Items predicted to be of high value are highlighted in the LEMR system's user interface,
helping physicians focus on essential data.[2] Preliminary prediction models having AUROC values as high as 0.92[2]
[4] support that the data-driven approach is promising.
Objectives
The LEMR system requires a display capable of effectively highlighting the most relevant
information for each patient. To better understand user needs[39] and to create a design in which the prioritization of high-value data is an integral
part of the EMR, we decided to inform development with observations, interviews, and
focus groups. We present paper prototypes for proposed design solutions that will
inform the implementation of a functional user interface.
Methods
We combined the identification of relevant design themes in the literature with observations,
interviews, and design activities with ICU physicians ([Fig. 1]).
Fig. 1 Study workflow. A non-systematic qualitative evidence synthesis provided insights
on design principles applicable to a LEMR system, which inspired the creation of preliminary
paper prototypes. To gain an understanding of information needs and practices in ICUs,
we interviewed ICU physicians and observed their interactions with the current EMR
system. A focus group of ICU physicians generated design ideas for the LEMR system
and provided feedback on the concept and our prototypes, leading to the creation of
a final series of prototypes, which will inform LEMR display development. ICU, intensive
care unit; LEMR, Learning Electronic Medical Record.
Identification of Design Themes in the Literature and Preliminary Prototypes
Qualitative Evidence Synthesis and Prototype Development
We conducted a non-systematic qualitative evidence synthesis in January 2017, updating
our search in July 2019 ([Fig. 2]). We limited our search to articles published between 1996 (year of publication
of the first paper describing clinical data visualization techniques by Plaisant et
al[7]
[40]) and 2019, indexed in three databases (PubMed,[41] IEEE Xplore Digital Library,[42] ACM Digital Library[43]). We used a broad search string:
Fig. 2 Non-systematic qualitative evidence synthesis flow diagram. A non-systematic qualitative
evidence synthesis provided insights on design principles applicable to a learning
electronic medical record system designed for ICU care.
((EMR) OR (EHR)) AND ((data visualization) OR (dashboard) OR (design)) AND ((ICU)
OR (intensive))
We utilized citation search and reference lists from overview papers written by information
visualization experts[24]
[44]
[45] to find additional articles. Retrieved articles (n = 193) were screened by L.C. using title and abstract. To be selected, papers had
to describe the use or development of ICU data visualization techniques, and/or provide
design recommendations for the display of clinical data. We included literature reviews,
and excluded papers describing interfaces not designed for clinical care, not focused
on design aspects, not providing recommendations for the display of clinical data,
or not published in English. H.H. independently repeated the selection process. Any
disagreements were reconciled by consensus. Of 183 unique identified documents, 51
met the inclusion criteria (see [Supplementary Appendix A1] for full list [available in the online version]). L.C. and H.H. looked for insights
on design aspects to be considered when displaying ICU data, grouped articles dealing
with similar aspects, and used groupings to identify design themes applicable to a
LEMR system. L.C. extracted data, coded all documents using emergent coding,[46] and developed a narrative synthesis. H.H. checked for accuracy of the extraction
and validated the narrative. Differences were reconciled by consensus. Our 2017 search
informed the creation of low-fidelity paper prototypes of the LEMR user interface.
We designed all prototypes using Evolus Pencil, an open-source prototyping software
tool,[47] and patient data from the HIgh DENsity Intensive Care (HIDENIC) dataset, containing
fully de-identified, and HIPAA-compliant EMR data on University of Pittsburgh Medical
Center ICU patients.[2]
Observations, Interviews, and Revised Prototypes
Participants
ICU fellows and attending physicians were contacted via email or personal contacts,
and compensated using gift cards.
Observations
Author L.C. conducted observation sessions in multiple ICU settings in Pittsburgh,
PA (Presbyterian, Children's, and Mercy Hospitals) from June to November 2017, observing
ICU physicians using their EMRs (Cerner, in all cases) in preparation for morning
rounds, and dealing with issues in finding information and workflow interruptions.
Physician actions in patient rooms were not observed. Every observation was recorded
using field notes.
Interviews
Observations were followed with semistructured interviews outside of the care setting.
Participants were asked to explain interactions with the current EMR system, workflow
and mental processes used to assess patients, and any interesting behaviors noticed.
Additional questions addressed challenges encountered by the physicians while using
the EMR system. All interviews were audio-recorded.
Data Analysis and Prototype Revision
Field notes and audio recordings were transcribed and independently coded by L.C.
and H.H. using emergent coding.[46] L.C. coded all transcripts, developing a codebook with examples which HH used to
recode one of the interview transcripts. Inter-rater agreement was calculated as a
measure of coding consistency. Differences were reconciled by consensus to achieve
high reliability (Cohen's Kappa > 0.98).[48] Codes were organized using QSR NVivo 12.[49] Extracted data were used to create a narrative model of cognitive processes underlying
physicians' interactions with the EMR system, which informed the revision of our preliminary
prototypes.
Focus Group and Final Prototypes
Participants
L.C. and H.H. conducted a focus group with fellows and attending physicians from multiple
ICU settings in Pittsburgh, PA (Presbyterian, Children's, and Mercy Hospitals, VA
Health System). Participants were contacted via individual emails or personal contacts,
and compensated with gift cards.
Session
A 2-hour focus group session was conducted in the University of Pittsburgh Department
of Critical Care Medicine in April 2018, using techniques inspired by participatory
design[50]—a user-centered design approach that seeks to involve stakeholders in designing
systems[51]
[52] by combining brainstorming sessions with design activities.[53] After learning about our goal of designing a LEMR system and providing their feedback
on the concept, participant groups of two to three members each brainstormed ideas
about how the EMR display could highlight high-value information. Participants represented
their ideas in low-fidelity sketches using craft materials. Physicians were asked
to evaluate our prototypes and identify some preferred combinations of their ideas
and ours. The session was audio-recorded, and artifacts and field notes were collected
for later analysis.
Data Analysis and Final Prototypes
After the focus group, field notes, audio recordings, and discussions were transcribed,
grouped by participant and group number, and independently coded by L.C. and H.H.
using the techniques described in section 2.2.4. Any differences among raters were
reconciled by consensus to achieve high reliability (Cohen's Kappa > 0.98).[48] Coded data, photographs, artifacts, and audio recordings were used to develop a
narrative describing ICU physicians' feedback on the LEMR system and our prototypes,
issues with the current EMR, and suggested design ideas. Data analysis informed the
creation of a final series of paper prototypes, based on design themes and preferences
on which all subjects unanimously agreed. Participants were asked to review and validate
the narrative[54] and verify that the designs reflected their feedback.
Results
Design Themes Identified by Literature Review and Preliminary Prototypes
The design themes we identified in the literature as applicable to a LEMR system's
interface are described below, together with examples of our resulting prototypes
(the complete series is available in [Supplementary Appendix A2]; [Supplementary Figs. A2.1]–[A2.4] [available in the online version]).
Conveying Clinical Information Effectively
EMR interfaces should convey information effectively, by encoding health information
using visual attributes,[24] prioritizing the display of high-value data,[10]
[23]
[30]
[31] organizing information into clinical concepts,[24]
[30]
[32] and providing overviews of the patient's conditions.[2]
[23] Our preliminary prototypes ([Fig. 3] and [Supplementary Appendix A2]; [Supplementary Figs. A2.1]–[A2.4] [available in the online version]) explored four approaches to the display of high-value
information: highlighting information in place ([Supplementary Appendix A2]; [Supplementary Fig. A2.1] [available in the online version]), utilizing ephemeral highlighting to show initial
in-place highlights that quickly fade,[55] using a reference map pointing to health information predicted to be of interest
([Supplementary Appendix A2]; [Supplementary Fig. A2.2] [available in the online version]), and utilizing a highlighted data panel ([Supplementary Appendix A2]; [Supplementary Fig. A2.3] [available in the online version]). To optimize screen information density,[10]
[24]
[27]
[31] we also explored ways to summarize clinical information, including leveraging Midgaard's
semantic zoom technique[27] to visualize variables at levels of detail that vary with the zoom level ([Fig. 3] and [Supplementary Appendix A2]; [Supplementary Fig. A2.4] [available in the online version]). Finally, in seeking to improve physician decision-making[30] and reduce cognitive load,[2]
[56] we designed panels that provide overviews of the patient's conditions,[2]
[23] organizing information into clinical concepts[24]
[30]
[32] and using perceptual attributes (color and shape) to facilitate data visualization[24] ([Supplementary Appendix A2]; [Supplementary Fig. A2.2] [available in the online version]).
Fig. 3 One of four preliminary paper prototypes: use of Midgaard's semantic zoom[27] to summarize clinical information, displaying a greater number of parameters at
once. As displayed in the yellow box on the right (which was superimposed on the prototype
for illustrative purposes and did not represent an actual component of the Learning
Electronic Medical Record user interface), Midgaard's semantic zoom technique allows
to visualize variables at levels of detail that vary with the zoom level.
Highlighting Trends and Changes in Clinical Outcomes
Patients showing unexpected trends and changes are prioritized by physicians.[22] We explored data visualizations that encode information on changes and trends,[24] such as arrows and triangles that indicate the presence of uptrends and downtrends[22] ([Supplementary Appendix A2]; [Supplementary Fig. A2.3] [available in the online version]).
Supporting Analytical Reasoning
EMR interfaces should support analytical reasoning[57] by visually grouping related data[36] and by facilitating the manipulation of information at the level of entities and
their relationships.[21]
[58] We considered grouping related laboratory test and medication data on the screen
([Supplementary Appendix A2]; [Supplementary Fig. A2.3] [available in the online version]). We also explored how linked selection, whereby
selecting an entity in clinical notes could automatically highlight all instances
of that entity in the EMR, might improve information retrieval[21] ([Supplementary Appendix A2]; [Supplementary Fig. A2.3] [available in the online version]).
Observations, Interviews, and Revised Prototypes
Five observation and interview sessions were conducted from June to November 2017
([Table 1]). Insights from these interviews are presented below, together with one example
of the resulting revised prototypes ([Fig. 4]). Representative direct quotes from participants are available in [Supplementary Appendix A3] [available in the online version]).
Table 1
Observations and focus group participant characteristics
|
Participant characteristics
|
Observations (n = 5)
|
Focus group (n = 5)
|
|
Gender
|
|
|
|
Male
|
2
|
3
|
|
Female
|
3
|
2
|
|
Age
|
|
|
|
Minimum
|
31
|
30
|
|
Average
|
32
|
31
|
|
Maximum
|
33
|
32
|
|
Years in clinical practice
|
|
|
|
Minimum
|
4
|
4
|
|
Average
|
5.4
|
5.6
|
|
Maximum
|
7
|
8
|
|
ICU team role
|
|
|
|
Fellows
|
5
|
5
|
|
Specialty
|
|
|
|
Surgery and critical care medicine
|
2
|
1
|
|
Cardiology and critical care medicine
|
1
|
–
|
|
Pediatrics and critical care medicine
|
1
|
–
|
|
Emergency and critical care medicine
|
1
|
1
|
|
Pulmonary and critical care medicine
|
–
|
1
|
|
Critical care medicine
|
–
|
2
|
Abbreviation: ICU, intensive care unit.
Fig. 4 One of four revised prototypes, showing how the Learning Electronic Medical Record
interface might prioritize the display of (1) new information and (2) high-value patient data in dedicated panels that support analytical reasoning by
(3) grouping related data, (4) highlighting changes and (5) trends, (6) providing unobtrusive alerts, and (7) augmenting clinical notes with links to related data items. For each parameter,
the green color is used in the graphs to identify in-range values, while red and blue
indicate values above or below the normal range, respectively.
Patient Assessment and Prioritization Process
Consistently with findings from previous studies,[1]
[2]
[3] physicians appeared to categorize patients using a preliminary mental schema and
then to assess how well data fit with that schema. We observed that the presence of
unexpected values, changes, and trends impacted patient prioritization and led physicians
to gather more data.
Electronic Medical Record Data Usage
To assess patients in preparation for morning rounds, each physician visualized a
limited subset of EMR data in a specific personalized order, influenced by observed
trends and unexpected values. Some data were often visualized in the same sequence.
In some circumstances, the need to document findings in the notes or to compare information
while placing orders, forced users to repeatedly switch back and forth between EMR
screens ([Supplementary Appendix A4]; [Supplementary Fig. A4.1] [available in the online version]).
Electronic Medical Record System Limitations
Participants expressed a need to reduce the amount of non-relevant information displayed
on the EMR screens and a desire for a dashboard that concisely presents essential
information, offering on-demand access to more details. The lack of a proper search
feature makes it challenging to retrieve information in past notes. Visualizing frequently
used data, such as ventilation settings or exams to be ordered, can require extensive
scrolling or clicking. Integration with other information sources is inconsistent,
and physicians are not alerted when specific pieces of information become available.
The inability to tailor the system to individual needs also causes concern.
Workarounds
Workarounds, such as physician-created note templates and manual annotations on paper,
are utilized to overcome limitations and data access difficulties. Rounding reports,
index cards, and sign-out papers are used to annotate items that require discussion
or attention, and to quickly look up information during chart review.
Feedback on the Learning Electronic Medical Record Concept
All participants expressed at the same time enthusiasm for the LEMR concept and prospected
capabilities, and concern that users might overlook essential data if overrelying
on system recommendations.
Focus Group and Final Prototypes
Five physicians from four ICU settings in Pittsburgh, PA participated in our focus
group ([Table 1]). Two of them had previously participated in our observational study. Two senior
physicians, including coauthor G.C., also attended part of the session. In the 2-hour
session, physicians provided feedback on LEMR and our revised prototypes ([Supplementary Appendix A5]; [Supplementary Figs. A5.1]–[A5.5] [available in the online version]) and worked in two subgroups to generate design
ideas, representing them using several artifacts ([Supplementary Appendix A6]; [Supplementary Figs. A6.1]–[A6.3] [available in the online version]). The subsequent discussion helped us reconcile
overlapping and conflicting design themes emerged across the various stages of the
study. Our final prototypes ([Fig. 4]; [Supplementary Appendix A7], [Supplementary Figs. A7.1]
[A7.2] [available in the online version]) and the findings that guided their creation ([Table 2]) are presented below. Representative direct quotes from participants are available
in [Supplementary Appendix A3] [available in the online version].
Table 2
The table summarizes: (1) the challenges posed by current electronic medical record
systems that emerged from our qualitative evidence synthesis and were reconfirmed
in the qualitative portion of the study, (2) the design principles identified in the
literature as applicable to electronic medical record systems whose validity was reconfirmed
by our study subjects, and (3) the novel design ideas emerged from our discussions
with the participants (in bold, the proposed design augmentations that can be considered
unique to LEMR-like systems)
|
Challenges posed by current EMR systems
|
Design principles applicable to EMR systems
|
Novel design ideas identified with study participants
|
|
|
In bold, design ideas unique to LEMR-like systems
|
|
• Insufficient cognitive support provided to physicians
• Limited capabilities for visualizing complex data that often complicate the identification
of trends and anomalies
• Fragmentation of information across multiple screens
• Large amounts of non-relevant information displayed on screen
• Lack of a proper search feature to retrieve information in past notes
• Extensive scrolling or clicking required to visualize frequently used data
• Inconsistent integration with other information sources
• Unavailability of alerts when specific pieces of information become available
• Inability to tailor the EMR system to individual needs
|
• Use of perceptual attributes (color and shape) to encode health information, including
data on changes and trends.
• Need to prioritize the display of high-value patient data, offering on-demand access
to more details
• Helpfulness of organizing information into clinical concepts
• Need to optimize screen information density
• Usefulness of panels that provide overviews of the patient's conditions, organizing
information into clinical concepts
• Need to support analytical reasoning by visually grouping related data
|
• Use of static screen components displaying data important for every patient, coupled
with dynamic components highlighting diagnosis-specific information
• Use of mouseovers or right-clicks on items to provide quick access to trend information
and customizable views pertinent to the patient's active problems
• Use of unobtrusive flags to highlight significant changes and trends in the data,
and new data items (diagnostics, cultures, and select relevant labs) as they become
available
• Use of a “patient context” box to display information useful to characterize the patient's
case
• Use of a to-do list integrated with physician workflow, offering suggestions for
frequent orders and the ability to manage orders/procedures and to track scheduled
medication administrations
• Use of design elements to provide physicians with the ability to give feedback on
the appropriateness of the system recommendations
|
Abbreviations: ICU, intensive care unit; LEMR, Learning Electronic Medical Record.
Feedback on the Learning Electronic Medical Record Concept
The LEMR concept received unanimous support. Physicians expressed concern, however,
that overreliance on the highlights might cause users to miss important information,
suggesting that data not predicted to be of high value should remain easily accessible.
Participants valued the ability to provide feedback on the appropriateness of the
highlights.
Screen Layout and Access to Additional Information
Two alternative EMR home screen layouts were proposed. In [Fig. 5], high-value information pertinent to the patient's specific diagnoses could be displayed
in a dedicated “highlighted data” box at the top of the screen (B). Alternatively
([Supplementary Appendix A7]; [Supplementary Fig. A7.1] [available in the online version]), static boxes displaying data important for every
patient could be coupled with dynamic screen components highlighting diagnosis-specific
information in place (e.g., lactate for a sepsis case). In both designs, boxes can
be expanded when clicked upon to display additional data and visualized side by side
to more easily compare information. Mouseovers or right-clicks on items provide quick
access to trend information and customizable views pertinent to the patient's active
problems ([Supplementary Appendix A7]; [Supplementary Fig. A7.3] [available in the online version]).
Fig. 5 One of two proposed electronic medical record home screen designs. Relevant data
pertinent to the patient's specific diagnoses, (F) newly available and (D) significantly changed information, and (C, E) unobtrusive alerts are displayed in a dedicated “highlighted data” box at the top
of the screen (B). Static boxes display (A, G, I, K, M–U) information important for every patient. Mouseovers or right-clicks on data items
provide access to additional information. Blue indicators identify newly available
data, while red indicators are used for alerts.
Display of Individual Electronic Medical Record Data Items
Reflecting participants' preferences, fishbones ([Fig. 5 N]) are used to display labs essential for every patient (e.g., basic metabolic panel,
complete blood count, coagulation, and liver diagram). Mouseovers on labs provide
access to trend information. In the graphs, expected normal ranges are shown by either
bands or dotted lines showing upper and lower bounds. Time ranges default to 24 hours
but can be expanded (H).
Highlighting Changes, Trends, and New Information
Color-coding out-of-range values was not seen as being helpful in an ICU. Physicians
are more interested in significant changes and trends over time, which could be highlighted
using unobtrusive flags ([Fig. 5 C, K]) to avoid information overload. Highlighting new data (diagnostics, cultures, and
select relevant labs) as they become available, using color-coding (L) or a dedicated
box (F) would also be helpful. Appropriately designed indicators for potential drug–drug
interactions (C, K) and missed medication administrations (E) were also discussed
as useful.
Display of Patient Context Information and Integration with Intensive Care Unit Workflow
Participants expressed the need for a “patient context” box ([Fig. 5A]) displaying information useful to characterize the patient's case: diagnoses, length
of stay, unit the patient was transferred from, history and physical examination,
presenting symptoms, assessment and plan, active problems, procedures, and links to
relevant notes and consults. Using a to-do list ([Fig. 5P]) integrated with ICU workflow, physicians can visualize and cancel pending orders/procedures
for the next 8 to 12 hours and add reminders. Scheduled medication administrations
(with the ability to track if they were administered) and suggestions for frequent
orders also appear on the to-do list.
Discussion
Advances Introduced by Our Work
The LEMR approach leverages EMR data access patterns (as instantiations of physicians'
expertise) to predict and highlight the most relevant information for each patient.
It is an approach to reducing information overload in the ICU that, to our knowledge,
has not been considered before.[2] Our consideration of the effect of EMR interface design elements on cognitive performance[17] also contributes to the novelty of our approach.
Limitations of Current Electronic Medical Record Systems
Findings from observations and focus group were consistent with prior studies indicating
that EMR systems do not offer adequate cognitive support to clinicians.[5]
[6]
[7]
[8] Information overload and challenging access to information are major concerns[17]: multiple participants cited the overwhelming number of entries displayed in the
medication ordering screens and the extensive scrolling/clicking required to access
ventilation settings and other frequently used data. Data fragmentation also affects
cognitive performance: by not allowing physicians to keep multiple windows simultaneously
open, current EMR systems make it difficult to evaluate the patient's condition in
its complexity.[17] To overcome these issues, physicians adopt workarounds such as custom note templates
and manual annotations of items requiring discussion or attention. Our findings are
consistent with prior studies, showing that workarounds are commonly associated with
EMR use[17]
[59]
[60]
[61] and introduce the risk of errors.[17]
[62]
Identification and Display of High-Value Data
Our results suggest that information of high value to ICU physicians ([Fig. 6]) is represented by a combination of (1) data important for every patient with (2)
diagnosis-related, patient-specific information, (3) significant changes and trends,
and (4) newly available data (diagnostics, cultures, and select relevant labs).
Fig. 6 A combination of information essential for every patient, diagnosis-specific patient
data, significant changes and trends, and select newly available information is predicted
by the LEMR system to be of high-value, and displayed in a prioritized way in the
LEMR interface by utilizing static and dynamic screen components. LEMR, Learning Electronic
Medical Record.
Despite the complexity of the cognitive approaches ICU physicians use for sensemaking,
the participants' artifacts expressed strikingly similar needs and solutions. Physicians
identified approaches to highlighting high-value information that could enhance their
cognitive performance: specifically, using dedicated panels and a combination of static
and dynamic screen components that allow them to compare information and to access
additional data via mouseovers ([Fig. 6]). Data presentations that encode changes and trends to visual attributes could enable
easy identification of such information, while color-coding and obtrusive alerts should
be minimized. To-do lists integrated with ICU workflow could offer cognitive support
in the data-intensive ICU environment.[63]
Feedback on the Learning Electronic Medical Record System
All participants expressed enthusiasm for the LEMR system, confirming a need well-documented
in the literature: EMR systems should provide better cognitive support to physicians.[21]
[36]
[39] Our design ideas were considered an acceptable approach to representing high-value
data effectively.
Participants expressed concern that system use might introduce a form of automation
bias, a “tendency to use automated cues as a heuristic replacement for active information
seeking and processing”[64]. Physicians could become too reliant on the system's recommendations to identify
information relevant to assess each patient, thus missing important data.[65] To address this concern, we are investigating how highlighting of predicted high-value
items may impact physician information-search and interpretation activities. Participants
suggested that non-highlighted data should always remain easily accessible.
Feedback on our designs suggested factors that physicians want to control when interacting
with EMR systems. Users expressed a desire for customizability and for mechanisms
to provide feedback on the appropriateness of the highlights. These desires could
be reflective of the fear many physicians have of losing the “human element” of medicine[52] – in this case, having an electronic system make decisions for them.
Implementation Path and Challenges
We believe that the design preferences and prototypes identified in this study can
usefully inform the implementation of a functional interface—initially as a standalone
dashboard, to be used by clinicians as an addition to their current EMR systems. Increasingly,
dashboards that provide access to high-value information in a visual, condensed format
have been introduced by emergency departments[66] and health care organizations in general as ways of improving care processes and
patient outcomes, with encouraging results.[67] We intend to build our dashboard using the Application Programming Interfaces relying
on Fast Healthcare Interoperability Resources standards[68] proposed by major health care actors,[69] such as Epic[70] and Cerner.[71]
Developing a LEMR system display will involve several challenges. Designs must balance
the potential benefits of highlighting high-value data items and supporting customization
with the potential costs of related loss of consistency, which improves system learnability
and facilitates locating information on screen.[72] Highlighting important and new information may seem appealing, but defining which
changes are significant presents additional complications: further study is needed
to compare general definitions based on the magnitude of a change (compared with overall
parameter variance) to alternative approaches, such as definitions that take specific
disease mechanisms into account. Careful consideration of information density principles
will also be important:[10] while condensed views that summarize patient data represent a potential solution
to the limited screen real estate available, denser views can easily lead to cognitive
overload.[52]
Study Limitations
A potential limitation of our study is its single-center design. Despite the diversity
of ICU settings within our sample, all participants worked at a single institution
and used the same EMR system to accomplish a single task (chart review). Moreover,
there is a chance we may not have captured the most representative feedback due to
our exclusive focus on physician trainees and to the small number of subjects interviewed
- even if, with only five participants in the observational study, the number of findings
quickly reached the point of diminishing returns. There is also a chance that the
preferences expressed by participants in a focus group setting might differ from those
that they might express in clinical use.
Conclusion
Based on the positive feedback received from potential users, we conclude there is
interest in pursuing the idea of a LEMR system. The findings of this study provide
preliminary evidence of the potential utility of using highlights of clinical data
predicted to be of high value as a potential means to deal with the problem of information
overload associated with modern EMR systems. Further studies will be necessary to
confirm the usefulness of our approach in a clinical setting. Future plans include
the identification of ways to measure and present to physicians the confidence of
the predictions generated by the system,[73] a usability evaluation with heuristics specifically designed for dashboard visualizations,[74] and laboratory studies of both the utility of the proposed designs on decision-making
and the possible impact of any automation bias.
Clinical Relevance Statement
Clinical Relevance Statement
By introducing novel ways to support physicians' cognitive abilities in using EMR
systems, our LEMR-based approach has the potential to enhance physician performance,
leading to better patient outcomes as a result of those performance improvements.[17]
Multiple Choice Questions
Multiple Choice Questions
-
What is the best argument in support of an EMR display that uses past information
access patterns to predict and highlight particularly helpful information?
-
Information accessed when reviewing previous patients is a completely reliable model
of information that will be needed on future patients.
-
High-quality models of past information access behavior provide accurate representations
of access patterns over a large range of prior encounters, thus providing a useful
representation of physicians' collective understanding of prior patients.
-
Information that was not accessed when reviewing prior similar patients is not likely
to be useful.
-
Given two similar patients, all physicians will access exactly the same data when
reviewing those patients.
Correct Answer: The correct answer is option b. As information access patterns vary across physicians,
no single model that does not highlight all information that was ever accessed by
any physician can be completely reliable (thus ruling out answer a). As information
access patterns tell us a great deal about what data physicians accessed, they do
not tell us anything about information that physicians fail to access, so answer c
cannot be correct. Finally, our small sample saw widely varying information access
strategies, suggesting that identical access patterns are unlikely, making b the best
answer.
-
When implementing an EMR system that predicts and highlights on screen the most relevant
information for each patient, which of the following steps should be taken to avoid
overreliance on the highlights?
-
Use of color-coding to highlight high-value information should be minimized.
-
Mechanisms to provide feedback on the appropriateness of the highlights should be
made available to physicians.
-
Patient data not predicted to be of high value should remain easily accessible.
-
Access to additional patient data should be provided via mouseovers or right clicks
on the highlights.
Correct Answer: The correct answer is option c. The physicians participating in our study expressed
concern that clinical use of an EMR system that predicts and highlights on screen
the most relevant information for each patient might introduce a form of automation
bias[64]: physicians could become too reliant on the system's recommendations, thus missing
important data available elsewhere in the EMR.[65] To address this concern, participants suggested that data not predicted to be of
high value should always remain easily accessible next to the highlights, making option
c the correct answer.
