User-Centered Clinical Display Design Issues for Inpatient Providers

Abstract

Background Suboptimal information display in electronic health records (EHRs) is a notorious pain point for users. Designing an effective display is difficult, due in part to the complex and varied nature of clinical practice.

Objective This article aims to understand the goals, constraints, frustrations, and mental models of inpatient medical providers when accessing EHR data, to better inform the display of clinical information.

Methods A multidisciplinary ethnographic study of inpatient medical providers.

Results Our participants' primary goal was usually to assemble a clinical picture around a given question, under the constraints of time pressure and incomplete information. To do so, they tend to use a mental model of multiple layers of abstraction when thinking of patients and disease; they prefer immediate pattern recognition strategies for answering clinical questions, with breadth-first or depth-first search strategies used subsequently if needed; and they are sensitive to data relevance, completeness, and reliability when reading a record.

Conclusion These results conflict with the ubiquitous display design practice of separating data by type (test results, medications, notes, etc.), a mismatch that is known to encumber efficient mental processing by increasing both navigation burden and memory demands on users. A popular and obvious solution is to select or filter the data to display exactly what is presumed to be relevant to the clinical question, but this solution is both brittle and mistrusted by users. A less brittle approach that is more aligned with our users' mental model could use abstraction to summarize details instead of filtering to hide data. An abstraction-based approach could allow clinicians to more easily assemble a clinical picture, to use immediate pattern recognition strategies, and to adjust the level of displayed detail to their particular needs. It could also help the user notice unanticipated patterns and to fluidly shift attention as understanding evolves.

Keywords

electronic health records - data display - visualization - abstraction - information overload - clinician burnout

Background and Significance

The art of organizing a medical record to facilitate clinical practice has been studied since at least 1968, when Larry Weed published his seminal observations on the topic.[1] Since then, many innovative clinical displays have been designed,[2] [3] [4] [5] [6] [7] but few have found their way into routine clinical practice.

Some of these displays, such as the LifeLines environment that places clinical events on a timeline,[8] have become quite successful. Powsner and Tufte[9] published a similarly famous example of a one-page inpatient clinical summary that has inspired downstream designs. But other designs turn out to be less popular among doctors for reasons that surprise the designers,[10] or they are perceived as not clinically useful despite incorporating insightful visualizations.[11]

Developing clinical displays that actually meet the needs of clinicians is difficult.[12] Doctors have been quite vocal about the unfriendly design of their electronic health records (EHRs),[13] [14] [15] with one writer summarizing the complaints as the feeling that their EHR “was written by the devil himself in the fires of hell.”[16] Much of this frustration stems from the arduousness of entering information into the record,[12] [17] such as when being forced to enter structured information instead of free text.[18] [19] [20] But although data entry is a major pain point, information retrieval is also notoriously tedious,[13] [21] to the extent that many clinicians rely on their own or their patient's memory rather than spend the time and effort to search the EHR.

The unfortunate reality is that clinical information needs are incompletely understood by the designers of clinical information tools. The gap is understandable; clinical practice is both socially complex and highly technical, which renders it difficult to comprehend without spending substantial time doing it oneself. And despite U.S. regulations requiring user-centered EHR design, many U.S. vendors use substandard design approaches, with some following the astounding practice of not using physicians at all in their usability testing.[22] Moreover, researchers routinely use evaluation contexts that only distantly resemble clinical practice.[5] [6] [11] [23] But given the enormous amounts of time and energy that clinicians spend interacting with their EHR,[24] it is crucial that EHR design support, rather than hinder, clinical practice.[25]

Objective

In an attempt to address this gap, we undertook an ethnographic study to understand the goals, constraints, frustrations, and mental models of medical providers when accessing EHR data, with a focus on hospital-based providers. Our objective was to identify themes and considerations that can inform the design of clinical information displays.

We focused on information display, rather than data entry. EHR data entry has many pain points,[12] [17] [18] [19] [20] but they are separate problems that merit their own study.

Of course, people in other roles also interact with the EHR. Nurses, pharmacists, administrators, billing specialists, and primary care providers routinely use the medical record, but with goals that differ from those of hospital-based providers. We focused on one specific group that shared common clinical goals of inpatient medicine, wherein a provider must understand a new patient with a potentially long history of many, complex medical problems, and must make timely decisions about acute interventions for a subset of those problems.

We distilled results from this study into design considerations for clinical information displays, framed in the context of what is already known about working in overwhelmingly data-rich environments. Of course, this goal is not new to the field[12]; similar work has recently been done, for example, focusing on dentistry[26] and critical care medicine,[27] and good evaluation heuristics exist.[28] Nevertheless, the design approach that would maximize the effectiveness of clinical data display is far from completely understood.

Methods

Setting

The setting for this work was Vanderbilt University Medical Center, an academic tertiary care institution. The research team included a PhD-trained ethnographer, an MD/PhD data scientist, a PhD design researcher, a practicing physician, a senior medical student/computer scientist, and an undergraduate intern. Team members had deep expertise in clinical medicine, health care ethnography, biomedical informatics, computer science, and product design.

Participants

Participants were recruited via email to department lists in the areas of anesthesia, critical care medicine, emergency medicine, and hospital medicine. Our sample included 13 participants (7 female), including 8 physicians and 5 physician extenders with clinical provider roles. Ten (including all physician extenders) practiced in the anesthesia department, with one hospitalist and two emergency physicians. Median time in current role was 7 (interquartile interval [3, 11]) years.

Interviews

We conducted in-depth interviews using a semistructured instrument. Initial interviews were conducted by our ethnographer, accompanied by varying members of the research team. Two physicians were interviewed by team members more than once for iterative refinement of ideas and themes. Interviews were recorded and transcribed.

Participants practiced in the fields of anesthesia, critical care medicine, emergency medicine, and hospital medicine. Interviews began with questions about the participant's educational background and experience with technology and EHRs in general. Interviews with physicians asked about strategies for acquiring and using information for the following activities: assessing patient status, making decisions, planning for procedures, and understanding large groups of patients at once. Interviews with nurse practitioners focused on gathering data from patients prior to planned surgeries, searching for additional data, documenting their findings, and preparing reports for physicians.

While the instrument was designed to investigate EHR user experiences, questions were not tethered to a particular EHR design. Participants answered questions with respect to any EHR of their experience (and most had worked with multiple designs and vendors), although the common experience was the custom-developed Vanderbilt EHR in use at the time. Some interviews were conducted after an institutional switch to Epic (Epic Systems, Verona, Wisconsin, United States).

Observations

Interviews provided the bulk of the data for this work. But we also conducted a small number of direct observations of the work of three nurse practitioners and one anesthesiologist, which informed the interviews and subsequent analysis. Pairs of researchers accompanied the participants as they interviewed patients, documented their interactions, and made sense of the data for decision-making. Researchers took handwritten notes during the observations and subsequently typed them up as complete field notes. Additionally, we collected other information artifacts such as blank forms and hand-drawn visualizations from subjects.

Data Analysis

Trained staff coded deidentified interview transcripts and field notes. Our ethnographer trained and supervised three other team members (data scientist, design researcher, and intern) in coding, using test transcripts and team discussions to ensure consistency. We used open coding in this analysis, documenting all themes that emerged, without the constraint of a theoretical framework, and without prior decisions on what themes were considered relevant.

Team-based collaborative interpretations of the coded data were a key aspect of the analytical process. In these sessions, the larger research team (including an additional data scientist, a senior medical student, and an additional physician) examined as a group the coded data and transcripts, refined and standardized code definitions, and discussed the meaning of the data from each of our disciplinary perspectives. The result was a fuller illumination of both the emic (insider) perspective of the provider being interviewed, and the etic (outsider) perspective required to translate the data into useful themes on the design of information display. Our ethnographer kept notes and images of the discussions and documented the synthesis of concepts and themes as they evolved.

Results

From a total of 732 coded excerpts, 7 primary and multiple secondary concepts emerged from the group analysis ([Table 1]). Details of the concepts as described by participants are given in this section, and representative statements for each are collected in [Table 2].

Table 1 Integrative concepts extracted from the interviews, with the number of times each appeared

Concept	Count
Abstraction What level of semantic detail do I want?	240
Shouldn't be too summarized	43
Shouldn't be too detailed	57
Shouldn't be too many	70
Roll up/Drill down	43
Assembling a Clinical Picture Do I have a complete understanding?	197
Completeness signals Am I seeing all of the relevant data?	91
Relevance Is this data pertinent to my clinical question?	222
Decreases with time	22
Don't show irrelevant things	49
Health indicators versus Disease indicators	11
Trajectories	77
Varies by Specialty/Task/Circumstances	80
Reliability signals Can I trust this data?	175
Data age	32
Conflicting details	14
Provenance	78
Redundancy	14
Time Pressure What do I have time to look at?	39
Type 1/Type 2 Processing How do I find what I need?	114
Search strategies	64
Type 1 processing	28
Type 2 processing	38

Note: Bold entries are parent concepts that also include counts from child concepts.

Table 2 Representative quotes for high-level topics (some are relevant to multiple categories)

Abstraction
[This visualization] helps you sort of solve the puzzle of what's going on with the patient. So, if you have something you can't really explain, you can kind of plot out some of the medication, see when they were on it, compare with it when their symptoms started. It is definitely easier to do that graphically than by looking at dates.
One of the things that's important to recognize is the fact that, at least the unit I work in is a 27-bed unit and I can't possibly know all 27 patients in excruciating detail. So, we have something that we call pathway, which essentially means that someone is following an expected post-operative course, isn't doing anything too untoward, and those are the patients that you're probably going to be spending less time on. Whereas, the patients who are out of the box, so to speak, are going to be the ones where you, particularly as the intensivist overseeing things, need to devote more of your time. So, it would be really important to visualize patients that are on the path, versus out of the box. I don't know that it's ever written down, but everyone on the care team who takes care of these patients knows what an expected post-operative course looks like and I think we're all speaking the same language when we say, “Hey this patient's out of the box. Here's what's going on.” You know, if I go talk to the nurse practitioner and they say, “No, the patient's progressing along a normal post-operative pathway.” You don't really need to have much more information exchanged beyond that.
Assembling a Clinical Picture
So, I had a patient who had the chief complaint of ankle pain. And it was a woman in her probably mid 60's with diabetes, heart failure, hypertension. I don't think she had any history of cancer, but she had a history of a current fracture from osteopenia and cardiac disease. She stood up, got woozy, slumped over and twisted her ankle. And so, she comes in for the ankle pain. Well, the real issue is that the increased steroids that she's on for her COPD probably worsened her diabetes and worsened her cardiovascular disease and probably contributed to the fracture.
It would be huge if you could overlay everything on the same visualization. If you take a diabetic patient, for example, what's their creatinine doing? How much protein are they spilling into their urine? What's their blood pressure doing? If there was a way to just get all that information on one screen. And then for the folks who are supposed to have intense treatment, like to achieve strict blood sugar control for diabetics or hypertension control, if you could visualize that we tried these medications at these doses, and had to stop them because of x, y, or z, and how close did that get to control.
Completeness signals
Of course even if I have a full cardiac history, I still ask them about it. Because there can be changes or something could have been missed or there could have been a miscommunication. Usually everything is complete, but you always just want to verify what's going on.
And so I just look at these clues and if I can put a story together that makes sense, then I can say that, that picture sort of holds, but you never know. You have to always be like there's probably something missing, that we're not seeing, or that is not there, and so I'll just keep my eyes open as I go forward.
Relevance
The relevance of different things degrades at different rates. If they had breast cancer 15 years ago, recurrence is still a consideration. You know, is this their recurrence? Is their altered mental status related to the fact that they have a pulmonary embolism because they had cancer and I just can't find it yet? Gastrointestinal bleeds tend to come back as well. Cardiac disease, if you had a [heart attack] 20 years ago, yep, that's still important. . . . I think the thing that degrades the most rapidly is vital signs and medication lists in terms of value. I don't really care what they were on a year ago. Although if I can get a sense of trajectory, past medications are important. And a visualization of when they change. I tried to do this actually for my son. We were changing medications on him and I was like “What was the freakin' timeline?” That is really hard to do. But that would be super helpful. You know, like stable on all these meds, and then bump, bump, bump, bump [increasing the dose]. You know that would be helpful. But our EHR can't do that, so I'm not even going to try.
What different people will consider important does differ… So, cardiology will put in the patient summary where each of the [bypass] grafts went. I'm like I don't care, CABGx5. [where CABGx5 is an abstraction meaning “Coronary Artery Bypass Graft using 5 bypass vessels”]
Reliability signals
You can't always trust everything you see and the big thing that you classically see that you can't trust is [fluid] inputs and outputs. So I would personally almost never look at inputs and outputs. I would always look at weights and I would just use the weight because that's the only thing that's truly reliable. Except in the ICU, in the ICU you can trust the inputs and outputs. When you have that Foley that has like an electronic meter that tells you exactly what the output is. That's different.
Say, for example, a seemingly simple question. Have any medications changed recently? Trying to sort that out in the record is extraordinarily difficult. What we'll do as providers, we're so sneaky that we know this medication is super expensive so we'll say, okay here's what I'm going to do. I want you to take 5 mg twice a day. I'm going to give you 20 mg tablets. Cut that into quarters. So what are they actually taking? I don't know.
Time Pressure
I don't get a lot of opportunities to ask about medication history because it takes too long. A good med rec is ∼45 minutes to an hour.
GI is so high throughput and the attendings in my opinion have a great difficulty with it. [Name redacted], for example, went down to GI the other day for her first time like in a couple months. … She did not start charting until 3:00 pm, because she's probably going through [the EHR] like an ICU intensivist would. You have to switch your mode. You have to realize this case needs to get done, there's going to be 20 more that are going to follow it. What minimal information do I need to care for this patient in the most safe and efficient manner?
Type 1/Type 2 Processing
I remember looking at [a display like [Fig. 1]], there was a patient who had pediatric congenital heart disease. Very easy to pick up. Just by looking at it.
So when I'm using the EHR, I'm usually searching. I'm trying to learn the answer to a question
So, really as a hospitalist, you end up using everything throughout the course of the hospitalization. Now what you use on hospital day one when you're admitting the patient is different than what you're using later on, but at some point, you're probably pretty much everywhere in the chart. Like when you admit the patient, for me, the first thing I do is I look to see the ER note is there and then it's vitals and labs, imaging and depending on the patient, often going to see like what the last chart discharge summary is and I'll usually look at all those things, even before I call the ED back. … Then once I agree that the patient should be admitted, it's a second sniff at the chart. That discharge summary gets a little bit closer of a look. The lab trends get a little bit closer of a look to say well has he been here for this before? Has this happened before? And then I do the med rec at the bedside.... And then once you have admitted the patient, once you've done that med rec and kind of gone through everything, then as you get further into hospitalization, that's when you start really digging in a little bit deeper. Old case manager … social worker notes, old consult notes. So, it's like different layers of depth each time you kind of go in.

Fig. 1 A possible top-level abstraction. This abstraction presents the density of data in a patient record, organized by organ system/International Classification of Diseases, 9th Revision (ICD-9) chapter in small multiples. Vertical axis on all graphs is data events per unit time. Horizontal axis is calendar date, spanning the life of the record. This patient's main problem is sickle-cell anemia, visible in the Hematologic section, with related effects evident in the Cardiovascular, Pulmonary, and Genitourinary sections. The gradually increasing intensity of disease and acute exacerbations are clearly visible.

Assembling a Clinical Picture under Uncertainty and Time Pressure

Assembling a complete clinical picture relevant to a specific decision was nearly always the immediate goal when clinicians searched the medical record, and time pressure was frequently cited as a primary factor that made this difficult.

We observed scenarios of assembling a clinical picture in several contexts, including an emergency physician considering whether the patient's stated complaint was really an unrecognized consequence of her existing conditions; a hospitalist understanding what was done in the emergency department before assessing the patient for potential admission; a physician understanding the history of response to a medication before adjusting the dose; a physician getting a picture of all current medications when considering a potential adverse interaction; or a physician understanding the history of a chronic illness when evaluating the cause of an acute exacerbation.

Relevance

Assembling a clinical picture usually means sorting through a large amount of irrelevant information to find the relevant pieces. The desire to minimize the time spent sorting through irrelevant information was frequently expressed, as was the desire for a display that would give only the information relevant to a particular question.

The perceived relevance of information was affected by its age, with older information considered less relevant. Nevertheless, the history of changes of a variable over time (coded as “trajectory”) remains relevant even when its earliest elements are old. Indicators of health (such as normal laboratory values, or the absence of a medication) could be as relevant as indicators of disease.

Completeness and Reliability

Participants described the unknown completeness and unknown reliability of the assembled picture as being at the heart of clinical decision uncertainty. Seeing agreement (coded as “redundancy”) from various sources—the medications the patient is taking, the laboratory test results, the visit history, and the notes from other physicians—can combine to give confidence in the full picture, even if the individual elements are unreliable. Conversely, conflicting details reduce confidence.

Different levels of reliability were attributed to different data sources: laboratory tests and other studies were very reliable, and billing codes were rather suspect. The age of information was perceived to affect its reliability. Other physicians' notes were generally considered reliable, but were also to a degree suspect, because they are subject to the goals, perspective, and bias of the writer. Reliability also increases with the experience of the writer: medical student < intern < resident < attending. Data provenance is therefore a crucial signal for assessing the reliability of individual elements.

Type 1 and Type 2 Processing

Cognitive processes for decision making can be divided into two types: type 1, which is fast and intuitive (but more error prone), and type 2, which is slow and analytic.[29] [30] Type 1 processes use immediate pattern recognition when looking at a whole picture, whereas type 2 processes involve two stages of searching through information: a breadth-first search to establish the range of possibilities, followed by a depth-first search to evaluate a specific choice. Participants described a preference for quick type 1 recognition because of the time savings, but they used type 2 searching when needed. Under time pressure, the search was often abbreviated into what was called a chart biopsy, where a high-value section of the chart (such as the last discharge summary, or the clinician's own last note) was quickly consulted, rather than doing a more complete search.

Abstraction

Participants described thinking at different levels of semantic detail for different tasks, and desiring the level of detail in the display to match the level of their thinking for the task. We use the computer science term abstraction to describe this type of thinking.

Higher-level abstractions are more summarized, and lower-level abstractions are more detailed. The desire not to wade through the details to see the big picture was frequently expressed, as was the desire to see all relevant information on the same page. High-level abstractions were more needed when first meeting the patient or first assessing a situation. Some tasks, such as admitting the patient to the hospital, require several passes through the chart at increasing levels of detail at different stages in the process.

Many of the participant comments on irrelevant clutter turned out actually to be requests to hide unnecessary detail for a particular piece of information—they wanted the essence of the information, but not necessarily the details. For example, an emergency physician would rather see the abstraction “CABGx5” in the patient record than read a detailed description of where all 5 of the coronary artery bypass grafts were located. By contrast, a cardiologist would often want to see the details, rather than the abstraction.

A graphical plot of repeated laboratory test results over time is another good example of what it means to abstract away detail in a data summary. The plot gives the essence of that analyte's behavior over time, and an indication of approximately when and how often it was measured, but the exact values, dates, times, and other metadata are omitted. The plot not only takes up less space than the raw data, but it more effectively allows for type 1 recognition. It also allows an assessment of completeness (results that change dramatically between measurements are probably not giving the full picture), and it supports quickly assembling a clinical picture.

Trajectories over time were the most common abstraction described by participants. Some mentioned a desire to compare trajectories for several variables at once to establish relationships or causality. One participant described thinking of a patient's course as following a path through the space of medical possibilities, and facing the task of assessing whether the patient was still on the typical path of a given disease or whether they had left that path and become “out of the box.”

Interestingly, the abstraction level of written notes increases with clinical experience (less experienced clinicians tend to provide more detail) suggesting that part of medical training is learning the important clinical abstractions.

Discussion

The overall goal of participants emerged as reducing cognitive uncertainty and its risk of medical error. To achieve this, they attempted to assemble a clinical picture of the patient by searching for data relevant to the question at hand, and assessing the picture's completeness and reliability as the data were discovered. Participants preferred using type 1 processing (recognition), rather than type 2 processing (searching) to assemble the picture. The conflict between completeness and reliability, and their interaction with time pressure is what makes the task challenging, and a role for data abstraction to mitigate this conflict emerged as a nearly universal theme. This section discusses these design themes and considerations in light of other research on working in overwhelmingly data-rich environments.

While we identified the themes with a de novo ethnographic study without previously defined targets, we do not claim that any individual finding is new to the field; our main contribution is the data-driven identification of a set of design considerations that would support the clinical thinking of our participants, but which were absent from the EHRs of their experience.

Taken as a whole, our results reveal clinicians wanting to view patient information at a level of abstraction that matches their clinical question, with unnecessary details summarized away. But this must be done in a way that does not hide the existence of other data elements, because they help to evaluate the completeness and reliability of the clinical picture, and they can fluidly redirect attention when unexpected information is present.[31] Visible but summarized data provides confidence that the clinician has found all of the relevant information, including pieces in unanticipated places.

Assembling a Clinical Picture under Uncertainty and Time Pressure

To efficiently assemble a clinical picture and enable type 1 recognition, all relevant data must be collected into the same view. The fact that this is even an identifiable task is itself an indictment of the default practice of separating data by mode or source (such as laboratory values on one page, medications on another, textual notes somewhere else). This separation is a vestige of paper medical records, and while keeping that organization may have been useful in making the transition to electronic records, it actually causes several problems for the end user and contributes to clinician burn-out.[13]

First, the mode-separated design makes type 1 recognition nearly impossible, and it imposes a navigation burden that one participant called “death by clicking.”

Second, switching between screens to assemble the picture increases the demand on memory and, when memory fails, it causes users to repeatedly look back to previous screens to remind themselves what they have already seen.[31] [32] [33]

Third, forcing the user to decide what to look at next in sequence increases the cognitive burden of assembling the picture.[34] [35] This burden is particularly heavy for less-experienced trainees.

Finally, the necessity of guessing where to find what you want, and not knowing whether it even exists, adds further cognitive burden and uncertainty.[35] This was described by one participant as clearing a minefield. She needs to find all of the mines, or all of the pieces of information that inform the decision; they could be buried anywhere, and if she misses one, it could lead to tragedy. But she has only a few minutes to look through the record and make decisions in real time, and a malpractice lawyer will have months to comb through it in detail, with the advantage of hindsight.

The work of assembling a clinical picture has been previously identified as central to the practice of medicine,[4] [11] [35] [36] [37] although it is commonly described only for getting a full, overall picture. Tools that do a reasonable job of providing the full overview may still fail in practice because more specific details, needed to answer more specific questions, might not be contained in that overview.[10]

Assembling a clinical picture is analogous to Klein's notion of sensemaking, or “a motivated, continuous effort to understand connections (which can be among people, places, and events) in order to anticipate their trajectories and act effectively,”[38] although in this case with the added constraint of time pressure. It also resonates with the data frame theory of sensemaking, in which “elements are explained when they are fitted into a structure that links them to other elements.”[39] These data frames sound like the mental abstractions clinicians use when assembling the data.

Our results reflect the fact that the clinical picture sought, and the level of detail needed in the data frame/abstraction, varies among provider types and clinical situations. An emergency physician reviewing information on a febrile pediatric cancer patient may do very little digging for information before making a decision to admit the patient to the hospital (in sensemaking terms, a concern about a cancer-related complication is an easily accessible data frame). On the other hand, the same physician reviewing a long social and clinical history on an elderly patient with a gait disturbance may need to make many more connections between variables related to the patient, their diagnosis, therapeutic regimen, health risks, and environment.

Relevance versus Completeness

There is an extensive literature of examples attempting to do what our participants wanted—place all data relevant to a given task on the same page, and hide everything else.[5] [10] [23] [40] [41] [42] [43] [44] This approach can work well when the visualization is accurately matched to a real clinical task.[42] [44] When tested in a simulated environment using the exact task for which they were designed, data-hiding visualizations tend to perform well. But when subsequently used in practice, where the task is subject to clinical variability, a data-hiding visualization can become a keyhole view that not only reduces completeness, but also hides that reduction from the user.[5] [31] [34] And because even clinicians of the same specialty performing the same task on the same patient record disagree about what data are relevant,[45] [46] [47] the keyhole effect is inevitable. Data-hiding solutions are therefore quite brittle,[34] and perhaps from experience, users mistrust them.[10] [23]

Abstraction

The skillful use of data abstraction could potentially solve the relevance versus completeness tradeoff by allowing the important patterns to emerge from the data field without hiding the existence of other patterns.[34] Instead of hiding data, an abstraction approach only hides details.

The key to a good abstraction is to hide the variations in detail that are irrelevant to the task at hand. Edsger Dijkstra,[48] one of the founding fathers of computer science, described it this way: “We all know that the only mental tool by means of which a very finite piece of reasoning can cover a myriad cases is called “abstraction”; as a result the effective exploitation of his powers of abstraction must be regarded as one of the most vital activities of a competent programmer. In this connection it might be worth-while to point out that the purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise.”

Organizing information around multiple levels of abstraction is a hallmark of skillful computational thinking, and the key to controlling the complexity of software.[49] [50] [51] Each new level hides some of the details of the level below it, bringing the conceptual essence of the information into sharper focus. Clinicians already produce artifacts with this organization. The standard inpatient progress note incorporates three different levels routinely: a one-sentence summary at the top of the note, giving the high-level essence of the patient's problem, followed by several low-level paragraphs giving details of history, physical exam, laboratory results, etc., and ended by a paragraph called Assessment and Plan or Impression and Recommendations, in which a mid-level abstraction of those details that is sufficient for clinical decision making is given. Each of these three sections is used for a different purpose by a future clinician searching the chart.

Illustrative Example

To illustrate how an abstraction-based design might address issues raised by our participants, we walk through a simple example ([Fig. 1]). It is an illustrative example that received positive reactions from our participants, but has not been rigorously tested.

The figure is a top-level abstraction that presents the density over time of all data in a real record, separated by organ system (in this case, using the International Classification of Diseases, 9th Revision chapter names). All of the data elements for a single patient are represented in a single view—instances of billing codes, laboratory test results, medication exposures, and clinical notes all contribute to the data density, potentially to multiple organ systems simultaneously if appropriate. Data density reflects attention to disease, suggesting disease volatility and the need for intervention or monitoring.

The trajectory of the patient's history and current status are clearly visible despite the absence of nearly all of the data details. The single view allows for a provider to assemble a top-level clinical picture of the patient, to use type 1 recognition to understand the essence of the patient's history and current state, and to begin assessing the completeness of the data for answering a given question. Importantly, it allows the provider to identify where additional relevant details may be located, including unexpected data[31] (such as, perhaps, the oncology activity around 2013).

We immediately see that the patient has about 10 years of data. The patient's major problem is hematologic, with several acute exacerbations, and severe deterioration occurring in the last 2 years. We see meaningful but less severe involvement of cardiovascular and renal systems. There is a continuous but low-intensity history of infections, endocrine problems, pulmonary, and musculoskeletal problems, with a hint of psychiatric and neurologic problems during the deterioration in the last 2 years. Moreover, we can determine with some confidence the absence of problems in several other organ systems. The only domain for which there is no data at all is perinatal.

To see more details without losing sight of this overview, we could imagine clicking on an organ system to expand (and again to contract) a space within the overview to show a similar display of the specific problems of that system (not shown in the figure). That display would represent the next level of abstraction, where we would see that the major hematologic condition is sickle-cell anemia, that the cardiovascular problems are dominated by primary pulmonary hypertension and congestive heart failure, that the endocrine problem is hemochromatosis, and that the renal issues are chronic renal failure, with end-stage renal disease in the last 2 years. This second-layer abstraction could also be the place where medications, laboratory tests, and notes are separated out into their own subgraphics. Descent into a third level could give all available detail on a particular single problem.

Organizing the display by abstraction levels makes it easier to accommodate the differences between users in what is considered relevant or necessary detail. The fact that all data elements are represented in some form on the screen most of the time means that they are rarely completely hidden, and users can make more specific and informed decisions about where to zoom in to show desired details.

Under a design like this, zooming into details does at some point need to move other information off the screen, or at least out of focus. But as long as the mechanism to zoom in and out is intuitive and natural, the cost of exposing those details as needed can be low compared with the cost of screen switching and guessing where to find hidden data elements.

Limitations

Our study was limited to hospital-based providers in a single tertiary care academic system, although many of our participants had practiced in other locations and contexts, and their comments drew on their full history as clinicians. The study was deliberately limited to this narrow population of providers, and our conclusions may not extend to other types of users.

An interesting special case is outpatient providers, who would likely benefit from displays organized around the themes presented here, because they face similar issues when accessing a medical record.[52] [53] However, they also have a subset of clinical tasks that are both frequent and predictable, such as routine follow-up visits for patients with chronic disease. In these cases, an information-hiding approach may actually be successful, especially if each provider could adapt the display to their own particular practice patterns.

Conclusion

The main goal of our participants (inpatient medical providers) when seeking information in an EHR was to assemble a clinical picture for a given patient around a specific clinical question, usually under the constraints of time pressure and incomplete information. In their experience, EHR designs were generally frustrating because they made it difficult to assemble the information relevant to the question into a single view, and difficult to have confidence that the gathered information was complete. The common design of separating displays by data mode frustrated their efforts to use type 1 cognitive processing (immediate pattern recognition), and forced them to resort to type 2 processing (search). They described a mental model of patients and disease processes that involved multiple layers of abstraction, in which higher layers represented information in a more summarized form, and they reached into deeper layers only to learn details when necessary. Display designs that more closely match this mental model may help overcome some expressed frustrations and may better meet the needs of clinical practice.

Clinical Relevance Statement

When accessing a patient's electronic health record (EHR), the usual clinical goal is to assemble a complete and reliable clinical picture around a specific question about the patient, under conditions of uncertainty and time pressure. Current EHR designs display clinical information organized by data mode (laboratory result, medication use, clinical notes, etc.), which causes unnecessary navigation, memory, and cognition burdens. An alternate design, organized in multiple levels of abstraction, would allow for easier pattern recognition, higher confidence in the completeness of the result, and easier shifting of attention as new information is learned.

Multiple Choice Questions

1. What would be the appropriate level of detail to provide to a physician when presenting information regarding a coronary artery bypass procedure?

• The number of vessels bypassed.

• The year in which the procedure occurred.

• The specific detail regarding which vessels were bypassed.

• It depends on the type of physician.

Correct Answer The correct answer is option d. It depends on the type of physician. Clinicians from different specialties may require different levels of detail depending on the clinical question they are investigating.

2. The benefits of data displays based on multiple layers of abstraction, in comparison to displays organized by data modes or displays that hide all nonrelevant data include which of the following:

• Abstraction-based displays are easier to design and implement because they don't require clinical knowledge.

• Abstraction-based displays make it easier to notice unexpected information.

• Abstraction-based displays are more efficient because they are specifically targeted at a single clinical question.

• Abstraction-based displays are more usable because they are so common.

Correct Answer: The correct answer is option b. One of the main benefits of abstraction-based displays is that they allow the user to notice relevant data that they hadn't anticipated, and to shift attention as needed. They tend to be more difficult to design, because they must take into account the natural abstractions present in clinical thinking. Abstraction-based displays are not specifically targeted at a single clinical question, but instead to the general levels of detail present in clinical thinking. Data-hiding displays are much more common than abstraction-based displays in the literature, and mode-based displays are much more common in practice.

Conflict of Interest

None declared.

Acknowledgments

We thank the study participants for their time and insight. We are grateful for the assistance of our summer intern Binula Illukpitiya.

Protection of Human and Animal Subjects

The research project and protocol were approved by the Vanderbilt Human Research Protections Program.

• References

• 1 Weed LL. Medical records that guide and teach. N Engl J Med 1968; 278 (11) 593-600
• 2 West VL, Borland D, Hammond WE. Innovative information visualization of electronic health record data: a systematic review. J Am Med Inform Assoc 2015; 22 (02) 330-339
• 3 Wanderer JP, Nelson SE, Ehrenfeld JM, Monahan S, Park S. Clinical data visualization: the current state and future needs. J Med Syst 2016; 40 (12) 275
• 4 Feblowitz JC, Wright A, Singh H, Samal L, Sittig DF. Summarization of clinical information: a conceptual model. J Biomed Inform 2011; 44 (04) 688-699
• 5 Waller RG, Wright MC, Segall N. et al. Novel displays of patient information in critical care settings: a systematic review. J Am Med Inform Assoc 2019; 26 (05) 479-489
• 6 Rind A, Wang TD, Aigner W. et al. Interactive information visualization to explore and query electronic health records. Found Trends Hum-Comput Interact 2013; 5 (03) 207-298
• 7 Wright MC, Borbolla D, Waller RG. et al. Critical care information display approaches and design frameworks: a systematic review and meta-analysis. J Biomed Inform X 2019; 3: 3
• 8 Plaisant C, Mushlin R, Snyder A, Li J, Heller D, Shneiderman B. LifeLines: using visualization to enhance navigation and analysis of patient records. Proc AMIA Symp 1998; 76-80
• 9 Powsner SM, Tufte ER. Graphical summary of patient status. Lancet 1994; 344 (8919): 386-389
• 10 Jensen LG, Bossen C. Factors affecting physicians' use of a dedicated overview interface in an electronic health record: the importance of standard information and standard documentation. Int J Med Inform 2016; 87: 44-53
• 11 Sultanum N, Brudno M, Wigdor D, Chevalier F. More Text Please! Understanding and Supporting the Use of Visualization for Clinical Text Overview. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. CHI '18. New York, NY, USA: ACM; 2018: 422:1-422:13
• 12 Zahabi M, Kaber DB, Swangnetr M. Usability and safety in electronic medical records interface design: a review of recent literature and guideline formulation. Hum Factors 2015; 57 (05) 805-834
• 13 Kroth PJ, Morioka-Douglas N, Veres S. et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2 (08) e199609
• 14 Shanafelt TD, Dyrbye LN, Sinsky C. et al. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc 2016; 91 (07) 836-848
• 15 Kunzman K. Why are EMRs So Terrible?. 2018 . MD Magazine 2018, Feb 19. Accessed September 7, 2018 at: https://www.mdmag.com/medical-news/why-are-emrs-so-terrible
• 16 Drummond D. Eight ways to lower practice stress and get home sooner. Fam Pract Manag 2015; 22 (06) 13-18
• 17 Poissant L, Pereira J, Tamblyn R, Kawasumi Y. PThe impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc 2005; 12 (05) 505-516
• 18 Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004; 11 (02) 104-112
• 19 Horsky J, Zhang J, Patel VL. To err is not entirely human: complex technology and user cognition. J Biomed Inform 2005; 38 (04) 264-266
• 20 Koppel R, Metlay JP, Cohen A. et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; 293 (10) 1197-1203
• 21 Christensen T, Grimsmo A. Instant availability of patient records, but diminished availability of patient information: a multi-method study of GP's use of electronic patient records. BMC Med Inform Decis Mak 2008; 8: 12
• 22 Ratwani RM, Benda NC, Hettinger AZ, Fairbanks RJ. Electronic health record vendor adherence to usability certification requirements and testing standards. JAMA 2015; 314 (10) 1070-1071
• 23 Hsu W, Taira RK, El-Saden S, Kangarloo H, Bui AAT. Context-based electronic health record: toward patient specific healthcare. IEEE Trans Inf Technol Biomed 2012; 16 (02) 228-234
• 24 Arndt BG, Beasley JW, Watkinson MD. et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med 2017; 15 (05) 419-426
• 25 Payne TH, Corley S, Cullen TA. et al. Report of the AMIA EHR-2020 Task Force on the status and future direction of EHRs. J Am Med Inform Assoc 2015; 22 (05) 1102-1110
• 26 Thyvalikakath TP, Dziabiak MP, Johnson R. et al. Advancing cognitive engineering methods to support user interface design for electronic health records. Int J Med Inform 2014; 83 (04) 292-302
• 27 Wright MC, Dunbar S, Macpherson BC. et al. Toward designing information display to support critical care. A qualitative contextual evaluation and visioning effort. Appl Clin Inform 2016; 7 (04) 912-929
• 28 Dowding D, Merrill JA. The development of heuristics for evaluation of dashboard visualizations. Appl Clin Inform 2018; 9 (03) 511-518
• 29 Croskerry P. From mindless to mindful practice--cognitive bias and clinical decision making. N Engl J Med 2013; 368 (26) 2445-2448
• 30 Saposnik G, Redelmeier D, Ruff CC, Tobler PN. Cognitive biases associated with medical decisions: a systematic review. BMC Med Inform Decis Mak 2016; 16 (01) 138
• 31 Woods DD, Watts JC. How not to have to navigate through too many displays. In: Helander M, Landauer TK, Prabhu P. , eds. Handbook of Human-Computer Interaction. Amsterdam: Elsevier; 1997: 617-650
• 32 Senathirajah Y, Kaufman D, Bakken S. User-composable electronic health record improves efficiency of clinician data viewing for patient case appraisal: a mixed-methods study. EGEMS (Wash DC) 2016; 4 (01) 1176
• 33 Nolan ME, Siwani R, Helmi H, Pickering BW, Moreno-Franco P, Herasevich V. Health IT usability focus section: data use and navigation patterns among medical ICU clinicians during electronic chart review. Appl Clin Inform 2017; 8 (04) 1117-1126
• 34 Woods DD, Patterson ES, Roth EM. Can we ever escape from data overload? A cognitive systems diagnosis. Cogn Technol Work 2002; 4 (01) 22-36
• 35 Nolan ME, Cartin-Ceba R, Moreno-Franco P, Pickering B, Herasevich V. A multisite survey study of EMR review habits, information needs, and display preferences among medical ICU clinicians evaluating new patients. Appl Clin Inform 2017; 8 (04) 1197-1207
• 36 Hornbæk K, Hertzum M. The notion of overview in information visualization. Int J Hum Comput Stud 2011; 69 (07) 509-525
• 37 Bossen C, Jensen LG. How Physicians 'Achieve Overview': A Case-based Study in a Hospital Ward. In: Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14). New York, NY, USA: ACM; 2014: 257-268
• 38 Klein G, Moon B, Hoffman RR. Making sense of sensemaking 1: alternative perspectives. IEEE Intell Syst 2006; 21 (04) 70-73
• 39 Klein G, Phillips JK, Rall EL, Peluso DA. . A Data-Frame Theory of Sensemaking. In: Expertise Out of Context: Proceedings of the Sixth International Conference on Naturalistic Decision Making. New York, NY: 2007: 113-155
• 40 Khairat SS, Dukkipati A, Lauria HA, Bice T, Travers D, Carson SS. The impact of visualization dashboards on quality of care and clinician satisfaction: integrative literature review. JMIR Human Factors 2018; 5 (02) e22
• 41 Pickering BW, Herasevich V, Ahmed A, Gajic O. Novel representation of clinical information in the ICU: developing user interfaces which reduce information overload. Appl Clin Inform 2010; 1 (02) 116-131
• 42 Ahmed A, Chandra S, Herasevich V, Gajic O, Pickering BW. The effect of two different electronic health record user interfaces on intensive care provider task load, errors of cognition, and performance. Crit Care Med 2011; 39 (07) 1626-1634
• 43 Pickering BW, Dong Y, Ahmed A. et al. The implementation of clinician designed, human-centered electronic medical record viewer in the intensive care unit: a pilot step-wedge cluster randomized trial. Int J Med Inform 2015; 84 (05) 299-307
• 44 Thomas SM, Overhage JM, Warvel J, McDonald CJ. . A comparison of a printed patient summary document with its electronic equivalent: early results. Proceedings AMIA Symposium; 2001: 701-705
• 45 Herasevich V, Ellsworth MA, Hebl JR, Brown MJ, Pickering BW. Information needs for the OR and PACU electronic medical record. Appl Clin Inform 2014; 5 (03) 630-641
• 46 Zeng Q, Cimino JJ, Zou KH. Providing concept-oriented views for clinical data using a knowledge-based system: an evaluation. J Am Med Inform Assoc 2002; 9 (03) 294-305
• 47 Van Vleck TT, Stein DM, Stetson PD, Johnson SB. . Assessing data relevance for automated generation of a clinical summary. AMIA Annual Symposium proceedings AMIA Symposium; 2007: 761-765
• 48 Dijkstra EW. The humble programmer. Commun ACM 1972; 15: 859-866
• 49 Kramer J. Is abstraction the key to computing?. Commun ACM 2007; 50 (04) 36-42
• 50 Verhoeff T. On abstraction and informatics. In: Bezáková D. In: Kalas I. , eds. Informatics in Schools: Situation, Evolution and Perspectives. 5th International Conference. 2011
• 51 Wing JM. Computational thinking. Commun ACM 2006; 49 (03) 33-35
• 52 Beasley JW, Wetterneck TB, Temte J. et al. Information chaos in primary care: implications for physician performance and patient safety. J Am Board Fam Med 2011; 24 (06) 745-751
• 53 Torsvik T, Lillebo B, Hertzum M. How do experienced physicians access and evaluate laboratory test results for the chronic patient? A qualitative analysis. Appl Clin Inform 2018; 9 (02) 403-410

Publication History

Received: 08 June 2020

Accepted: 12 August 2020

Article published online:
21 October 2020