Keywords
electronic health records - interfaces - usability - graphical user interface - clinical
documentation - qualitative - methodology
Background and Significance
Background and Significance
Health information technologies (HITs), such as Electronic Health Record (EHR) systems,
are considered as critical factors in transforming the health care industry.[1] Despite the high EHR adoption rates, substantial gaps exist between the current
state of EHRs and their potential usefulness.[2]
Recently, the HIT end-user community and EHR experts have pointed specifically to
the cognitive challenges resulting from poor EHR usability as one of the key reasons
for this gap.[2] Also, substantial level of disparity exists around perception of HIT usage and its
possible outcomes among its various users, having wide range of technology skills,[3]
[4] further confound the situation. A well-designed EHR graphical user interphase (GUI)
could help address these challenges by improving system usability leading to improvements
in health care delivery.[5]
Usability has been defined in various ways and typically encompasses a set of evaluation
methods to understand user experiences for the purpose of creating more desirable,
usable, and useful products.[6] The International Organization for Standardization (ISO) defines usability as, “an
extent to which a product can be used by specified users to achieve specified goals
with effectiveness, efficiency, and satisfaction in a specified context of use.”[7] Nielsen defines usability as, “a quality attribute that assesses how easy user interfaces
are to use” and describes five basic principles (i.e., easy to learn, easy to remember,
efficient, having minimal errors, and with greater user satisfaction).[8]
[9] An essential approach to account for and resolve usability problems is User-Centered
Design (UCD), which is guided by the philosophy that “the final product should suit
the users, rather than making the users suit the product.”[10]
To date, several EHR usability studies employing various methodological approaches
have been conducted in diverse contexts, such as clinical decision support systems
and dental EHR systems.[2]
[11]
[12]
[13]
[14] Among these methods, ethnography is one of the earliest techniques in which subjects
are observed in a naturalistic setting. Ethnography has also been employed in the
software development cycle for evaluating information systems.[15] This approach to data collection provides rich, realistic, and holistic view of
user behavior in task completion and could aid in gathering additional detailed information,
which users sometimes fail to communicate during more controlled (e.g., laboratory
based) methodological approaches. Similar observational study methodologies have been
used widely in health care research.[16]
[17]
[18]
There is a growing amount of literature providing guidelines and recommendations that
could help improve EHR usability and ultimately enhance patient safety and quality
of care.[19]
[20]
[21] For a comprehensive usability evaluation, a multimethod approach is preferred.[22]
[23]
[24] Despite these recommendations, there is a limited number of studies where the HIT
usability has been assessed employing more than one methodological approach. A few
examples of such multi-method studies are dental EHR evaluation employing user testing
along with observations, interviews, and Goals, Operators, Methods, and Selection
(GOMS) modeling techniques[25]; computerized provider order entry system assessment using two different sets of
heuristics along with usability testing[26]; and diabetes mHealth system evaluation employing combination of user testing with
semistructured interviews and questionnaires around patients' experiences using the
system.[22] Furthermore, there is a limited number of research studies that present usability
comparisons from viewpoints of people with a diverse set of perspectives, e.g., expert
users versus novice users,[27] physician versus patients,[28] and users versus usability experts.[29]
One specific area needing attention is the design and functionality offered by these
EHR systems' GUI around clinical notes usage. There are several challenges associated
with clinical notes usage. Clinical notes may be difficult to find, time consuming
to enter, contain poorly formatted information that is difficult to read, incorporate
erroneous or out-of-date information, or lack standardized content display within
EHR systems.[30]
[31] Despite these known usability problems, EHR clinical notes remain essential resources
for clinicians who use them to communicate, summarize, and synthesize patient care
information for decision making. Physicians and other clinicians are challenged, both
when entering information into and retrieving information from clinical notes, as
current EHRs may not sufficiently support these tasks. To date, few studies have examined
usability of the user interfaces pertaining to clinical notes. A few examples of more
recent studies are time-and-motion studies reporting that note documentation should
be treated as synthesis rather than composition, and the documentation process could
be best supported by incorporation of various search tools that could facilitate note
construction[32] and eye tracking studies on physicians' visual attention while reading electronic
progress notes revealing that most time was spent in slowly reading the “impression
and plan” section of progress notes with minimal time spent on sections, such as “medications,”
“vital signs,” and “laboratory results,” even when there was additional information
on these sections.[33]
Comprehensive understanding of currently employed EHR systems in terms of the functionality
and design elements offered by their GUIs, is an essential initial step toward redesigning
future, user-centered EHR systems.
Objective
This research study was conducted to answer for the following questions: What are
the various designs and functionality features pertaining to the clinical notes usage
offered by GUIs of two existing EHRs systems, and how could these features potentially
influence EHR usability as ascertained by usability evaluators' and users' viewpoints?
The insights derived from user observations and comments would provide interface designers
an initial platform to help generate the future EHR clinical notes interface that
is better aligned with user's needs, usability evaluators' suggestions, and usability
guidelines.
Methods
General Description and Setting
An ethnographic field study,[34]
[35] supplemented by a postobservation questionnaire, was performed to collect data about
the daily activities of EHR users in their naturalistic settings. Participant observation
was performed by immersing in physicians' routine daily activities and collecting
rich data about their interaction with EHRs while performing clinical documentation
tasks. Participant physicians were briefed about project goals, the methodology employed
to collect data, and instructions on traditional, concurrent think aloud method (i.e.,
to share their thoughts audibly about clinical notes usage while interacting in “real
time” with the GUI of a particular EHR system). Informal conversation was also performed
between observers and physicians to gain an understanding of any emerging issues.
Field notes were documented with an electronic tablet using a time-stamped application
(Timestamped Field Notes Application 3.0).[36]
Internal Medicine resident physicians were observed interacting with one of the two
different EHR systems in the inpatient environment of two tertiary care centers (System-1,
a commercial vendor system at the Location-A and System-2, an open source system at
the Location-B). Because residents who participated in this study spent most of their
time interacting with EHRs in workrooms, particularly for clinical notes usage-related
tasks, the majority of observations were performed in physician workrooms. Each resident
was observed on different days of the week (4–5 days) and during various sections
of the day (e.g., prerounding, rounding, and postrounding). In general, Location-A
had a more diverse patient population needing treatment for more complex medical and
surgical conditions, whereas at Location-B, patients were older, predominantly males,
and mainly coming in for treatment of chronic medical conditions and psychiatric diseases.
Study Sample
A total of 12 (6 per system) mid- and senior-level resident physicians, in their second
through fourth years, enrolled in Internal Medicine Categorical or Internal Medicine
Combined programs, were recruited for the study. Interns, medical students, advanced
practice providers, attending physicians, and other clinicians (nurses, physicians'
assistants etc.) were excluded. The characteristics of participants, summarized in
[Table 1], were similar across the two sites. Study participants were given a $50 gift certificate
as an incentive for their participation.
Table 1
Characteristics of resident participants
|
Location-A
|
Location-B
|
|
Mean age (y)
|
31 (±3.6)
|
29.5 (±1.6)
|
|
Mean years in training
|
2.8 (±0.4)
|
3 (±0.6)
|
|
Gender
|
|
|
|
Female (%)
|
4 (66.6)
|
3 (50)
|
|
Male (%)
|
2 (33.3)
|
3 (50)
|
Because of the complexities associated with evaluating EHR system usage, employing
usability evaluators with dual domain knowledge (both usability experience and health
care knowledge) was crucial. Two of the authors (R.F.R., a health informatician and
physician and G.M.H., a health informatician and clinical researcher with a Masters
of Public Health) were assigned this role.
Data Collection
Data regarding the usability and functionality of each EHR's clinical notes were collected
at both sites by R.F.R. and G.M.H. As noted earlier, the majority of data collection
was done in the residents' workrooms. To ensure a representative sampling of different
activities for each EHR system, each resident was observed on various days of the
week (e.g., on-call and off-call days [refer to admitting and nonadmitting days, respectively],
weekends, and inpatient sections of clinic days) for a total of 4 to 5 days. Observation
times were approximately between 7:00 a.m. and 6:00 p.m., where each resident was
individually observed for 2.0 to 2.5 hours/day and during various sections of the
day (e.g., prerounding, rounding, and postrounding). On average, each participant
was observed for 9 hours (±2.5) at Location-A and 9.6 hours (±1.9) at Location-B,
with a total of over 110 hours spent on observation. The total time included time
spent on note documentation, order entry, chart review, and others. Note documentation
consumed an average of 20 to 30% of the total time, a proportion of time that aligns
with findings from previous time-motion studies.[37]
Observation data were further supplemented by a postobservation questionnaire. Both
close and open-ended questions were employed to collect residents' subjective responses
from two standpoints, clinical notes entry and information-seeking tasks (the sample
questions from the questionnaire can be seen in [Appendix]).
Appendix
Sample questions from the postobservation questionnaire
|
Q1.
|
How much time on average do you think you spend entering a specific note type (e.g.,
H&P progress note, discharge summary)?
|
|
Q2.
|
How do you work around templates of various note types (e.g., H&P progress note, discharge
summary)?
|
|
Q3.
|
What style do you prefer while entering a specific note type, i.e., chronological
order of various sections of different notes (e.g., H&P progress note, discharge summary)?
|
|
Q4.
|
What style do you prefer while reading a specific note type, i.e., chronological order
of various sections of different notes (e.g., H&P progress note, discharge summary)?
|
|
Q5.
|
What are the major limitations of the EHR's GUI in terms of note entry/note retrieval
tasks?
|
|
Q6.
|
How do you think these limitations can be rectified?
|
|
Q7.
|
What are the major strengths of the EHR's GUI in terms of note entry/retrieval tasks?
|
|
Q8.
|
How do you think these strengths can be further improved?
|
Data Analysis
An Ethnographic Content Analysis (ECA)[38] of qualitative data was performed on the observatory notes documented as “field
notes,” employing an integrated qualitative–quantitative research design.[39] These field notes consisted of information on clinical documentation tasks (e.g.,
clinical notes entry or related information-seeking tasks) noted while physicians
were interacting with EHRs and were a combination of direct observations by observers
and comments volunteered by resident physicians. These raw data were later dissected
into groups of words or phrases (the meaning unit, referred as “usability references”
in this study). Each usability reference pertaining to the study “theme,” i.e., functionality
and design elements around clinical documentation tasks, was coded in terms of the
EHR system (e.g., System-1 or System-2) it is referring to and its perceived impact
on usability (positive [P], negative [N], or equivocal [E]). Usability was coded as
positive, negative, or equivocal if the usability evaluators considered the EHR features
to be desirable, undesirable, or ambivalent, respectively. NVivo (version 10.1.3),[40] a qualitative data analysis tool, was used in this study.
The screen shot of the field observers' data collection tool with nodes is shown in
[Fig. 1]. The coding schema pertaining to functionality and design elements around clinical
documentation tasks (i.e., clinical notes entry or related information-seeking, [Fig. 2]) was generated through an iterative process of brainstorming and refinement among
research team members. The team included health informaticians (R.F.R., G.M.H., T.J.A.,
G.B.M., J.L.M.), physicians (R.F.R., T.J.A., G.B.M.), and usability evaluators (R.F.R.,
G.M.H., J.L.M., K.A.H.)., with the latter two members having additional industrial
engineering and experimental cognitive psychology expertise, respectively. Conflicts
were iteratively addressed and resolved.
Fig. 1 Screen shot of the data collection tool and nodes generated.
Fig. 2 Visual depiction of coding scheme used in content analysis.
Two team members (primarily R.F.R. and G.M.H.) coded the notes through repetitive
and comprehensive scanning of the field notes and brainstorming among other coauthors,
ensuring that the final coding schema represented the majority of the source domain
and not merely a small nonrepresentative slice. Intercoder agreement was 98%, with
a kappa value of 0.8. Any remaining coding discrepancies were discussed and resolved
through a consensus process.
Data was analyzed and presented at three hierarchical levels: (1) at the higher level
of subthemes, (2) at the more granular level of categories within those subthemes,
and (3) at the deepest levels of codes within those categories. We analyzed the usability
reference data in the context of various usability features from two standpoints:
(1) frequency (percentage) of being evaluated as positive, negative, or equivocal
under each subtheme, category, or code and (2) their impact on usability as measured
through gauging references to denote a specific usability feature. The references
were gauged by assigning weights against a severity impact scale by two evaluators
(coauthors), R.F.R. and T.J.A., both physicians and health informaticians with expertise
in EHR usability evaluation. A 7-point severity rating scale employed was based on
three variables: (1) percentage frequency of total references, (2) the perceived impact
on user interaction/performance, i.e., the subjective assessment of impact of usability
feature on user interaction/performance, (3) the usage (sporadic or recurrent) of
that particular usability feature. Score for each feature was averaged out between
two evaluators and was categorized into three levels, i.e., high impact (>5), medium
impact (3–5), and low impact (<3). The results were further validated by analyzing
responses obtained from physicians through post-observation questionnaires.
Results
In total, there were more usability references specific to clinical notes use for
System-1 (347) than System-2 (132). Both Systems (1 and 2), had greater number of
references under note entry (276, 103) than information-seeking tasks (71, 29). Usability
references were dissected at three levels of granularity, i.e., subthemes, categories,
and codes ([Figs. 3]
[4]
[5]), cataloged as either positive, negative, or equivocal and were reported as percentage
frequency.
Fig. 3 Frequency analysis of usability references at the level of subthemes. SY, system.
Fig. 4 Frequency analysis of usability references at the level of categories. EC, error
control; NS, navigation and search ability; SY, system; UF, user control and freedom;
WA, workflow accelerators.
Fig. 5 Frequency analysis of usability references at the level of codes. AP, auto population;
CM, communication; CP, copy pasting; DT, dictation and transcription; ED, editability;
EP, error prevention; FO, formatting; NN, navigating for notes; NT, navigating for
templates; OH, online help; OT, others; SC, spell check; SO, screen options; SY, system.
Systems Comparison at the Level of Subthemes
Analysis at the level of subthemes ([Fig. 3]) revealed that System-1, as compared with System-2, excelled in note entry features
by having higher percentage of positive usability references (P = 26% vs. 12%) and
substantially lower negative references (N = 12% vs. 34%). Inconclusive results were
attained for information-seeking tasks, as System-1 in comparison to System-2 had
both lower percentages of positive (P = 14% vs. 28%) and negative references (N = 34%
vs. 41%).
Systems Comparison at the Level of Categories
More granular analysis at the level of categories ([Fig. 4]) showed similar results, i.e., System-1 surpassed System-2 in note entry by having
higher percentage of positive and lower percentage of negative usability references,
specifically with respect to error control, user control and freedom, and work flow
accelerators. Whereas inconclusive results were obtained for information-seeking tasks
related to navigation and ability to search, i.e., System-1 as compared with System-2
showed both lower percentages of positive and negative usability references.
Systems Comparison at the Level of Codes
Analysis done at the deepest level of codes ([Fig. 5]) further revealed the details of note entry features having higher percentage of
positive and lower percentage of negative usability references under System-1 as compared
with System-2, for example, error prevention and spell check, edit ability and formatting,
dictation and transcription, screen options, autopopulation, and communication, except
under copy pasting. With respect to information-seeking tasks related to navigation and ability to search,
the percentages of positive and negative references under System-1 versus System-2
under all four codes, i.e., navigating for notes, navigating for templates, online
help, and others, showed inconclusive results. Overall, at all three levels, a greater
percentage of references were coded as equivocal for System-1 than for System-2 under
both note entry and information-seeking tasks to the coders' uncertainty surrounding
particular usability items warranting further studies.
Severity Impact Rating
The data on usability references denoting a specific usability feature were further
analyzed by assigning an overall severity score. The references were gauged by two
coauthors (R.F.R. and T.J.A.) after assigning each feature a score against a severity
impact scale based on percentage frequency of total references, its perceived impact
on user interaction/performance (positive, negative, or equivocal), and its usage
(sporadic or recurrent). The score was later categorized into three groups as high
impact (>5), e.g., navigating for notes (score = 7), autopopulation (score = 6), screen
options (score = 5.5) and others (score = 5.5); medium impact (3–5), e.g., communication
(score = 5), error prevention (score = 4.5), copy pasting (score = 4.5), edit ability
(score = 4), and dictation and transcription (score = 3.5); and low impact (<3), e.g.,
spell check (score = 2.5), formatting (score = 2.5), navigating for templates (score = 2.5),
and online help (score = 2.5; [Fig. 6]). The severity impact scale used was grounded on three variables: (1) proportion
of references in total, (2) the perceived impact it will have on user interaction/performance,
and (3) its usage (sporadic or recurrent). The severity impact scale is presented
as mean of individual ratings from R.F.R. and T.J.A. (physician and health informaticians).
The above results were further validated by consolidating residents' quotes collected
during observation and from a questionnaire administered to physicians. ([Tables 2] and [3]).
Table 2
Representative sample of quotes from users
|
Negative
|
Equivocal
|
Positive
|
|
Users-System-1
|
–“Screens with too many options/tabs that are not needed or used.”
–“Too many ways to perform the same task adds confusion.”
–“Autopopulation introduces tons of junk and nobody wants to look at this crap.”
–“The autopopulated data are not accurate always.”
“It can be overwhelming at times, because there are so many options to do the same
thing.”
–“Filters are cumbersome.”
|
–“Probably, we spend similar amount of time interacting with EHRs at both locations,
i.e., System-1 at Location-A (more complicated patients, but also more efficient system)
and System-2 at Location-B (less complicated and less efficient system).”
–“Notes comes last, patient care comes first.”
|
–“Summary tab is very useful. I can customize it the way I want.”
–“It has much more reliability/support to have notes/data from outside uploaded in
the charts. I know that if something was given to the records department, it will
be there.”
–“Best thing in it is the short-cut templated phrases!”
–“I can create a well-organized note with different fonts/colors stressing importance.”
–“Note entry is way better!”
|
|
Users-System-2
|
–“To multitask is one of its biggest limitations, and the ability to open multiple
patient charts (in one instance) would greatly simplify this.”
–“I feel that the biggest challenge is multitasking, as we can only work on one patient
at a time without being able to look at multiple data (split screen), very frustrating
when entering notes on a complex patient.”
–“It is quite slow at retrieving large number of notes, which is necessary for complex
patients to be able to look further back into their history.”
–“I find it challenging to retrieve records from outside location-B. The ability to
find records from nationwide is certainly a strength, although it can be rather challenging
to actually find what you're looking for.”
|
–”Notes documentation is the least important chores for the day.”
|
–“I like it's black and white, simplistic interface.”
–“Retrieving notes is awesome, the reason why we love this system.”
–“Retrieving notes function is pretty good.”
–“Consistency in finding documents is one of the strengths of System-2.”
–“I like its simplicity, since there is only one way to find most data points you
would like to see.”
|
Table 3
Innovative ideas from users
|
Ideas
|
|
Users-System-1
|
–“If the physician entered a term BNP in the notes, it should pull up the most recent
BNP lab results of that particular patient.”
–“Other encounters and clinician notes (telephone encounters/nurses' notes), crowd
provider notes. There should be separate tabs for these.”
–“Limited search function could be improved if it had a Google-type search engine
for notes, labs, orders.”
|
|
Users-System-2
|
–“I think System-2 would most likely benefit from the ability to have multiple charts
open at the same time and from use of sidebar similar to System-1.”
–“If we could better understand/billing requirements for note entry, we can have more
structured/standardized notes.”
“In order to address the variability issue in notes structure, we should have standard
templates.”
–“What if the current problems get blown in and then you can actually click on the
problem, which takes you to the relevant previous notes.”
|
Abbreviation: BNP, brain natriuretic factor.
Fig. 6 Frequency comparison of total usability references under System-1 and 2. AP, autopopulation;
CM, communication; CP, copy pasting; DT, dictation and transcription; ED, editability;
EP, error prevention; FO, formatting; NN, navigating for notes; NT, navigating for
templates; OH, help; OT, others; SC, spell check; SO, screen options.
Analysis
Usability evaluation was performed on two widely implemented EHR GUIs around critical
tasks of clinical notes usage through data collected from ethnographic studies along
with postobservation questionnaires. Each EHR system was appraised in terms of percentages
of respective usability references being perceived and cataloged by usability evaluators
as positive, negative, or equivocal. Results were later validated by analyzing physicians'
responses.
EHR Usability Pertaining to Note Entry
Under note entry, System-1 had considerably more positive and comparatively less negative
feedback. The most desirable note entry-related features were autopopulation and screen
options, classified as high impact. Autopopulation functionality, executed through smart phrases,
served as a catalytic agent in the note writing process and was thought to improve
user efficiency during task performance. Conversely, it was also considered as a source
of introducing inaccurate, repetitive, dated, and redundant information leading to
lengthy notes as quoted by various users ([Table 2]). Similarly, the ability to have various screen display options (e.g., split panes,
floating screens) was also considered as a strength, because these features facilitated
concurrent information-seeking tasks with note entry-related tasks. On the contrary,
the inability to multitask was considered to be one of the least favorable aspects
of the system despite the fact that multitasking could be associated with increase
chances of errors. For instance, users were not allowed to open more than one patient's
chart at a time, an error prevention feature, or view previous notes/data within the
same window of the same patient's chart to inform the content of the current note,
thus hindering timely access to relevant patient information.
The ease of communication between other clinicians and EHRs with regard to interoperability,
error prevention through screen alerts, ability to copy paste/easy edit options, and
proficient dictation and transcription services were few of the other medium-impact usability strengths pertaining to the note entry
task that was repeatedly praised by the respective system users. The formatting and
spell check feature, despite having a low impact on usability, was also frequently
praised because it gave users the freedom to customize their notes in different font
styles/sizes/colors.
EHR Usability Pertaining to Information-Seeking Tasks
Under information-seeking tasks, System-2 had a greater percentage of positive as
well as negative observations, whereas ease of navigating for notes was the most favorable
feature having the greatest impact on usability. The likely explanation for the positive
feedback was the simplistic GUI design with intuitive default notes listing display
(e.g., notes from previous encounters were cataloged according to the specialties
with better consistency and ease of finding desired notes). This was in contrast to
the frustration users expressed with the extensive list of notes containing several
options to perform the same tasks (overfunctionality) and the perception that note
filters, offered as a feature, were cumbersome to use. Hence, a sense of information
overload negatively affects intuitiveness and ease of use. Similarly, “others,” corresponding
to the ease of locating ancillary data (e.g., laboratories, imaging), was considered
to be another important aspect of GUI that could substantially impact its usability.
Having ancillary data accessible through various screens rather than through a sole
homepage and a search box to find specific information are a few of the favorable
features that could enhance EHR usability pertaining to clinical notes usage. In addition,
navigating for templates and online help were also considered to be desirable features
despite of their low impact on usability.
Equivocal Results
Under both subthemes for the two systems, i.e., note entry and information-seeking
tasks, a considerable portion of data was coded into the equivocal category more under
System-1 than System-2 because of their uncertain effect on usability. These items
would require a more in-depth and individual study of each feature/item to understand
their influence on usability. We expect that this analysis, however, could yield some
interesting additional findings about these systems.
Discussion
Suboptimal EHR usability, resulting from lack of incorporation of UCD design approach
in the Systems Development Life Cycle (SDLC) process is one of the primary factors
leading to ineffective and inefficient tasks performance (e.g., poor quality or missing
data, increase error rate, challenges with care coordination, compromised patient
safety), dissatisfaction among users (providers), and ultimately poor health care
delivery.
This research study explores the two existing EHRs in terms of their design and functionality
features pertaining to critical tasks centered on clinical notes usage. Data were
collected using multimethod approach, analyzed both from users' and usability evaluators'
perspectives and employing both qualitative and quantitative approaches.
We discovered that GUI of each EHR system being evaluated offered varied sets of design
and functionality features pertaining to the clinical notes usage. Each of these features
could potentially influence EHR usability either positively, negatively, or equivocally,
as ascertained by usability evaluators' and users' viewpoints and could also be assigned
“usability impact score” as measured through a 7-point severity rating scale.
Systems Comparison and Its Implications
We discovered that overall, System-1 surpassed System-2 in clinical notes usage specific
to note entry-related tasks, while both Systems performed equally well on information-seeking
tasks associated with clinical notes usage. Usability features scored as “high impact”
were autopopulation, screen options, navigating for notes, and others; as “medium
impact” were communication, error prevention, copy pasting, edit ability, and dictation
and transcription, and as “low impact” being spell check, navigating for templates,
and online help.
In-depth understanding of desirable and undesirable usability features offered by
existing EHR GUIs could serve as an initial platform to help redesign future EHR interface.
Hence, more efficient and effective task performances associated with greater user
satisfaction that could ultimately result in enhanced health care delivery and better
health outcomes.
Comments and Innovative Ideas by Users
We also solicited several suggestions from users of both systems, which could help
us in designing a new and improved GUI having better overall usability. One user recommended
incorporating advanced technologies, such as login with finger scans or pupil iris
scan to enhance the EHR usability, whereas having a “Google” like search engine was
a common suggestion received from several users. According to some users, standardizing
the structure of the templates used for different note types and establishing a structured
curriculum for medical students/residents about the coding/billing requirements for
notes writing, could result in more standardized note entry, potentially decreasing
note format and content variability. According to one of the users, linking the name
of a laboratory test with the most recently reported result would enhance user efficiency.
With respect to improving usability pertaining to information-seeking tasks associated
with clinical notes usage, users offered several suggestions, such as the idea of
reducing the crowding of notes by incorporating separate locations/tabs based on encounter
types and authors and enhancing user efficiency by entering current problems automatically
and retrieving relevant data pertinent to these problems (e.g., notes, laboratories,
imaging results) by clicking on them.
Study Limitations
Several limitations are associated with this study including a small sample size and
restriction to users from one specialty. All users were second to fourth year residents,
working in an academic setting, having similar ages, training experience, and technology
skills. Also, the field studies were limited to the inpatient setting, whereas EHR
use in a patient care area was not studied. Because of limited resources and paucity
of double evaluators, we employed two authors as evaluators rather than recruiting
them from outside the study team. Our findings are limited by a lack of robust statistical
analysis, because of our small sample size and the qualitative nature of our data.
Also, we did not employ any validated instrument for measuring severity impact rating.
In addition to these limitations, there are potential biases linked with qualitative
data collection and analysis methods, which could result in variability in how results
were presented.
Future Work
In future, comparative analysis of usability features, embedded in various other competing
EHR systems, could be performed by employing different usability evaluation methods
(e.g., heuristic evaluations, cognitive walk through, formal usability testing). To
enhance generalizability of our study findings, EHR usability could also be evaluated
by employing varied and larger sets of clinicians (e.g., attending physicians, specialists,
nurses) and usability evaluators, and in diverse settings (e.g., ambulatory, urgent
care, emergency department). Time-motion studies could also be performed to gauge
the efficiency of performing a particular task and report more precise time to task
data. In addition, further studies are warranted to understand observed discrepancies
in users and usability evaluator feedback about the impact of various features on
usability.
Conclusion
This study helps to illuminate the strengths and weaknesses of varied sets of clinical
notes usage-related usability features offered by two widely implemented EHRs. By
incorporating the desired usability elements and eliminating the undesired ones in
the future EHR design process, we could generate an ideal system that is better aligned
with users' needs and usability guidelines.
Clinical Relevance Statement
Clinical Relevance Statement
Insufficient EHR usability resulting from lacking UCD approach is the leading cause
of the current state of EHRs and their potential usefulness. This study provides an
in-depth analysis of usability strengths and weaknesses of two widely implemented
EHR GUIs with respect to critical tasks around clinical notes usage through analyzing
data collected from real users observed in their actual work environment. The knowledge
gained could serve as a guide in designing a future EHR interface that is better aligned
with a user-centered approach and could ultimately result in improved end-user clinical
notes usage.
Multiple Choice Question
In the system development life cycle process (SDLC), following personnel is often
neglected resulting in suboptimal system usability.
-
Software developers
-
Programmers
-
Users
-
Usability experts
Correct Answer: The correct answer is C, users. Despite the reported benefits ensuing from the meaningful
use of Electronic Health Record (EHR) systems, there exists a substantial gap between
the current state of their use and perceived potentials. One of the fundamental reasons
for this discrepancy is lack of incorporation of a “User-Centered Design” (UCD) approach
during the EHR SDLC process.
