Keywords COVID-19 - dashboard - hospitals - information systems - Iran
Background and Significance
Background and Significance
The coronavirus disease 2019 (COVID-19) was first detected in Wuhan, China, on December
30, 2019. The World Health Organization recognized the disease as a global epidemic
on March 11, 2020.[1 ] The outbreak quickly turned into a critical condition for the health systems around
the world.[2 ]
[3 ] Given the priority of treating COVID-19 patients, health care systems devoted the
majority of their resources to combat the epidemic, which in turn led to the reorganization
of health care services.[2 ]
[4 ] Decision making during the COVID-19 pandemic was a challenge for health system managers,
especially hospital managers at all organizational levels, due to the unknown characteristics
of the emerging disease; lack of vaccines, drugs, and acceptable therapies; the complexity
of the disease in terms of clinical manifestations and modes of transmission; and
the unpredictable consequences of the disease on one hand and its devastating and
deadly waves on the other.[5 ] Moreover, the decisions of health system managers at different levels have an important
impact on the effectiveness of health services[6 ] and the success of health organizations in controlling the COVID-19 pandemic.[4 ]
In the case of the COVID-19 pandemic, the reaction of health systems has been faster
and much more complex than normal.[7 ] However, health system managers and decision makers faced more challenges in the
process of decision making and the provision of appropriate and timely responses due
to the lack of proper understanding of the environment and the high volume of information
they are confronted with in critical situations.[8 ]
[9 ]
Based on the previous experiences, the main challenges of decision making in the current
situation include misunderstanding the scope of the current situation, as well as
information overload experienced by the decision makers.[8 ] In such critical situations, it is very important to design a suitable system that
collects, analyzes, and reports information from available sources.[10 ] In addition, having access to high-precision reports can facilitate the implementation
of support operations, identification of system weaknesses, and provision of effective
solutions for reliable planning.[11 ]
[12 ]
In this regard, technical advancements and integration of all provided information
in a suitable field accelerate decision making.[13 ]
[14 ] Data visualization has provided a huge capacity for the examination of different
dimensions of information and the visual exploration of the relationships among different
components.[15 ] Dashboards are one of the most important data visualization tools.[16 ] They make data evaluation easier and assist users in analyzing information through
the processing of information and identification of factors that can be evaluated.[17 ] Consequently, dashboards provide the possibility of informed and evidence-based
decision making for the managers.[15 ]
The first step in the development of a dashboard is the selection of main performance
indicators and the determination of a basic relationship between them.[18 ] Research showed that a dashboard should provide users with content that is tailored
to their needs and be designed in a way to be easily understood by different users.[19 ]
[20 ] It should be noted that valid and reliable data for the user should be displayed
completely, accurately, and timely.[20 ] Moreover, the dashboard should be easy to use and facilitate the process of decision
making.[21 ]
The challenges of patient care administration and monitoring suspect cases made it
vital to know about the number of daily cases, diagnosis results, and transfers of
COVID-19 patients to hospitals for treatment. During the COVID-19 epidemic, various
studies have designed information dashboards at different levels to better manage
COVID-19 patients.[22 ]
In most previous related studies, information dashboards have been designed nationally
to address the demographic characteristics of patients with COVID-19. Berry et al
developed open access epidemiologic data and an interactive dashboard to monitor the
COVID-19 outbreak in Canada, which included such information as demographic characteristics,
location, report date, travel history, and exposure source. All data are openly accessible
and updated daily.[23 ]
Dong et all developed an online interactive dashboard, hosted by the Center for Systems
Science and Engineering at Johns Hopkins University, Baltimore, Maryland, United States,
to visualize and track reported cases of COVID-19 in real time.[24 ]
Moreover, some studies have designed hospital-level dashboards to track the status
of COVID-19 patients in hospital wards. For example, Vizcaychipi et al developed a
COVID-19 near real-time traffic light system in an acute hospital setting. The input
variables were age, gender, first recorded blood pressure, respiratory rate, temperature,
heart rate, indices of oxygenation, and C-reactive protein.[25 ]
Therefore, this study aimed to design (develop) and implement a COVID-19 management
dashboard at the hospital level in Iran.
Methods
This descriptive developmental applied study was conducted to develop and assess a
COVID-19 dashboard in a big hospital in Mashhad, Iran, using a focus group technique
and a panel of experts. This dashboard was designed in Imam Reza Hospital, which is
a tertiary care teaching hospital in Mashhad, Iran. This hospital has approximately
1,000 beds and provides care to around 6,000 inpatients and 19,000 emergency patients
monthly.
User-centered design (UCD) was adopted for the development of the dashboard. The UCD
is an iterative design process in which designers and other stakeholders focus on
the users and their needs in each phase of the design process.[26 ] The UCD process ensures a stable integration of later users throughout all phases
of the project. With more user focus, the UCD method is one of the most suitable methods
for the recognition of the user's problems.[26 ]
[27 ]
The general phases of the UCD process include:
Specification of the context of use: Identification of the people who are supposed
to use the product, the purpose of use, and the conditions of use.
Specification of requirements: Identification of any business requirements or user
goals that must be met for the success of the product.
Creation of design solutions: This part of the process may be done in stages, building
from a rough concept to a complete design.
Evaluation of designs: Evaluation—ideally through usability testing with actual users—is
an integral part of software development.[26 ]
In other words, UCD steps for the development of a dashboard include identification
of goals, users, data, design and layout, user feedback, dashboard design, and overall
best practices.
This study was performed in four steps according to UCD stages.
The main goals of designing this dashboard for real-time analysis and management of
COVID-19 disease and the general features of this dashboard were examined through
brainstorming during the first focus group discussion. This focus group discussion
was held with the presence of 10 potential users of the system, including hospital
managers, nursing managers, heads of relevant departments (infectious disease specialists),
and heads of COVID-19 committees (emergency medicine specialists). The main criteria
for the selection of participants included their experience and knowledge of hospital
management and their willingness to participate in the study.
In the second phase, the focus group discussion was held in the hospital, with those
who were present in the previous focus group, and the meetings were chaired by the
hospital manager. Initially, users were asked such questions as “What information
do they need to better manage COVID-19 in their field?,” “Why do they need this information?,”
“When do they need this information?,” “How should they access this information?,”
and “Where should they get the needed information?.”
Moreover, the focus group discussed any user goals that must be met for the success
of the product. Therefore, the necessary approach for getting access to each piece
of information, as well as the information sources for inclusion in the COVID-19 information
management dashboard was identified subsequently.
At the end of this phase, the comments were written and summarized, and voting was
done using the nominal group technique in the form of a list of information that needs
to be placed in the COVID-19 information management dashboard. The nominal group method
was used to analyze the focus group data and get a conclusion at the end of the sessions.
Access to each required indicator and information resource was provided and approved
by the majority (75%) of the participants.
Subsequently, the study team including five specialists (i.e., two physicians, one
nurse, one health information management, and one manager) classified indicators in
three classifications of inputs, process, and output after the second focus group
meeting, according to related previous studies.
In the creation of the design solutions phase, the specialized software design team
developed the dashboard prototype using the Power BI Desktop software based on the
identified data elements and requirements from the previous step.
The use of colors in a dashboard can be a plus; however, it is necessary to choose
them according to known rules, as indicated in the study conducted by Pestana et al.[28 ] In this regard, different colors were applied to indicate the importance of information.
In this regard, dark-colored fonts were used to highlight information that required
more user's attention,[28 ] and the barely discernable pale background color was used to provide a more soothing
and less starkly contrasting surface on which the data can reside.[29 ]
Afterward, the designed prototype was integrated with the hospital information system
(HIS) and its capabilities and visual features were displayed to the users. In the
final phase (i.e., evaluation of the designs), 1 week after the prototype was available
to users, the third focus group was held with the same participants who had attended
the second focus group. These 10 main users were asked to comment on suggestions provided
for upgrading the designed dashboard. Furthermore, their satisfaction was measured
based on the end-user satisfaction model with such items as content review, accuracy,
ease of use, and timeliness using a questionnaire that was scored based on a five-item
Likert scale, similar to that used in the Rouhani and Zamenian study,[21 ] in which very good = 4, good = 3, moderate = 2, bad = 1, and very bad = 0. The validity
and reliability of the questionnaire were confirmed. The dashboard software was then
provided to participants to evaluate after working with the dashboard through the
completion of a designed paper questionnaire. The mean ± standard deviation (SD) of
the questions were calculated as well.
Eventually, the final version of the dashboard system was developed after the comments
and suggestions received from users and the required changes were applied to the designed
prototype.
Results
Opinions from potential users of the COVID-19 information management dashboard were
gathered in the initial phase of the study. According to the comments, objectives
of dashboard development included the possibility of informed and evidence-based decision
making for the managers, easier analysis of information through information processing,
and identification of the main performance indicators and factors that can be evaluated.
Regarding general features dashboards, participants agreed that dashboards are likely
to solve the problems of presentation format and information load when certain visualization
principles and features are present (e.g., high data-ink ratio and drill-down features).
Dashboards should have some level of flexibility so that users can switch between
alternative presentation formats. Pop-ups and warnings can help users select an appropriate
presentation format.
At the end of the session, the main users of the dashboard were given their votes.
The criteria for selecting these individuals included membership in the hospital board
of directors or one of the COVID-19 committees, such as the Nosocomial Infection Committee,
and at least 2 years of working experience in the hospital.
The mean ± SD age and mean ± SD work experience of the selected main users were 44 ± 4.11
and 12 ± 6.04 years, respectively. The participants in the study included four females
and six males and their demographic information is presented in [Table 1 ].
Table 1
Demographic characteristics of participants
Variable
Frequency (%)
Gender
Female
4 (40)
Male
6 (60)
Age[a ]
36–41
3 (30)
42–47
5 (50)
48–53
2 (20)
Education status
Bachelor of Science
4 (40)
Master of Science
3 (30)
Specialist
2 (20)
PhD
1 (10)
Field of study
Health economics
1 (10)
Health information management
2 (20)
Management
2 (20)
Nursing
3 (30)
Medicine
2 (20)
Work experience[b ]
5–10
4 (40)
11–16
2 (20)
17–22
3 (30)
23–27
1 (10)
a Mean ± standard deviation (SD) age is 44 ± 4.11.
b Mean ± standard deviation (SD) work experience is 12 ± 6.04.
In the second phase, the main information requirements of caring for COVID-19 patients
were identified with the participation of the main users. Based on the available data,
25 indicators have been selected to be displayed in the COVID-19 dashboard. For a
better understanding of information, participants decided to classify the selected
indicators into three categories (performance, clinical, and demographic categories)
([Table 2 ]). Data on these indicators were extracted from the HIS, automatically. Moreover,
at the request of participants in the focus group, users could visualize the data
summary of each patient and compare the data obtained from different departments.
It was suggested that the number of admissions, discharges, and deaths over a period
of time could be observed as well. Moreover, it was decided to use a table to show
the average request for the diagnostic tests.
Table 2
Indicators for display in a dashboard
Criteria
Indicator
Percentage of agreement
Input
The number of patients admitted to triage
76%
The number of patients admitted to the emergency department
88%
The number of patients referred from other centers
78%
The Number of empty beds
94%
The gender of patients
96%
The age of patients classified into 19 groups (based on the standard table of life
expectancy pertaining to World Health Organization [57])
81%
Location of living (Urban or rural)
77%
Job title (Based on 6 job titles, including unemployed, employed, retired, housekeeper,
student, health professional identified in HIS)
96%
The number of severe COVID-19 patients
92%
Process
The Number of occupied beds
75%
The mean length of stay in departments for COVID-19 disease
95%
The mean length of stay in ICUs for COVID-19 disease
95%
The mean time between graph request and response
99%
The mean time between laboratory tests request and response
84%
The mean time between CT-scan request and response
93%
The number of laboratory tests requests
84%
The number of requested radiographies
82%
The number of CT-scan requests
93%
The number of COVID-19 specific drugs
93%
The number of patients ready to be discharged from the inpatient department
91%
The number of patients ready to be discharged from an emergency department
87%
Output
The number of deaths
93%
The number of patients discharged
84%
The number of patients referred to other centers
98%
The number of patients leaving the triage
88%
Abbreviations: CT, computed tomography; HIS, hospital information system; ICU, intensive
care unit.
Dashboard users could view demographically classified data, such as gender-based data.
In total, eight items were suggested by the second focus group as dashboard requirements:
(1) the ability to access the system through Web browsers without the need to install
a specific platform or operating dashboard, (2) the ability to connect to the health
information system for data gathering and analyzing, (3) the capability to define
access levels for the users, (4) the possession of user-friendly dashboard, (5) the
ability to change, develop, configure, and service the dashboard, (6) the use of color
capabilities in the dashboard, (7) the ability to report online and drill down, and
(8) the possibility to visualize data over a period of time.
Subsequent to the development of the initial dashboard prototype using the Power Bi
server, each user was given a URL to use the dashboard, and access levels were also
defined.
According to the access level set in the initial focus group of the prototype dashboard,
it was possible to provide daily, weekly, and monthly reports according to the selected
indicators. It was also possible to create exception reports based on the users' requests.
This prototype dashboard utilized data from March 2020 to August 2020 based on HIS.
The HIS is an administrative and clinical database containing data from all departments
in the hospital, which provided care to COVID-19 patients. The prototype dashboard
presents summary data of COVID-19 patients at the hospital level for the main users.
In the third step of the focus group, suggestions were made by the members of the
focus group to upgrade the dashboard according to the feedback provided by the users.
After consultation with the technical team for the modification of the relevant dashboard,
it was agreed that the number of deaths, admissions, and discharges should be displayed
in three separate graphs, and the distribution of patients by days of hospitalization
and the average length of stay be displayed in a graph as well. Afterward, a questionnaire
was provided to participants and their satisfaction with the designed dashboard was
measured. The internal and external validity, reliability, and statistical validity
of this questionnaire have been approved by Aggelidis and Chatzoglou.[30 ] The average user satisfaction of the model is presented in [Table 3 ].
Table 3
User satisfaction of the dashboard
Variables based on EUCS model 29
Research variables
Average
The independent variable format
Satisfaction with the dashboard format
4.782 ± 0.321
The independent variable being up to date
Satisfaction with up-to-date status of the dashboard
4.402 ± 0.987
The independent variable ease of use
Satisfaction with the dashboard's ease of use
3.985 ± 0.654
The dependent variable final user satisfaction
Overall satisfaction of the dashboard
4.36 ± 0.451
The independent variable content
Satisfaction with dashboard content
4.46 ± 0.475
The independent variable accuracy
Satisfaction with the accuracy of the dashboard
4.191 ± 0.430
Abbreviation: EUCS, end-user computing satisfaction.
The images of the report page of the designed information dashboard are presented
in [Figs. 1 ]
[2 ]
[3 ].
Fig. 1 The final dashboard visualizes data for patients screened for COVID-19 disease. The
prototype dashboard presents a deidentified patient record populated with clinical
data (imaging diagnostic tests).
Fig. 2 The number of daily admissions, daily discharges, and daily deaths in a sample time
period.
Fig. 3 Distribution of length of stay and mean ± standard deviation (SD) of patient age
in deidentified department.
Discussion
As previously mentioned, the main objective of the present study was to develop and
implement a dashboard system to manage the crisis caused by COVID-19 in a third-level
hospital. In this regard, the main indicators and requirements of such a system were
identified. Therefore, identification of the main indicators and requirements of the
developed dashboard through the participation of potential users in all stages of
development and implementation was the difference between this study and similar studies.[25 ]
[31 ]
Some of the previous studies on dashboard design for pandemic diseases, such as COVID-19,[1 ]
[4 ] severe acute respiratory syndrome, and H1N1 influenza,[32 ]
[33 ]
[34 ] had their focus on the clinical aspect and analysis of data outcomes. The ignorance
of users' views and their participation in designing such information systems has
led to their lack of interest in such systems.[35 ]
[36 ] Therefore, the present study focused on the identification of the information needs
and requirements of such a system from the users' point of view. Based on the study
results, several methods were used to identify and extract indicators and requirements
of the dashboard system which included investigation of similar information systems,
user interviews, and questionnaires.[37 ]
[38 ]
[39 ]
In this study, a UCD was adopted for the development, implementation, and evaluation
of the COVID-19 dashboard. The results of a study performed by Setyawan et al indicated
that this method had a positive relationship with supporting user's participation
in the process of application development.[27 ] In studies in which a similar method has been used, researchers designed a safety
dashboard for patients as well. Their findings showed that this method increased patients'
participation.[40 ]
[41 ] Another study based on this method showed that according to the users' feedback,
visualizations improved situational awareness and have probably provided valuable
information to facilitate informed operational decision making.[42 ]
In this regard, indicators and requirements of the dashboard system were identified
through the examination of similar dashboard systems and interview with users through
focus group meetings.
Users tend to utilize a variety of tools, such as mobile devices, especially smartphones,
as well as other electronic devices, including personal computers (PCs), laptops,
and tablets to access information in Web-based applications, such as dashboards.[37 ]
[43 ] The designed dashboard in the present study is Web-based and can be accessed by
all users through common browsers on electronic devices, such as PCs, laptops, and
tablets, without the need for installation on a specific operating system.
Reasonable decision making is a key element in health care organizations at managerial
levels. A dashboard is a tool that can integrate data from several different sources,
summarize the key performance indicators (KPIs), and provide them to users in a colorful
graphic in real-time to facilitate informed decision making.[38 ] Studies on the design of COVID-19 management dashboards showed that these dashboards
connect to and integrate with other health information systems, such as electronic
health records and health information systems, to receive data from these systems.[14 ]
[39 ]
[44 ]
[45 ] Similarly, the dashboard designed in the present study was integrated with the HIS.
Therefore, the data could be used in real-time to display and manage COVID-19 disease.
Determination of accurate KPIs is one of the primary requirements in the development
of a dashboard. These indicators should be developed in accordance with the needs
and goals of the given organization to provide valid and reliable measurements.[46 ]
[47 ]
[48 ] Demand for intensive care unit (ICU) services will increase under conditions similar
to the COVID-19 pandemic. Therefore, patient triage is critical in situations like
this since there is always a risk that patients may not be properly admitted to the
ICU which may cause more harm and injury to the patients.[35 ] Regarding the fact that the triage, emergency, and ICU departments are involved
in the process of providing health care services to COVID-19 patients, it is important
to monitor the workflow of these units. Accordingly, the indicators required for COVID-19
management in the present study were identified through the assessment of the information
needs of dashboard users. These indicators were classified into three main groups
based on clinical and managerial needs of users, including performance, clinical,
and demographic criteria. Subsequently, the output data and reports of the dashboard
were displayed to managers and clinical specialists in form of color charts and tables.
Based on the results of previous studies, the use of colors in dashboard design led
to a better understanding of the situation and improved data visualization which in
turn reduced the response time and error rate.[36 ]
Similarly, in the study conducted by Grange et al, the related key indicators were
identified and applied to the designed COVID-19 management dashboard.[14 ] Furthermore, interactive Web-based dashboards have been designed during the COVID-19
pandemic surge in different countries to help managers and clinicians better manage
the limitations in human resources, medicine, and medical equipment and take more
effective actions.[1 ]
[31 ]
[49 ]
[50 ]
In addition, user satisfaction is one of the important factors affecting the acceptance
and application of different software.[51 ] However, the review of studies related to the design of COVID-19 dashboards showed
that user's satisfaction with the designed dashboard has been examined in a small
number of them.[52 ] In the same line, user's satisfaction with the designed dashboard was examined in
the present study in terms of the simplicity of use and the extent to which the information
and operational needs of dashboard users were met.
As a result, the examination of users' satisfaction and application of their comments
and feedback in dashboard design led to improved dashboard performance and increased
users' acceptance of the designed dashboard.
It seems that the structure and minimum data included in this dashboard can be used
for similar hospitals due to the similar nature of the disease.
Regarding the limitations of the present study, one can refer to the fact that due
to the critical situation of the COVID-19 pandemic, focus group interviews with potential
users were withheld. Moreover, since the present study was conducted in only one center,
the number of participants involved in the development and implementation of the dashboard
was small.
Conclusion
Information dashboards are important for the provision of access to necessary information
in the shortest possible time. Information dashboards keep managers informed about
the process of patient care, admission, discharge, and incidents, such as the number
of deaths and the use of hospital services. The application of this tool leads to
more evidence-based decision making in the crisis and reduces the likelihood of error
and failure in crisis management.
Furthermore, the simplicity of design and the possibility of connection to the HIS
database allows administrators and users to access the reports they need with higher
confidence and speed and avoid the repeated collection of the required information.
Eventually, the dashboard developers need to receive periodic feedback from the users,
update their information needs, and evaluate the quality of data and dashboard information
to preserve the users' desire to use this system continuously and make the most of
such information management dashboards.
Clinical Relevance Statement
Clinical Relevance Statement
This study can be used as a guide for the implementation and development of a dashboard
for managing various crises, such as COVID-19 disease in hospitals.
Multiple Choice Questions
Multiple Choice Questions
According to the USD method, which is not one of the general phases?
Specification of the context of the use
Specification of requirements
Creation of design solutions
Development a prototype
Correct Answer: Option a is the correct answer. The final phase of the USD method is evaluating designs:
Evaluation—ideally through usability testing with actual users—is as integral as quality
testing is to good software development.
Dashboard indicators in this study are divided into how many main categories?
Correct Answer: Option c is the correct answer; three categories including inputs, process, outputs.
