U.S. COVID-19 State Government Public Dashboards: An Expert Review

Naleef Fareed; Christine M. Swoboda; Sarah Chen; Evelyn Potter; Danny T. Y. Wu; Cynthia J. Sieck

doi:10.1055/s-0041-1723989

Applied Clinical Informatics, Table of Contents

Appl Clin Inform 2021; 12(02): 208-221
DOI: 10.1055/s-0041-1723989

Research Article

U.S. COVID-19 State Government Public Dashboards: An Expert Review

Naleef Fareed

¹CATALYST—The Center for the Advancement of Team Science, Analytics, and Systems Thinking, College of Medicine, The Ohio State University, Columbus, Ohio, United States

²Department of Biomedical Informatics, College of Medicine, Institute for Behavioral Medicine Research, The Ohio State University, Columbus, Ohio, United States

,

Christine M. Swoboda

¹CATALYST—The Center for the Advancement of Team Science, Analytics, and Systems Thinking, College of Medicine, The Ohio State University, Columbus, Ohio, United States

,

Sarah Chen

¹CATALYST—The Center for the Advancement of Team Science, Analytics, and Systems Thinking, College of Medicine, The Ohio State University, Columbus, Ohio, United States

,

Evelyn Potter

³Department of Biochemistry, Ohio University, Athens, Ohio, United States

,

Danny T. Y. Wu

⁴Department of Biomedical Informatics, College of Medicine, University of Cincinnati, Cincinnati, Ohio, United States

,

Cynthia J. Sieck

¹CATALYST—The Center for the Advancement of Team Science, Analytics, and Systems Thinking, College of Medicine, The Ohio State University, Columbus, Ohio, United States

⁵Department of Family and Community Medicine, College of Medicine, Institute for Behavioral Medicine Research, The Ohio State University, Columbus, Ohio, United States

› Author Affiliations

Abstract

Background In the United States, all 50 state governments deployed publicly viewable dashboards regarding the novel coronavirus disease 2019 (COVID-19) to track and respond to the pandemic. States dashboards, however, reflect idiosyncratic design practices based on their content, function, and visual design and platform. There has been little guidance for what state dashboards should look like or contain, leading to significant variation.

Objectives The primary objective of our study was to catalog how information, system function, and user interface were deployed across the COVID-19 state dashboards. Our secondary objective was to group and characterize the dashboards based on the information we collected using clustering analysis.

Methods For preliminary data collection, we developed a framework to first analyze two dashboards as a group and reach agreement on coding. We subsequently doubled coded the remaining 48 dashboards using the framework and reviewed the coding to reach total consensus.

Results All state dashboards included maps and graphs, most frequently line charts, bar charts, and histograms. The most represented metrics were total deaths, total cases, new cases, laboratory tests, and hospitalization. Decisions on how metrics were aggregated and stratified greatly varied across dashboards. Overall, the dashboards were very interactive with 96% having at least some functionality including tooltips, zooming, or exporting capabilities. For visual design and platform, we noted that the software was dominated by a few major organizations. Our cluster analysis yielded a six-cluster solution, and each cluster provided additional insights about how groups of states engaged in specific practices in dashboard design.

Conclusion Our study indicates that states engaged in dashboard practices that generally aligned with many of the goals set forth by the Centers for Disease Control and Prevention, Essential Public Health Services. We highlight areas where states fall short of these expectations and provide specific design recommendations to address these gaps.

Keywords

data visualization - coronavirus infections - pandemics - geographic information systems - access to information

Full Text

References

References
1 Wu DTY, Chen AT, Manning JD. et al. Evaluating visual analytics for health informatics applications: a systematic review from the American Medical Informatics Association Visual Analytics Working Group Task Force on Evaluation. J Am Med Inform Assoc 2019; 26 (04) 314-323
2 Streeb D, El-Assady M, Keim D, Chen M. Why visualize? Untangling a large network of arguments. IEEE Trans Vis Comput Graph 2019
3 Wahi MM, Dukach N. Visualizing infection surveillance data for policymaking using open source dashboarding. Appl Clin Inform 2019; 10 (03) 534-542
4 Hester G, Lang T, Madsen L, Tambyraja R, Zenker P. Timely data for targeted quality improvement interventions: Use of a visual analytics dashboard for bronchiolitis. Appl Clin Inform 2019; 10 (01) 168-174
5 Kummer BR, Willey JZ, Zelenetz MJ. et al. Neurological dashboards and consultation turnaround time at an academic medical center. Appl Clin Inform 2019; 10 (05) 849-858
6 Shneiderman B, Plaisant C, Cohen M, Jacobs S, Elmqvist N, Diakopoulos N. Designing the User Interface: Strategies for Effective Human-Computer Interaction. 6th ed.. Hoboken, NJ: Pearson Education; 2016
7 Ware C. Information visualization: Perception for design. 3rd ed.. Waltham, MA: Morgan Kaufmann; 2013
8 Koch T, Denike K. Crediting his critics' concerns: remaking John Snow's map of Broad Street cholera, 1854. Soc Sci Med 2009; 69 (08) 1246-1251
9 Jordan EO. Influenza encyclopedia. Epidemic influenza: a survey. Accessed July 22, 2020 at: http://hdl.handle.net/2027/spo.8580flu.0016.858
10 Cheng CK, Lau EH, Ip DK, Yeung AS, Ho LM, Cowling BJ. A profile of the online dissemination of national influenza surveillance data. BMC Public Health 2009; 9: 339
11 Cheng CK, Ip DK, Cowling BJ, Ho LM, Leung GM, Lau EH. Digital dashboard design using multiple data streams for disease surveillance with influenza surveillance as an example. J Med Internet Res 2011; 13 (04) e85
12 World Health Organization. Ebola situation report – 21. Accessed July 1, 2020 at: https://apps.who.int/ebola/en/status-outbreak/situation-reports/ebola-situation-report-21-january-2015
13 World Health Organization. WHO director-general's opening remarks at the media briefing on COVID-19 - 11 March 2020. Accessed June 1, 2020 at: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020
14 Kamel Boulos MN, Geraghty EM. Geographical tracking and mapping of coronavirus disease COVID-19/severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) epidemic and associated events around the world: how 21st century GIS technologies are supporting the global fight against outbreaks and epidemics. Int J Health Geogr 2020; 19 (01) 8
15 Rosenkrantz L, Schuurman N, Bell N, Amram O. The need for giscience in mapping covid-19. Health Place 2020; (e-pub ahead of print)
16 Muhareb R, Giacaman R. Tracking COVID-19 responsibly. Lancet 2020; (e-pub ahead of print)
17 Grandi S, Bernasconi A. Convergence of web design and spatial, statistical, genomic and epidemiological information: the case of geo-dashboards in the covid-19 crisis. Doc Geogr 2020; 1: 463-476
18 Matheus R, Janssen M, Maheshwari D. Data science empowering the public: Data-driven dashboards for transparent and accountable decision-making in smart cities. Gov Inf Q 2018; 37 (03) 101284
19 Bonifati A, Cattaneo F, Ceri S, Fuggetta A, Paraboschi S. Designing data marts for data warehouses. ACM Trans Softw Eng Methodol 2001; 10 (04) 452-483
20 Center for Systems Science and Engineering (CSSE) at Johns Hopkins University. COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU). Accessed June 1, 2020 at: https://gisanddata.maps.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6
21 Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis 2020; 20 (05) 533-534
22 WorldoMeter. World/countries/United States. 2020 . Accessed June 2, 2020 at: https://www.worldometers.info/coronavirus/country/us/
23 Wissel BD, Van Camp PJ, Kouril M. et al. An interactive online dashboard for tracking COVID-19 in U.S. counties, cities, and states in real time. J Am Med Inform Assoc 2020; 27 (07) 1121-1125
24 Anderson M, Vogels EA. Americans turn to technology during covid-19 outbreak, say an outage would be a problem. Accessed October 7, 2020 at: https://www.pewresearch.org/fact-tank/2020/03/31/americans-turn-to-technology-during-covid-19-outbreak-say-an-outage-would-be-a-problem/
25 Rogers K. Johns Hopkins dashboard: the people behind the pandemic's most visited site. Accessed November 16, 2020 at: https://www.cnn.com/2020/07/11/health/johns-hopkins-covid-19-map-team-wellness-trnd/index.html
26 Limousin P, Azzabi R, Bergé L, Dubois H, Truptil S, Gall LL. How to build dashboards for collecting and sharing relevant informations to the strategic level of crisis management: an industrial use case. 2019 International Conference on Information and Communication Technologies for Disaster Management (ICT-DM); 2019 Paris, France: 1-8
27 Irtaimeh HJ, Obaidat A, Khaddam A. Strategic role of dashboard application in enhancing crisis management capabilities in organizations field study on jordanian cellular companies. Int J Management Sci Business Res 2016;5(10):
28 National Governors Association, Association of State and Territorial Health Officials. Roadmap to recovery: a public health guide for governors. Accessed October 7, 2020 at: https://www.astho.org/COVID-19/Roadmap-to-Recovery/
29 State of Nevada Department of Health and Human Services. Current status. Accessed July 1, 2020 at: https://app.powerbigov.us/view?r=eyJrIjoiMjA2ZThiOWUtM2FlNS00MGY5LWFmYjUtNmQwNTQ3Nzg5N2I2IiwidCI6ImU0YTM0MGU2LWI4OWUtNGU2OC04ZWFhLTE1NDRkMjcwMzk4MCJ9
30 Florida Department of Health Division of Disease Control and Health Protection. Florida's covid-19 data and surveillance dashboard. Accessed July 1, 2020 at: https://experience.arcgis.com/experience/96dd742462124fa0b38ddedb9b25e429
31 Everts J. The dashboard pandemic. Dialogues Hum Geogr 2020; 10 (02) 260-264
32 Centers for Disease Control and Prevention. 10 essential public health services. Accessed November 16, 2020 at: https://www.cdc.gov/publichealthgateway/publichealthservices/essentialhealthservices.html
33 Jaccard P. Distribution de la flore alpine dans le bassin des dranses et dans quelques regions voisines. Bullet Soci Vaudoise Sci Naturelles 1901; 37: 241-272
34 Finch H. Comparison of distance measures in cluster analysis with dichotomous data. J Data Sci 2005; 3 (01) 85-100
35 Ward J. Hierarchical grouping to optimize an objective function. J Am Stat Assoc 1963; 58: 236-244
36 Kulper FK, Fisher L. 391: A Monte Carlo comparison of six clustering procedures. Biometrics 1975; 31 (03) 777
37 Overall JE, Gibson JM, Novy DM. Population recovery capabilities of 35 cluster analysis methods. J Clin Psychol 1993; 49 (04) 459-470
38 Hands S, Everitt B. A Monte Carlo study of the recovery of cluster structure in binary data by hierarchical clustering techniques. Multivariate Behav Res 1987; 22 (02) 235-243
39 Caliński T, Harabasz J. A dendrite method for cluster analysis. Commun Stat Theory Methods 1973; 3 (01) 1-27
40 Duda RO, Hart PE, Stork DG. Pattern Classification. 2nd ed.. New York, NY: Wiley; 2000
41 Ettorchi-Tardy A, Levif M, Michel P. Benchmarking: a method for continuous quality improvement in health. Healthc Policy 2012; 7 (04) e101-e119
42 Centers for Disease Control and Prevention. Information for health departments on reporting cases of COVID-19. Accessed July 22, 2020 at: https://www.cdc.gov/coronavirus/2019-ncov/php/reporting-pui.html
43 Merrilees A. Inconsistencies in official Missouri COVID-19 numbers point to need for standard data reporting. Accessed July 22, 2020 at: https://www.stltoday.com/lifestyles/health-med-fit/coronavirus/inconsistencies-in-official-missouri-covid-19-numbers-point-to-need-for-standard-data-reporting/article_f022ac55-0678-51ef-8a0f-0d053c07e3cf.html
44 Servick K. ‘Huge hole’ in COVID-19 testing data makes it harder to study racial disparities. Accessed July 22, 2020 at: https://www.sciencemag.org/news/2020/07/huge-hole-covid-19-testing-data-makes-it-harder-study-racial-disparities
45 Wamsley L. Fired Florida data scientist launches a coronavirus dashboard of her own. Accessed July 22, 2020 at: https://www.npr.org/2020/06/14/876584284/fired-florida-data-scientist-launches-a-coronavirus-dashboard-of-her-own
46 Harrison TMS, Transparency DS. participation, and accountability practices in open government: A comparative study. Gov Inf Q 2014; 31 (04) 513-525
47 Villeneuve JP. Transparency of transparency: The pro-active disclosure of the rules governing access to information as a gauge of organisational cultural transformation. The case of the swiss transparency regime. Gov Inf Q 2014; 31 (04) 556-562
48 Nacoti M, Ciocca A, Giupponi A. et al. At the epicenter of the covid-19 pandemic and humanitarian crises in Italy: changing perspectives on preparation and mitigation. NEJM Catal 2020; 1 (02) 1-5
49 Wadhera RK, Wadhera P, Gaba P. et al. Variation in COVID-19 hospitalizations and deaths across New York City boroughs. JAMA 2020; 323 (21) 2192-2195
50 Khunti K, Singh AK, Pareek M, Hanif W. Is ethnicity linked to incidence or outcomes of covid-19?. BMJ 2020; 369: m1548
51 Ogen Y. Assessing nitrogen dioxide (NO₂) levels as a contributing factor to coronavirus (COVID-19) fatality. Sci Total Environ 2020; 726 (138605): 138605
52 Allio MK. Strategic dashboards: Designing and deploying them to improve implementation. Strategy Leadersh 2012; 40 (05) 24-31
53 Bharosa N, Janssen M, Meijer S, Brave F. Designing and evaluating dashboards for multi-agency crisis preparation: A living lab. In: Wimmer MA, Chappelet JL, Janssen M, Scholl HJ. eds. Electronic Government. 9th IFIP WG 8.5 International Conference, EGOV 2010, Lausanne, Switzerland: 180-191
54 Ackoff RL. From data to wisdom. J Appl Syst Anal 1989; 16: 3-9
55 Goodman RA, Patrick LR, Howard RJ. Communicating information for action within the public health system. In: Teutsch SM, Churchill RE. eds. Principles and Practice of Public Health Surveillance. New York, NY: Oxford; 2000
56 German RR, Lee LM, Horan JM, Milstein RL, Pertowski CA, Waller MN. Guidelines Working Group Centers for Disease Control and Prevention (CDC). Updated guidelines for evaluating public health surveillance systems: recommendations from the Guidelines Working Group. MMWR Recomm Rep 2001; 50 (RR-13): 1-35, quiz CE1–CE7

Supplementary Material

Supplementary Material