Toward a Learning Health Care System: A Systematic Review and Evidence-Based Conceptual Framework for Implementation of Clinical Analytics in a Digital Hospital

Han Chang Lim; Jodie A. Austin; Anton H. van der Vegt; Amir Kamel Rahimi; Oliver J. Canfell; Jayden Mifsud; Jason D. Pole; Michael A. Barras; Tobias Hodgson; Sally Shrapnel; Clair M. Sullivan

doi:10.1055/s-0042-1743243

Applied Clinical Informatics, Table of Contents

CC BY-NC-ND 4.0 · Appl Clin Inform 2022; 13(02): 339-354
DOI: 10.1055/s-0042-1743243

Review Article

Toward a Learning Health Care System: A Systematic Review and Evidence-Based Conceptual Framework for Implementation of Clinical Analytics in a Digital Hospital

Authors

Han Chang Lim^*

¹Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia

²Department of Health, eHealth Queensland, Queensland Government, Brisbane, Australia
Jodie A. Austin^*

¹Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia

²Department of Health, eHealth Queensland, Queensland Government, Brisbane, Australia
Anton H. van der Vegt^*

³Information Engineering Lab, School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Brisbane, Australia
Amir Kamel Rahimi

¹Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia

⁴Digital Health Cooperative Research Centre, Australian Government, Sydney, New South Wales, Australia
Oliver J. Canfell

¹Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia

⁴Digital Health Cooperative Research Centre, Australian Government, Sydney, New South Wales, Australia

⁵UQ Business School, Faculty of Business, Economics and Law, The University of Queensland, St. Lucia, Brisbane, Australia
Jayden Mifsud

¹Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
Jason D. Pole

¹Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia
Michael A. Barras

⁶School of Pharmacy, Faculty of Health and Behavioural Sciences, The University of Queensland, PACE Precinct, Woolloongabba, Brisbane, Australia

⁷Pharmacy Department, Princess Alexandra Hospital, Woolloongabba, Brisbane, Australia
Tobias Hodgson

⁵UQ Business School, Faculty of Business, Economics and Law, The University of Queensland, St. Lucia, Brisbane, Australia
Sally Shrapnel

¹Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia

⁸School of Mathematics and Physics, Faculty of Science, The University of Queensland, St Lucia, Brisbane, Australia
Clair M. Sullivan

¹Centre for Health Services Research, Faculty of Medicine, The University of Queensland, Herston, Brisbane, Australia

⁹Department of Health, Metro North Hospital and Health Service, Queensland Government, Herston QLD, Australia

Abstract

Objective A learning health care system (LHS) uses routinely collected data to continuously monitor and improve health care outcomes. Little is reported on the challenges and methods used to implement the analytics underpinning an LHS. Our aim was to systematically review the literature for reports of real-time clinical analytics implementation in digital hospitals and to use these findings to synthesize a conceptual framework for LHS implementation.

Methods Embase, PubMed, and Web of Science databases were searched for clinical analytics derived from electronic health records in adult inpatient and emergency department settings between 2015 and 2021. Evidence was coded from the final study selection that related to (1) dashboard implementation challenges, (2) methods to overcome implementation challenges, and (3) dashboard assessment and impact. The evidences obtained, together with evidence extracted from relevant prior reviews, were mapped to an existing digital health transformation model to derive a conceptual framework for LHS analytics implementation.

Results A total of 238 candidate articles were reviewed and 14 met inclusion criteria. From the selected studies, we extracted 37 implementation challenges and 64 methods employed to overcome such challenges. We identified common approaches for evaluating the implementation of clinical dashboards. Six studies assessed clinical process outcomes and only four studies evaluated patient health outcomes. A conceptual framework for implementing the analytics of an LHS was developed.

Conclusion Health care organizations face diverse challenges when trying to implement real-time data analytics. These challenges have shifted over the past decade. While prior reviews identified fundamental information problems, such as data size and complexity, our review uncovered more postpilot challenges, such as supporting diverse users, workflows, and user-interface screens. Our review identified practical methods to overcome these challenges which have been incorporated into a conceptual framework. It is hoped this framework will support health care organizations deploying near-real-time clinical dashboards and progress toward an LHS.

Keywords

learning health care system - electronic health records and systems - clinical decision support - hospital information systems - clinical data management - dashboard - digital hospital

Full Text

References

References
1 Charles D, Gabriel M, Furukawa MF. Adoption of electronic health record systems among U. S. non -federal acute care hospitals: 2008–2015. Accessed October 7, 2021 at: https://www.healthit.gov/sites/default/files/briefs/2015_hospital_adoption_db_v17.pdf
2 Carroll JS, Quijada MA. Redirecting traditional professional values to support safety: changing organisational culture in health care. Qual Saf Health Care 2004; 13 (Suppl. 02) ii16-ii21
3 Aggarwal A, Aeran H, Rathee M. Quality management in healthcare: The pivotal desideratum. J Oral Biol Craniofac Res 2019; 9 (02) 180-182
4 Oaten J, Stayner G, Ballard J. et al. Baby deaths: hospital failures: an independent investigation has found a series of failures may have contributed to the deaths of 7 babies at a regional Victorian hospital. Accessed July 21, 2021 at: https://search-informit-org.ezproxy.library.uq.edu.au/doi/10.3316/tvnews.tsm201510160050
5 Barnett A, Winning M, Canaris S, Cleary M, Staib A, Sullivan C. Digital transformation of hospital quality and safety: real-time data for real-time action. Aust Health Rev 2019; 43 (06) 656-661
6 Sullivan C, Staib A, McNeil K, Rosengren D, Johnson I. Queensland digital health clinical charter: a clinical consensus statement on priorities for digital health in hospitals. Aust Health Rev 2020; 44 (05) 661-665
7 Mandl KD, Kohane IS, McFadden D. et al. Scalable Collaborative Infrastructure for a Learning Healthcare System (SCILHS): architecture. J Am Med Inform Assoc 2014; 21 (04) 615-620
8 Olsen LA, Aisner D, McGinnis JM. eds; Institute of Medicine (US) Roundtable on Evidence-Based Medicine. The Learning Healthcare System: Workshop Summary. Washington, DC: National Academies Press (US); 2007
9 Platt JE, Raj M, Wienroth M. An analysis of the learning health system in its first decade in practice: scoping review. J Med Internet Res 2020; 22 (03) e17026
10 Li X, Zhao X, Pu W, Chen P, Liu F, He Z. Optimal decisions for operations management of BDAR: a military industrial logistics data analytics perspective. Comput Ind Eng 2019; 137: 106100
11 Sullivan C, Staib A, Khanna S. et al. The National Emergency Access Target (NEAT) and the 4-hour rule: time to review the target. Med J Aust 2016; 204 (09) 354
12 Auliya RAknuranda I, Tolle H. A systematic literature review on healthcare dashboards development: trends, issues, methods, and frameworks. Adv Sci Lett 2018; 24 (11) 8632-8639
13 Buttigieg SC, Pace A, Rathert C. Hospital performance dashboards: a literature review. J Health Organ Manag 2017; 31 (03) 385-406
14 Dowding D, Randell R, Gardner P. et al. Dashboards for improving patient care: review of the literature. Int J Med Inform 2015; 84 (02) 87-100
15 Khairat SSDA, Dukkipati A, Lauria HA, Bice T, Travers D, Carson SS. The impact of visualization dashboards on quality of care and clinician satisfaction: integrative literature review. JMIR Human Factors 2018; 5 (02) e22
16 Maktoobi SMelchiori M. A brief survey of recent clinical dashboards. Accessed January 25, 2022 at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1073.1848&rep=rep1&type=pdf
17 West VL, Borland D, Hammond WE. Innovative information visualization of electronic health record data: a systematic review. J Am Med Inform Assoc 2015; 22 (02) 330-339
18 Wilbanks BALP, Langford PA. A review of dashboards for data analytics in nursing. Comput Inform Nurs 2014; 32 (11) 545-549
19 Veritas Health Innovation. Covidence: systematic review. Accessed January 25, 2022 at: https://unimelb.libguides.com/sysrev/covidence
20 Sirriyeh R, Lawton R, Gardner P, Armitage G. Reviewing studies with diverse designs: the development and evaluation of a new tool. J Eval Clin Pract 2012; 18 (04) 746-752
21 Bersani K, Fuller TE, Garabedian P. et al. Use, perceived usability, and barriers to implementation of a patient safety dashboard integrated within a vendor EHR. Appl Clin Inform 2020; 11 (01) 34-45
22 Cox ZLP, Lewis CMNPCC, Lai P, Lenihan DJMD. Validation of an automated electronic algorithm and “dashboard” to identify and characterize decompensated heart failure admissions across a medical center. Am Heart J 2017; 183: 40-48
23 Fletcher GS, Aaronson BA, White AA, Julka R. Effect of a real-time electronic dashboard on a rapid response system. J Med Syst 2017; 42 (01) 5
24 Fuller TE, Garabedian PM, Lemonias DP. et al. Assessing the cognitive and work load of an inpatient safety dashboard in the context of opioid management. Appl Ergon 2020; 85: 103047
25 Mlaver E, Schnipper JL, Boxer RB. et al. User-centered collaborative design and development of an inpatient safety dashboard. Jt Comm J Qual Patient Saf 2017; 43 (12) 676-685
26 Paulson SS, Dummett BA, Green J, Scruth E, Reyes V, Escobar GJ. What do we do after the pilot is done? Implementation of a hospital early warning system at scale. Jt Comm J Qual Patient Saf 2020; 46 (04) 207-216
27 Schall Jr MC, Cullen L, Pennathur P, Chen H, Burrell K, Matthews G. Usability evaluation and implementation of a health information technology dashboard of Evidence-Based Quality Indicators. Comput Inform Nurs 2017; 35 (06) 281-288
28 Ye C, Wang O, Liu M. et al. A real-time early warning system for monitoring inpatient mortality risk: prospective study using electronic medical record data. J Med Internet Res 2019; 21 (07) e13719
29 Franklin A, Gantela S, Shifarraw S. et al. Dashboard visualizations: supporting real-time throughput decision-making. J Biomed Inform 2017; 71: 211-221
30 Ibrahim H, Sorrell S, Nair SC, Al Romaithi A, Al Mazrouei S, Kamour A. Rapid development and utilization of a clinical intelligence dashboard for frontline clinicians to optimize critical resources during Covid-19. Acta Inform Med 2020; 28 (03) 209-213
31 Kurtzman G, Dine J, Epstein A. et al. Internal medicine resident engagement with a laboratory utilization dashboard: mixed methods study. J Hosp Med 2017; 12 (09) 743-746
32 Merkel MJ, Edwards R, Ness J. et al. statewide real-time tracking of beds and ventilators during coronavirus disease 2019 and beyond. Crit Care Explor 2020; 2 (06) e0142
33 Staib A, Sullivan C, Jones M, Griffin B, Bell A, Scott I. The ED-inpatient dashboard: Uniting emergency and inpatient clinicians to improve the efficiency and quality of care for patients requiring emergency admission to hospital. Emerg Med Australas 2017; 29 (03) 363-366
34 Yoo J, Jung KY, Kim T. et al. A real-time autonomous dashboard for the emergency department: 5-year case study. JMIR Mhealth Uhealth 2018; 6 (11) e10666
35 Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical decision support systems could be modified to reduce ‘alert fatigue’ while still minimizing the risk of litigation. Health Aff (Millwood) 2011; 30 (12) 2310-2317
36 Health IT. gov. 2020–2025 federal health it strategic plan 2020. Accessed October 5, 2021 at: https://www.healthit.gov/sites/default/files/page/2020-10/Federal%20Health%20IT%20Strategic%20Plan_2020_2025.pdf
37 Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci 2016; 11 (01) 33
38 CFIR Research Team-Center for Clinical Management Research. Consolidated framework for implementation research. Accessed July 2, 2021 at: https://cfirguide.org/
39 Hunter SC, Kim B, Mudge A. et al. Experiences of using the i-PARIHS framework: a co-designed case study of four multi-site implementation projects. BMC Health Serv Res 2020; 20 (01) 573
40 Safaeinili N, Brown-Johnson C, Shaw JG, Mahoney M, Winget M. CFIR simplified: pragmatic application of and adaptations to the Consolidated Framework for Implementation Research (CFIR) for evaluation of a patient-centered care transformation within a learning health system. Learn Health Syst 2019; 4 (01) e10201
41 Waitman LR, Phillips IE, McCoy AB. et al. Adopting real-time surveillance dashboards as a component of an enterprisewide medication safety strategy. Jt Comm J Qual Patient Saf 2011; 37 (07) 326-332
42 Moorman LP. Principles for real-world implementation of bedside predictive analytics monitoring. Appl Clin Inform 2021; 12 (04) 888-896

Supplementary Material

Supplementary Material (PDF)