An HIT Ecosystem Capability Maturity Model for System-Wide Implementation, Management, and Governance

Abstract

Background An effective health system is dependent on the people, processes, and technology required to support its successful use. Technology, including essential health information technology (HIT) components, is not enough to successfully support quality patient care delivery. Without a strong support structure, HIT tools and data alone will fail to meet the desired goal of quality patient care that achieves health equity. Multiple factors, including governance processes, policies, and standards, are critical for the development, implementation, and consistent and effective use of clinical HIT systems and the resulting data. A comprehensive model that provided support for a holistic assessment of the HIT ecosystem could not be identified. The Facility HIT Ecosystem Capability Maturity Model (ECMM) was designed in response to this need.

Objectives This project was designed to create and propose a comprehensive framework for self-assessment and support of a maturation pathway for a facility's comprehensive HIT ecosystem, including governance processes, policies, and standards.

Methods Iterative methods based on agile and human-centered design practices were used to create and validate the framework and its contents.

Results The ECMM and toolkit support a health care facility's clinical or technical leadership, including chief medical officers, chief information officers, and chief medical/health/nursing informatics officers, in identifying and assessing policies, processes, and capabilities surrounding HIT. This framework supports iterative evaluation and step-ladder style progression and goal setting to achieve desired capabilities, both at the local level and at the level of supporting organizations. Reviewer feedback was used to provide iterative tool improvement and refinement.

Conclusion The proposed HIT maturity model toolkit showed potential as a tool to help empower health care facilities and their leadership to advance their HIT ecosystem.

Keywords

clinical information systems - governance - system improvement - evaluation - leadership (enterprise level)

Background and Significance

Indian Health Service (IHS) provides a comprehensive health service and is the primary source of individual and public health care services for 2.6 million American Indians and Alaska Natives. Its health information technology (HIT) system was adequate in the past, but the need for improved interoperability and usability prompted a modernization evaluation.[1] In 2018 to 2019, a research initiative was undertaken to develop recommendations for the IHS through an HIT modernization decision-making process. This evaluation uncovered a generic need for a repeatable process to support facility and organizational IT self-assessment and improvement, resulting in the creation of the current proposed model.

The Challenge

An escalating need to meet quadruple objectives—to help ensure health care access that meets the clinical needs of individuals, as well as populations, address the burden of chronic disease and the associated costs, achieve consistent health care quality and safety, and improve patient care experience—necessitates a framework built upon the principles of continuous quality improvement.[2] The IHS Modernization Initiative, executed through a human-centered design lens, revealed the need for a tool that supports implementation, modernization, and secure data management for HIT systems.

Maturity Model Solution

A process model for improving software development was first initiated in 1986 by the Software Engineering Institute (SEI) at Carnegie Mellon University.[3] Leveraging Phillip Crosby's management stages and ideas around continuous improvement,[4] the SEI developed the model to measure levels of organizational maturity. Since the development of this first model, maturity model use has expanded to disciplines outside of software development.[3]

Maturity models have proven to be useful in helping achieve continuous improvement, the development of shared visions, and the creation of a prioritized roadmap.[5] When applied to HIT, maturity models facilitate the improvement and management of HIT function.[6] These models support assessment and improvement in multiple domains including governance management, HIT maturation, and change management. They are particularly well suited in the governance of HIT systems–systems crucial to the efficiency, quality, and safety-of-care delivery. Leveraging a maturity model in the governance of information assets can reap significant benefits, including cost reductions associated with information management, creation, and support for continuous security assessment postures, appropriate policies and secondary compliance, increased workforce policy awareness, as well as enhanced analytics capabilities, for the coordination of patient care and population health management.[7]

The proposed Facility HIT Ecosystem Capability Maturity Model (ECMM) synthesizes existing maturity models, the best practices, and lessons learned during the HIT Modernization Initiative. The ECMM and toolkit are designed to promote the use of the best practices that enable and support quality patient care through continually maturing HIT governance, policies, capabilities, and processes in a step-ladder style progression.

Objectives

Although HIT has become essential in modern medical facilities, system components alone are inadequate to successfully deliver and support quality patient care. The individual governance processes, policies, and standards that surround HIT are essential for the consistent use of the system and data. Effective information governance is at the heart of safe and efficient health care delivery, and the ECMM is a tool to ensure that governance is established and matured.[2]

This comprehensive tool was designed to strengthen the HIT ecosystem by the following:

• Delineating institutional components that contribute to the successful use of HIT to improve patient care.

• Promoting continuous learning and improvement by providing self-assessments with clear guidance for interventions and milestones for advancing maturity.

Methods

The model creation process initially identified existing models, but extended these models through an iterative process that was informed by published guidance, subject matter experts (SMEs), and findings from the IHS Modernization Initiative. While the synthesized model was developed during this initiative, the model was intentionally designed to be generalizable and useful in other settings.

The process used to define and develop the model is displayed in [Fig. 1]. Although the process is depicted linearly, the actual evolution of the model consisted of iterative and repeated feedback and refinement consistent with agile development.

Fig. 1 ECMM iterative development process. ECMM, ecosystem capability maturity model; HIS, Indian health service.

Defining Levels of Maturity

Existing models and the best practices relevant to HIT governance, IT security, and other domains supporting Facility-level HIT were reviewed to identify maturity levels. Because MEASURE Evaluation, which focuses on strengthening HIT in low-resource settings, had performed research to converge on maturity levels, the team used the levels of maturity as synthesized in the MEASURE Evaluation Health Information Systems Interoperability Maturity Toolkit.[8]

Defining Domains and Subdomains

The team synthesized domains from multiple other existing models,[8] [9] [10] integrating data generated during the IHS Modernization Initiative to identify domains, and subdomains needed to support an HIT ecosystem for facilities and their parent organizations. Once the initial domains were identified, clinical and technical field-based SMEs iteratively evaluated and refined the model's domains and subdomains.

Defining ECMM Cell Contents

An iterative process was used to generate and synthesize the model contents. Content from other maturity models for health and related technology fields were reviewed and synthesized to align with the ECMM framework. Content was also generated from consultation with individuals with expertise in designing and implementing HIT projects, health informatics, and the facility HIT system.

Model Validation Methods

The team used two different methods to elicit feedback and assess the ECMM's applicability and usability in a health care setting. To assess the model's breadth, IHS Modernization Initiative personnel provided feedback on the domains and subdomains of the maturity model. To assess the model's usability and application, five SMEs from different facilities performed assessments using the Healthcare Information and Management Systems Society (HIMSS) maturity model[9] and provided feedback using the “think aloud” usability testing method ([Fig. 2]).[11]

Fig. 2 ECMM validation process. ECMM, ecosystem capability maturity model; HIT, health information technology; SME, subject matter expert.

During each “think-aloud” session, the ECMM team introduced the purpose of the ECMM and the assessment process outlined in the toolkit. While reading the maturity description for a specific subdomain, SMEs described their thoughts on relevance and applicability. SMEs were also asked about the usability of the ECMM and the ECMM assessment process. The SMEs' feedback was incorporated into the ECMM.

Results

The project resulted in the creation of the ECMM and toolkit. The maturity model comprised a vertical and a horizontal axis. The columns, representing the vertical axis, identify the sequential levels of maturity. The first level of the model is Nascent where activities are primarily undeveloped or may not be occurring. The highest level of maturity is Optimized where all activities are defined, documented, standardized, reproducible, and routinely reviewed for improvements or adaptations to the HIT ecosystem.[8] These levels are further defined in [Fig. 3].

Fig. 3 ECMM maturity levels. ECMM, ecosystem capability maturity model; HIT, health information technology.

An individual facility's goal may not be able to reach the Optimized level in every domain and subdomain. Facilities will determine their desired maturity level for each domain; these goals may change over time. Because of the nature of the model, levels are progressive within a domain and are not intended to be skipped. For example, to begin assessing maturity at the Institutionalized level, all criteria at the Established level should have been met for that domain.

The rows, representing the horizontal axis, contain the domains and subdomains for the HIT capability maturity model. [Fig. 4] lists the domains in the circles and the subdomains in the bullets. Since the capabilities required to support complex and interrelated health care business processes are often interconnected, some domains and subdomains may overlap. The goal is to improve patient care; these domains are designed to strengthen HIT to better support health workers’ abilities to meet patient needs.

Fig. 4 ECMM domains and subdomains. ECMM, ecosystem capability maturity model.

While the model is focused on health care facilities, parts of the HIT ecosystem may be supported by a parent organization. Each subdomain of the model displays capability maturity descriptors for the organization (O) and the individual local facility (F). Each row contains cells with descriptions of the capabilities for a specific domain, level, or maturity. Cooperation between the facilities and their parent organization is imperative for the organization to mature successfully. As part of a learning organization, facilities need to be able to provide their knowledge and input into the organization and vice versa.

Using data governance as an example, [Fig. 5] illustrates the security, privacy, and confidentiality subdomain of the ECMM. In [Fig. 5], the individual cells in the Organization row represent the capabilities that were synthesized from other models,[8] [10] [12] [13] the IHS Modernization findings, and SMEs. The cells in the Facility row represent the capabilities that a facility will need to mature the subdomain. In the model, the first column is Nascent where activities are primarily undeveloped or may not be occurring at all, and the Optimized column represents the capabilities that would be demonstrated in a mature organization. The cells in between represent the steps necessary to mature the subdomain of security, privacy, and confidentiality. For example, for a facility to be evaluated as Emerging, all of the capabilities in the Emerging column in the Facility row would need to be met.

Fig. 5 ECMM subdomain example. ECMM, ecosystem capability maturity model.

Discussion

Successes

The ECMM displayed potential to improve a health facility's HIT maturity. Qualitative feedback from the SMEs during validation highlighted its unique aspects. The model was deemed flexible enough to meet the scope and goals of the facility by allowing omission of domains and subdomains. The model supports the practice of reaching optimization through stepwise progression, allowing reviewers to better understand the concrete steps to improve the organization from within. Most importantly, the separation between the Facility and Organization levels helped to delineate and attribute facility and organizational responsibilities.

Lessons Learned

The ECMM is a model developed and designed to support a maturing health care system. The following key lessons were learned during the creation and evaluation of the model:

• Early engagement with end users is essential to ensure that the model meets the needs of the stakeholders and to generate ownership of the model.

• Organizations and facilities have their own language and organizational culture. It is important that the model is flexible and can be modified to meet contextual needs.

• Where applicable, separate Facility- and Organizational-level maturity assessments are key; parent organizations usually provide some level of governance, policy, and or services that facilities are expected to adopt. Other maturity models fail to note this separation, potentially creating a maturity mismatch, and subsequent tension between organization and facility.

• A maturity model's value lies in facilitating a conversation about the maturity of specific capabilities and prioritizing the continued improvement of those capabilities. While a maturity model contains a continuum of maturity levels, it should neither be thought of as a grading scale nor should it be used to compare facilities.

• Having a stepwise roadmap for improving a capability is well received by users, as opposed to simply providing guidance for the best practices without some sort of roadmap or path to maturity.

Conclusion

The ECMM is a comprehensive maturity model designed to support health care facility and organizational leadership to assess their ecosystem and chart a path toward maturation. This tool is designed to spark reflection and highlight the action needed to mature the many capabilities, processes, and policies required to maximize the benefit of health technology solutions and provide the technical foundation essential in providing quality care.

As the model progresses, pilot use and continued development of the ECMM is recommended. The model is a living artifact that can grow and evolve with the changes in facility HIT over time. The tool should be evolved by adopters to meet their specific needs for their ECMM.

Clinical Relevance Statement

Health information technology (HIT) CMMs are tools that allow health care facility leadership to assess their HIT capability levels. The ECMM can be used to perform an organizational self-assessment on the comprehensive domains outlined in the ECMM and toolkit. This assessment can inform a path toward continually strengthening health care facility HIT ecosystems with the goal of ultimately benefiting care teams and, consequently, improving health care outcomes.

Conflict of Interest

J.S., J.A., and T.C. report funding from the DHHS Office of CTO during the conduct of the study.

Acknowledgments

We thank the IHS CMM team of the IHS HIT Modernization Project for its expertise and support in the development of the ECMM and toolkit. We also thank all other IHS HIT Modernization Project personnel for their expertise, feedback, and support during all stages of the ECMM model and toolkit's development and verification process. We thank the Indian Health Service and all reviewers who provided their perspectives. We thank the Department of Health and Human Services and Emerging Sun, LLC, for supporting this project.

Authors' Contributions

Design and refinement of ECMM: J.S. and J.A. Model validation: T.C. Manuscript drafting: J.S. and J.A. Manuscript approval: T.C.

• References

• 1 Indian Health Service. Background information: HHS OCTO Indian Health Service Health Information Technology Modernization Initiative (IHS HIT MOD). Available at: https://www.ihs.gov/hit/background-information/ . Published July 14, 2019. Accessed February 15, 2020
• 2 Baird A, Furukawa MF, Rahman B, Schneller ES. Corporate governance and the adoption of health information technology within integrated delivery systems. Health Care Manage Rev 2014; 39 (03) 234-244
• 3 Paulk MC. . General knowledge: a history of the capability maturity model for software. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.216.199&rep=rep1&type=pdf . Accessed February 15, 2020
• 4 Boti LA, Oltean MA, Hadarean-Somlea I-N. Philip Crosby's contributions in quality management. Rev Management Economic Engineering 2011; 10 (01) 203-208
• 5 Williams PA, Lovelock B, Cabarrus T, Harvey M. Improving digital hospital transformation: development of an outcomes-based infrastructure maturity assessment framework. JMIR Med Inform 2019; 7 (01) e12465
• 6 Carvalho JV, Rocha Á, Abreu A. Maturity models of healthcare information systems and technologies: a literature review. J Med Syst 2016; 40 (06) 131
• 7 Gomes J, Romão M. Information system maturity models in healthcare. J Med Syst 2018; 42 (12) 235
• 8 MEASURE evaluation. Health information systems interoperability maturity toolkit. Available at: https://www.measureevaluation.org/resources/tools/health-information-systems-interoperability-toolkit . Accessed February 15, 2020
• 9 HIMSS. Healthcare provider models. Available at: https://www.himssanalytics.org/healthcare-provider-models/all . Accessed February 15, 2020
• 10 ARC Advisory Group. Cybersecurity maturity model. Available at: https://www.arcweb.com/industry-concepts/cybersecurity-maturity-model . Accessed July 9, 2019
• 11 Birch PDJ, Whitehead AE. Investigating the comparative suitability of traditional and task-specific think aloud training. Percept Mot Skills 2020; 127 (01) 202-224
• 12 Krebs B. . What's your security maturity level? Available at: https://krebsonsecurity.com/2015/04/whats-your-security-maturity-level/ . Accessed July 9, 2019
• 13 Spitzner L. . Defining the security awareness maturity model. Available at: https://www.sans.org/security-awareness-training/blog/defining-security-awareness-maturity-model . Accessed July 9, 2019

Publication History

Received: 17 March 2020

Accepted: 19 June 2020

Article published online:
12 August 2020