KOI Pond: The “Elevator Pitch” that Adapts to Describe the Breadth and Variety of Clinical Informatics Practice

Abstract Background The practice of clinical informatics (CI) is diverse and frequently tailored to individual skills and interests or to organizational/departmental needs. Prior studies have sought to define core content areas, educational milestones, and practice patterns within the clinical informatics subspecialty (CIS). Unfortunately, no single tool or framework currently exists that can succinctly define an informatician's role regardless of setting. The diversity of informatics practice makes it difficult to have one “pitch” that describes all possible domains, tasks, knowledge, and skills available to an informatician. Objective Using qualitative data from multiple informaticians, provide a succinct framework to describe and compartmentalize the various functions an informatician can contribute to the healthcare field. Methods We created an iterative focus group of five CIS fellows enrolled in different fellowship programs nationwide, one CIS program director, and an MD-PhD candidate in biomedical informatics. After much discussion, iteration, and consideration of career options within a young and burgeoning subspecialty, a dual-axis model was created that describes CIS practice in terms of settings (internal, external, and policy) and focus areas (knowledge, operations, and innovation). Results and Conclusion By combining both axes into a single “KOI pond” and then prioritizing sections by interest/resource investment, we are able to generate a unique snapshot for each informatician. These snapshots can be used (1) by informaticians to characterize their own practice succinctly as a pitch, (2) by CIS fellows who are considering career options, and (3) by those unfamiliar with CIS who want to learn more.


Background and Significance
The clinical informatics subspecialty (CIS) is a newly accredited subspecialty, the scope of which expands into all areas of healthcare. Although some aspects of healthcare informatics date back as far as the 1960s, 1 the inaugural examination from 2013 serves as the time in history that nearly 500 informaticians were awarded the first CIS board certification. 2 Currently, the Accreditation Council for Graduate Medical Education (ACGME) defines CIS as "the subspecialty of all medical specialties that transforms healthcare by analyzing, designing, implementing, and evaluating information and communication systems." 3 Diplomates from clinical informatics (CI) fellowship programs are uniquely multidisciplinary: often times a CI fellow will represent an entirely different specialty from their program director, mentors, and peers in the same program. This camaraderie necessitates a broader scope of thinking, and as such, fellows frequently harness interest-driven projects to advance their understanding of CI core content.
This variety in projects, tasks, and lessons learned has its disadvantages. For trainees who engage in project-based learning, it is nearly impossible to replicate one's educational experience, even within the same institution. Additionally, given its relative nascency, CI is often poorly understood by the medical community at large and frequently mischaracterized as "computers in medicine." 2 The presence of similar-sounding terms such as bioinformatics, public health informatics, and imaging informatics help add to the confusion, 4 and for years even these definitions were subject to debate within the literature. [5][6][7] Thus, an established and agreed-upon core content 8 and a description of training program requirements 9 were integral to the creation of CIS. 2 Unfortunately, CIS remains poorly understood. One cannot simply describe the role of CI by core content alone (►Fig. 1), and those just learning about CIS may feel confused or overwhelmed as they read about the competencies and curricula for CI, pathology informatics, nursing informatics, dentistry informatics, medical librarianship, or related fields. 10,11 Adding to this, no set standard or job description exists for those who have undertaken CIS training. In 2016, an American Medical Informatics Association (AMIA) task force defined a chief clinical informatics officer (CCIO) role in terms of knowledge, education, and operational skillsets. 12 This role sought to expand beyond the chief medical information officer (CMIO) position 13 in a manner that also included nursing, dental, and pharmacy informatics perspectives. Despite this, neither CCIO nor CMIO roles consider informaticians working in academic, startup/consultant/vendor, or government/public policy settings. Also, neither role is exclusive to CIS diplomates.
In 2018, a practice analysis was sent to the 1,695 CIS diplomates who were certified by the American Board of Preventive Medicine. 14 Data collected from the 316 responses (18.8%) were compiled to create a delineation of practice (DoP) containing five domains, 42 tasks, and 139 knowledge and skill (KS) statements. The five domains from this DoP are: (1) fundamental KS, (2) improving care delivery and outcomes, (3) Enterprise Information Systems, (4) data governance and data analytics, and (5) leadership and professionalism. These domains have now supplanted the 2009 core content paper 9 as the basic roadmap for CI fellowship training programs, and they will be summarily assessed in future CIS certification exams.
In many ways, this CIS practice analysis draws parallels to the Model of the Clinical Practice of Emergency Medicine (EM Model). 15 Both models are critically important for defining their specialty's skills and core content in a manner that ultimately determines its educational curriculum and specialty board examinations. However, neither the EM Model nor the CIS DoP can stand alone as a succinct description of their respective medical positions. The breadth of CIS compounds this fact, and as such the need exists for a shorter characterization of the subspecialty.

Objectives
More than ever, informaticians are in dire need of a succinct description of what they do. As CIS continues to grow, informaticians will need to be able to clearly and accurately describe their skills and interests in a manner that is Fig. 1 Word cloud of clinical informatics core content areas. This word cloud was generated using core content areas defined in the 2009 Gardner publication. 8 The relative size of each term denotes its frequency of use. ACI Open Vol. 4 No. 1/2020 accessible to those unfamiliar with CI. Furthermore, given how this field has already expanded, the traditional CMIO/CCIO role is unlikely to serve as the sole framework that describes every career path an informatician may take.

Methods
From 2016 to 2017, a model was created using an iterative focus group 16 comprised of five CI fellows enrolled in different CI fellowship programs, one CI program director, and an MD-PhD candidate in biomedical informatics. Initial concepts were created by authors J.D.M. and M.W., who were considering career options at the time, and with some overall guidance by B.P.L. As each new member was included into the focus group (C.S., E.J.B., J.F.P., and L.V.G.), the model was continually revisited to include each member's perspectives and considerations. Discussions were held in-person at AMIA's 2016 and 2017 annual symposia, asynchronously via Slack (Slack Technologies, San Francisco, CA), and synchronously via Google Hangouts (Google LLC, Mountain View, CA). Concepts were expanded and refined into a dual-axis approach that follows the motif of a broad and adaptable subspecialty. During much of this time, the group's CI fellows were actively considering their career options in this new subspecialty, which served as a major inspiration for the model. The final version of this model has remained stable since 2017, defines each practice area loosely in terms of setting (Axis 1) and focus area (Axis 2), and is currently being used by the authors to help manage their own CI projects and employment.

Model: The KOI Pond
Our model builds on the notion of a Japanese KOI pond. KOI are ornamental carp that differ widely in their colors, patterns, and scales. 17 Given their diverse morphologies, their adaptability to many pond environments, and their ease of care, unique patterns of KOI can be appreciated in decorative ponds worldwide. CIS is equally diverse, so we found this motif ideal. Consider each informatician as a single fish in an expansive, interconnected pond (►Fig. 2).

Axis 1: the pond ¼ practice setting
The first axis is the pond. For fish to remain healthy in their environment, they need: • Water ¼ internal health systems that "provide the care" • Container ¼ external organizations that "support the care" • Filter ¼ policy agencies that "oversee the care" Internal: Provides the Care Our analogy defines internal systems (water) as any health system or setting where direct or indirect patient care is provided. This may be in the form of a hospital, a clinic, a single practitioner, a skilled nursing facility, or a telemedicine service. Indirect or ancillary care areas are also included, such as radiology, pathology, pharmacy (hospital and retail), allied health professionals, and similar areas.

External: Supports the Care
External (container) is defined as any organization that supplies a product or service to the internal system. Importantly, the organization that supports the care should not be providing direct or indirect patient care as their service. This support may be in the form of physical products, such as medical devices or medications; digital products, such as electronic medical records or apps; or a basic service, such as a survey or nonmedical follow-up call. Startups, incubators, and accelerators that focus on healthcare without providing direct or indirect patient care are also included. External products may include publications, conferences, and studies that target multiple health systems. Finally, the work that a healthcare system does when coordinating with one of these support organizations may also be considered external, albeit from a different perspective.

Policy: Oversees the Care
Policy (filter) is what defines the environment for both internal and external systems. In the same way that oxygenation and cleanliness of a pond can dictate how active KOI may be or the size to which they grow, policy creates the incentive structure in which informatics operates. Any guideline, rule, or regulation placed upon either of these systems counts as policy. This may include government organizations, accreditation/regulating bodies, and organizational committees such as data governance. Textbooks, evidence-based medicine practices, and established standards of care are also included in this category.

Axis 2: the KOI ¼ focus areas
The second axis is the KOI, as defined by three focus areas: While each KOI focus area will typically work with and around data, the tasks performed and skill requirements for each focus area tend to differ substantially. As shown in ►Fig. 3, this axis was loosely based on Rogers' diffusion of innovations model 18 in that the target audience of something entirely new (I) may require a different approach than something that targets the majority of users (O). Some tasks, such as with K, may not require end-user adoption at all (e.g., observational research and data mining).
►Fig. 3 shows the high-level relationship between each focus area. By adding the internal, external, and policy practice settings to each focus area (►Fig. 2), we now have a model that describes what each person does (Axis 2) and where they do it (Axis 1). Note that each informatician will likely have their own unique snapshot, and this snapshot may change over the span of their career. Additionally, a single project may progress from one focus area to another over time or may overlap different areas simultaneously. Finally, it is presumed that topics arising from the K focus area will often help define or guide topics in the O or I areas.
Below, we provide a description of each KOI focus area, followed by example snapshots of an informatician who prioritizes that focus area and a basic pitch that describes them. By no means is this meant to be an exhaustive list. Rather, it should serve as an example of how to adapt the KOI pond (snapshot and pitch) to match the informatician. While each example listed below is fictional by design, existing job descriptions and real-world career choices were considered heavily.

Knowledge: Defines the Care
The Knowledge (K) focus area obtains and analyzes healthcare data in a manner that adds value to the overall understanding of a topic or condition. Using the ACGME definition of CIS listed above, 3 K focuses most strongly on "analyzing" and "evaluating" systems. As a rule of thumb, if it requires approval from an institutional review board, chances are it is deeply rooted in K. Quality improvement and operational metrics are also a part of K, though work within these projects will typically span multiple focus areas. Insights from K focus area activities may expand beyond a single health system in the form of publications (K-external), which in turn may help guide evidence-based medicine and standards of care (K-policy). ►Table 1 shows an example of an academic informatician.
Pitch 1: (academic informatician) Harnessing informatics to pursue K is where I excel. I have an R01 grant for my Fig. 3 High-level relationship among KOI pond focus areas. High-level relationship between each KOI pond focus area in a manner that also considers Rogers' diffusion of innovation curve. 18 Note that a given project may overlap multiple focus areas at once. It may also shift focus areas over time.
ACI Open Vol. 4 No. 1/2020 research on (core content topic here), and I am core faculty for our CI fellowship training program. My research is in the midst of a multicenter trial, which we hope will positively impact clinical practice. We have established a national committee to define and guide our standard of care in this area.

Operations: Implements the Care
Again considering the ACGME's definition of CIS, 3 the Operations (O) focus area ties most closely to "implementing" systems. This is in large part through leading and managing change, effective communication, and team dynamics. What makes the O focus area distinct is its targeted end-user group. If a population's majority-as defined by Rogers' diffusion of innovations model 18 -serves as the primary target, then the O focus area is likely dominant. Navigating requests for proposals and large-scale vendor implementations are typically considered O-external. Those who manage organizational policies, such as data governance, would consider this work under O-policy. ►Table 2 shows an example snapshot of a CMIO/CCIO. Pitch 2: (CMIO/CCIO) I want our clinicians to be happy today, and thus my role focuses on keeping O streamlined. We work very closely with (vendor here) to ensure electronic orders are high yield and easy to navigate. We have also sought to standardize clinical workflows across the various care areas I manage. I just started meeting with (consultant/ vendor/startup here) who I found via a standard request for proposal process to make (workflow issue here) easier. We are currently planning a waterfall implementation of their product at our institution.

Innovation: Disrupts the Care
As with informatics, the concept of innovation is somewhat vague and often overused in a manner that promotes confusion. [19][20][21] In a 2009 literature review, Baregheh found 60 different definitions of innovation published across disciplines. 22 Within healthcare, some of the most important characteristics of innovation are: "(1) novelty, (2) an application component, and (3) an intended benefit," noting that the application component can be "new services, new ways of working, and/or new technologies." 23,24 For our purposes, the Innovation (I) focus area should match the ACGME's definition of CIS 3 about "designing" systems. If new ideas and/or intellectual property are generated that previously did not exist or if something is so new that it only targets the innovator/early adopter section of Rogers' diffusion of innovations model, 18 it is likely an I focus area. This may occur as a physical device, a digital product, or an abstract idea. Examples of I-internal activities include in-house development, process redesign, and the newer concept of "intrapreneurship." [25][26][27] Someone working as an entrepreneur, as a consultant/vendor, or as a collaborator for a project that spans multiple institutions (e.g., opensource development) would often consider their work I-external. Those who are willing to undertake risks to create something disruptive and positively impactful may consider themselves intrapreneurs or entrepreneurs, depending on the context in which they work. ►Table 3 provides a KOI pond snapshot of an entrepreneur who focuses on new and upcoming healthcare technologies.
Pitch 3: (Healthcare technology entrepreneur) I am most interested in how we can leverage cutting-edge technology into our healthcare system today. While I still practice clinically part time, I spend a lot of my efforts at a mid-sized startup that integrates (concepts from other fields here) and applies them to healthcare. I also serve as the liaison between my startup and its client healthcare organizations through which I help test and implement design concepts into practice.

Applications
Much like KOI, the practice of CI comes in many different shapes and colors. This model can be applied by an informatician to concisely describe how they invest their own time and resources. Note that each table above shows a snapshot of an informatician in a manner that is mostly agnostic to CI core content. Take, for example, a single core content area: cybersecurity. Within cybersecurity alone, one can conduct research (K), implement measures (O), or create entirely new  security measures (I). These cybersecurity efforts can also be deployed across the full spectrum of informatics settings, such as within a hospital organization (internal), through a consulting firm managing a cybersecurity portfolio (external), or in a manner that manages national/organizational standards for cybersecurity (policy). By combining both axes, we arrive at a simple yet effective way to describe the work of an informatician. This same structure can be used by an individual describing their current position (or desired position) or by a healthcare organization interested in creating new informatics positions for employment. The K focus area can be combined with any of the settings listed above. A hospital organization's K-internal combination would likely reflect an academic informatician at the start of their career or one who gathers metrics for operational/quality use. If that informatician focuses more on grant funding, publications/conferences, or multicenter trials, then he/she would now consider much of his/her work K-external. By comparison, an informatician who is interested in national standards (K-policy) may do this by serving on an advisory board. ►Table 4 provides an example snapshot of an informatician who focuses on K-policy.
Pitch 4: (Advisory board informatician) As my primary specialty continues to grow, it has been my passion to ensure that we expand in a healthy and meaningful way. I serve on the (council/board of directors/advisory board) of my specialty's nonprofit (association/college). Though my role entails several tasks, my primary work is to ensure the quality and rigor of our national data registry. This registry serves an important role in our specialty's reimbursement and in large-scale academic research.
The O focus area may be similarly stratified across the care settings, such as the CMIO and other healthcare leadership positions (internal), consultation assistance for key organiza-tion informatics areas (external) or organizational/national guidelines for hospital system use of informatics assets (policy). Note additionally that healthcare leadership positions are not strictly limited to O focus areas. The chief innovation officer (CINO), 28 chief digital officer (CDO), 29 and chief strategy officer (CSO) 30,31 are I-focused career options those with CIS training may consider, though the exact roles and tasks should be contextualized to match the organization's needs and priorities. 32 As with the CMIO/CCIO path, these roles are not mutually exclusive to CIS diplomates. ►Tables 5 and 6 provide KOI pond snapshots of an implementation consultant and a CINO, respectively.
Pitch 5: (Implementation consultant) For each practice setting and care area, clinical processes tend to vary significantly. Our company works with large health organizations to help them understand their workflows and opportunities for improvement. After deriving current state workflows and metrics, we host kaizen and brainstorming sessions with the clinical staff. We help them build from their suggestions and navigate the path of change management. My knowledge of how clinical data ties into workflow is critical for our company's ability to understand and gather accurate and reliable metrics for our clients. The ability to distinguish practice variation/patient variety from inefficiencies/edge outliers is where I apply informatics.
Pitch 6: (CINO) If we want true impactful change, then we have to be forward-thinking. My team regularly hosts sessions where we teach employees at our health system how to apply (design thinking, design sprints, agile, and lean) methodologies in their departments. We also work with (startup/accelerator/ ecosystem here) to see which products may integrate best into our health system and to pursue opportunities for collaboration.   While each snapshot may be unique to the individual, that snapshot may change or narrow as one's career path matures. Additionally, the 0 to 5 investment of resources scale used in our examples was derived using subjective assessments of fictional positions. It is possible to use objective data, such as time investments or full-time equivalents (FTEs) to help standardize this scale. However, objective data alone may not fully capture a person's activities and interests. While CI does not have relative value unit assignments like other medical specialties, qualitative estimates of effort, such as those provided in the tables above, may be useful as a whole.
Qualitative and quantitative data could also be combined to show a snapshot's maturity over time, to compare current and ideal states, or to provide a basic job description that includes skills and interests with FTE breakdown. Since one cannot bill patients directly for informatics work, these snapshots may also assist CI diplomates who work on a departmental level (e.g., housed in their primary specialty). Given the paths to financial support of CI activities may vary significantly based on focus area, it may be helpful to see how other CI diplomates with similar KOI snapshots and departmental scopes were able to fund their work.

Discussion
CIS is still a very young subspecialty. From October 2014 to March 2019, the number of nationwide ACGME-accredited CI fellowship programs increased by a factor of eight (from 4 to 33 programs). 33 With many programs accepting an average of two or more fellows per year, it is likely that each fellow will pursue a different career path than their predecessors from the same program. As such, the ability for informaticians to define their practice succinctly will be important regardless of where they are working and to whom they are reporting. For those in CI, KOI pond snapshots can serve as the lexicon we use to describe our roles and responsibilities to each other and to our employers.
While core content material should be considered heavily when deciding career interests, it is also important to consider what tasks are done (K-O-I) and the setting in which they are performed (internal, external, and policy). For CIS trainees planning their intended career path, it may be helpful to look at focus areas first, then consider core content areas that are most appealing and use this information to target job positions that can support said interests. One may also consider job options within his/her primary specialty, harnessing CI in a manner that focuses more on the department than the organization.
Although we believe this model provides a comprehensive way to describe the various pathways informaticians may follow, it does have several limitations. First, this construct may still fail to fully encapsulate every setting and focus area experienced in CI practice, especially as the subspecialty continues to mature. Conversely, in an attempt to be all-inclusive, our loosely defined construct may not be specific enough and therefore may fail to communicate the nuanced differences between some CI settings and focus areas. This may in-turn lead to inconsistencies and subjectivity among where each informatician defines their own tasks and interests within the KOI pond.
Additionally, while the method chosen for model generation (iterative focus group) has its own inherent limitations, 16,34 the inclusion of CI fellows from various programs, interests, and primary specialties should help mitigate these limitations. Iterative focus groups have been used successfully in model generation 35 and healthcare end-user prototyping, 36 and to some extent our chosen method draws parallels to user-based usability methods 37 and collaborative participatory design. 38 Nevertheless, our focus group mainly included CI fellows at the start of their informatics careers. As such, this model should be juxtaposed with the domains, tasks, and KS statements from Silverman's 2019 DoP analysis, in which respondents were more broadly distributed across age groups (35- 14 While outside the scope of this paper, a more rigorous comparison of our KOI pond and the initial core content areas 8 with the recent DoP analysis may add value to our subspecialty as we continue to define and refine the breadth of CI practice.

Conclusion
Informaticians are unique in their diversity, their multidisciplinary nature, and their adaptability to many settings. We have been called the subspecialty that "transforms healthcare," 3 but there are many paths and directions we can take to achieve this. Rather, our greatest connection is our willingness to work with and around data to achieve said goals. Physicians from any specialty of medicine may pursue CIS training, and they may undertake projects unrelated to their home specialty. As CIS continues to grow, KOI pond snapshots and pitches may help provide concrete examples of the various paths other CI diplomates have taken in a way that provides value to those just learning about CI. By creating a common lexicon for describing our activities and skills, we promote our strengths (diversity, adaptability, and problem-solving) to physicians who are considering CIS, to colleagues in other specialties, to employers, and to ourselves.