A Standard Approach to Project-Based Learning in a Clinical Informatics Fellowship

Abstract

Background The Accreditation Council for Graduate Medical Education suggests that Clinical Informatics (CI) fellowship programs foster broad skills, which include collaboration and project management. However, they do not dictate how to best accomplish these learning objectives.

Objectives This study aimed to describe a standard approach to project-based learning for CI, to share its implementation, and to discuss lessons learned.

Methods We created a standard approach to project-based learning based on concepts from adult learning theory, the project life cycle framework, the Toyota Production System, and Improvement Science.

Results With this standard approach in place, we learned how best to support fellows in its use. In addition to this approach to supporting needs assessment, risk/change management, implementation, and evaluation/improvement skills, we found the need to develop fellow skills in collaboration, leadership, and time management/managing up. Supported by project-based learning using this standard approach, and with targeted project selection to meet topic-based learning objectives, fellows reached the ability to practice independently in 15 to 21 months.

Discussion Fellows are uniquely positioned to ensure the success of projects due to their increased availability and protected time compared with attendings. They are readily available for project teams to draw upon their expertise with clinical workflows and understanding of technological solutions. Project-based learning addressing organizational priorities complements fellow project management coursework and improves fellows' ability to function successfully in large, complex, and dynamic organizations. Exposing fellows to contemporary problems, then addressing them through projects, provides fellows with up-to-date applied informatics knowledge.

Conclusion Project-based learning can ensure that many general CI learning objectives are supported inherently. It reinforces project management teachings, while providing fellows with a marketable project portfolio to aid with future job applications. Having projects tightly aligned with organizational priorities supports ongoing investment in fellowship programs.

Keywords

medical informatics - medical education - clinical informatics

Background and Significance

Since envisioned,[1] Clinical Informatics (CI) fellowships were accredited by the Accreditation Council for Graduate Medical Education (ACGME) in 2014[2] and there are now 60.[3] CI training relies on a mix of didactic education and experiential learning. CI Program Requirements[4] and Milestones[5] include expectations for educational programs, with content related to managing projects, collaboration, and other skills implying a need for project-based learning ([Supplementary Appendix A]>, available in online version only). Learning objectives are accomplished via an “appropriate blend of supervised patient care (or for CI, applied informatics) responsibilities, clinical (informatics) teaching, and didactic educational events.”[6]

The University of Washington (UW) CI Fellowship became accredited in 2016. Four regular-track fellows are funded by the UW Medicine Information Technology (IT) Services (UWM ITS) and one pediatrics-track fellow by Seattle Children's Hospital IT. Fellows spend 36 hours/week on 1 of 4 core rotations (inpatient informatics, ambulatory informatics, priority projects/elective, and enterprise informatics/analytics/security). Each rotation is 3 months long and repeated the second year. Each core rotation has a primary preceptor (faculty attending), who facilitates and assesses fellow growth.

Experience-based learning is common in clinical teaching[7] and newer in quality improvement.[8] While publications describe CI curricula[9] [10] [11] [12] [13] [14] [15] and use of project-based learning,[16] [17] [18] [19] [20] [21] [22] we believe this is the first description of how to implement project-based learning to satisfy CI training objectives. The approach we describe is complemented by fellow didactics[23] and collaboration on larger projects with UWM ITS's project management office (PMO).

Objective

To describe a standardized process for CI project-based learning, to share its implementation, and to discuss lessons learned.

Educational Methods

Standard Approach: Overview

Our project approach mirrors a traditional project life cycle framework,[24] with added steps for project identification, project assignment, and dissemination.

The full approach described below applies to projects with significant unknowns or a large scope (more than 2 months). Smaller projects may employ selected steps ([Fig. 1]). Formal IT project intake and management processes are used in parallel. These complementary approaches augment fellows' Project Management didactics[23] with real-world examples.

Fig. 1 Standard Process for Clinical Informatics projects important to the Health System. CCIO, Chief Clinical Information Officer; CMIO, Chief Medical Information Officer. * = Step is optional, usually depending on the size of the project.

Project Identification, Feasibility Assessment, and Assignment

Fellow projects are identified using both “top-down” and “bottom-up” approaches. “Top-down” recommendations come from informatics leaders (e.g., Chief Information Officers or Chief Clinical/Medical Information Officers [CCIO/CMIOs]), IT governance committees, and IT project requests (e.g., new pandemic, safety incident). Rotation preceptors also suggest ongoing projects that would benefit from fellow input or offer tailored fellow education. “Bottom-up” projects emerge from clinicians through faculty and fellow clinical use of the electronic health record (EHR), interactions with colleagues, and rounding. These project ideas often focus more on user experience and efficiency and offer a narrower but more deliverable scope.

Assessment of Health System Partners

Focusing on projects of importance to UWM is key to demonstrating value and creating opportunities to practice informatics and leadership skills. Fellows begin by identifying all partners and influential stakeholders at project onset. Understanding the needs of partners is important for defining project success and developing ongoing business relationships. This also reduces the risk of unexpected changes in direction, scope, and timeline while identifying potential avenues for escalation and assistance. Also, as an educational tool, this allows fellows to understand team structures and their areas of oversight, allowing for efficient decision-making and change management.

For some complex projects, fellows place project stakeholders into an Influence-Involvement (Power-Interest) Grid to organize stakeholder relationship management[25] [26] ([Fig. 2]), complementing the PMO's Responsibility Assignment Matrix.[24] High-influence high-involvement stakeholders (e.g., leaders of affected areas) require consistent engagement and inclusion in all key decisions. High-influence and low-involvement stakeholders require less frequent updates yet may participate in key decisions.

Fig. 2 Sample Influence-Involvement (Power-Interest) Grid. IT, information technology.

Feasibility

The preliminary project approach is determined together with stakeholders and IT analysts, examining both development costs and workflow changes. For large projects, numerous discussions may be required to determine deliverables and estimated effort, but feasibility is not typically in question for smaller projects (i.e., creating a new order set).

Assignment

The rotation preceptor assigns projects to foster fellow education and alignment with organizational priorities ([Table 1]). From the identified projects believed to have high value, we also emphasize projects with strong potential for experiential learning and/or alignment with specific interests or expertise of individual fellows (e.g., analytics, social determinants of health, artificial intelligence, clinical decision support). This ensures a balanced and engaging project portfolio. Analogous to clinical rotations, where providers see whichever patient is in front of them (as opposed to choosing which patients to treat), fellows are given the expectation that their rotation project portfolio interdigitates institutional needs with their educational goals and interests as part of their progress toward milestones. To create a highly marketable portfolio of project experience and expertise, we aim for 2 to 5 projects per fellow per quarter, to ensure 15 to 20 projects throughout their fellowship.

Table 1 Characteristics of good projects for experiential learning

Can be accomplished (sociopolitically and technically feasible)

Important to the organization

Meets learning objectives from Accreditation Council for Graduate Medical Education (ACGME) and/or certifying boards  ● Program requirements, milestones  ● Certification exam content description

Impact can be measured, stated concisely, and can serve as the basis for further improvement

Where possible:  ● Important to the learner  ● Tailored to learner's educational needs

Design

IT requests are often underspecified (e.g., “Need order set for hyperkalemia”) and requestors' proposed solutions (i.e., “Build an alert”) may not fully consider the possible solutions or constraints. Taking a balanced approach, fellows must first understand the root problem and clinical workflow before constructing appropriate technology solutions with project requestors. Fellows first observe current processes to better understand what constitutes success for all stakeholders, particularly clinical teams and patients/families. This foundational prework informs future state design and measures.

Informatics is an integrative field that benefits from multiple disciplines. Learning how to respectfully communicate through discussions with stakeholders, and then eventually lead the discussions, is unique to CI. Faculty can help to hone fellow proficiency with engaging stakeholders and end-users through modeling, direct observation, and feedback. Participating in multiple projects and rotations with core faculty allows reliable assessment and development of communication skills to accurately capture stakeholder needs.

Current State Mapping

Prior to proposing or implementing EHR modifications, fellows are expected to develop a deep understanding of the sociotechnological interplay between clinical workflows and electronic systems. To promote this understanding, fellows “go to gemba (actual place—where work is done)” to directly observe work processes, as inspired by the Toyota Production System.[27] Fellows observe clinicians and staff in real time as they use computer systems to provide care. This grounding helps to avoid the pitfalls of designing changes for “work as imagined” rather than “work as done,” helping to build positive relationships with end-users while capturing their insights.

Depending on project complexity, fellows may create a workflow process map to inform design changes or may simply return with notes and a mental model of how EHR interfaces both support and impede efficient care.

Environmental Scan/(Systematic) Literature Review

For large projects, risk is addressed by learning what is already known about the problem that a project intends to solve. Simple first steps include searching for published articles in PubMed with the assistance of a clinical librarian, searching for talks and discussion in the EHR vendor database and customer e-mail listserv, and discussing the project with faculty and regional collaborators. If national performance benchmarks are publicly available, fellows are encouraged to identify comparable organizations that perform well to inform focused outreach and/or collaboration with those leaders. With faculty, fellows then generate proposed future state alternatives.

Risk/Change Management Analysis

Understanding change management, identifying scope, and mitigating project risks, including scope creep, are critical skills for informaticians. Right-sizing a project is one of the first skills fellows learn. Based on background work, fellows plan several solutions to solve project problems and work with faculty to identify the most favorable solutions given time and resource constraints. For new initiatives, project risk is formally assessed, from organizational buy-in to technological challenges, and contingency planning occurs at the outset and on an ongoing basis. For large projects, if the expected change is too significant, smaller changes are considered first. A phased approach may be chosen (rolling out some features followed subsequently by larger features) based on the “minimum viable product” idea.[28] As a part of this approach, fellows weigh constraints and obstacles to identify which aspects of a project are safe to implement soon versus aspects that pose the highest risk for dissatisfaction or failure if implemented too early.

Change management plans also must address how the intervention will be maintained beyond the life of the project. As key stakeholders are identified, fellows work with faculty to determine how best to engage them throughout the project and beyond, referencing the project's Power-Interest grid. As a project nears completion, the level of engagement increases to include development of training materials and communication planning. Ultimately, the project is handed off to operational leadership for ongoing change management.

Fellows plan for change management by applying two tools adapted from the Institute for Healthcare Improvement Model For Improvement: Specific, Measurable, Achievable, Relevant, Time-bound (SMART) aim statements and Key Driver Diagrams (KDD).[29] The acronym SMART summarizes the essential dimensions of what the project seeks to accomplish. KDDs depict how possible interventions could result in improvement and can mitigate project risk. Fellows are encouraged to develop KDDs with a mutually exclusive and collectively exhaustive list of possible EHR and workflow changes. This approach discourages jumping to familiar solutions, instead encouraging creative solutions. Fellows discuss the aim statements and KDDs with supervising faculty and project stakeholders, to support future change management and learning. When applicable, fellows and their faculty preceptors discuss legal, regulatory, confidentiality/security, ethical, and/or equity-related considerations for the project and schedule conversations with leaders in these areas.

Projected Impact Analysis/Measurement Plan

For all projects, fellows are encouraged to determine a proposed future state and explore how the change provided by the project results in measurable impact. In addition to the SMART aims, fellows are encouraged to focus on changes in clinical outcomes for patients, user satisfaction, and usability. If an environmental scan/literature review was done, this may inform measures. More recently, we have focused on additional measures related to health care equity. Time to create measurement tools and dashboards to effectively visualize relevant outcomes is incorporated into the project plan.

Build and Test

In this phase, fellows develop and refine technical skills, align with project deadlines, and work closely with other informatics specialists assigned to a given project. The informatics specialists range from clinical informaticists who specialize in understanding clinical workflows across the organization, technical analysts experienced in building and testing clinical applications, and trainers with advanced skills in adult education and communication. Larger projects are managed by dedicated ITS project managers; small projects may be managed by informatics specialists with project management tools. Proposed solutions are tested in a nonproduction domain and reviewed with stakeholders to ensure that the solution can be operationalized and that limitations are understood.

Fellows meet regularly with stakeholders and often take leadership roles for the duration of their participation in a given project. Stakeholders include: clinicians, IT specialists, and when assigned, project management. On more complex projects, fellows provide crucial coordination between ITS and operational groups to make sure that technical requirements are addressed, and there is operational readiness for project go-live. They develop go-live communications, identify stakeholder groups to be included, create training materials, and route communications via a variety of channels to maximize impact.

Fellows also meet weekly with their primary rotation preceptor and separately with the program director. These are opportunities to troubleshoot project challenges, consider what additional resources might be needed, and to 'upport fellows' well-being. The program director monitors workload and duty-hour commitments, as fellows are often balancing demands from several projects. There are also regular meetings with the CCIO/CMIO and his/her team of informatics physicians, providing an opportunity to ensure alignment with organizational priorities, receive guidance, and celebrate successes with a community of peer informaticians. There have been times when organizational urgencies have necessitated redirecting IT energies, such as organizational response to the coronavirus disease 2019 crisis.[30] These sorts of rapid responses have provided a unique challenge and learning opportunity for our fellows, who are instrumental in the ability of IT to provide high-quality solutions to meet time-sensitive organizational needs.

Fellows can choose to keep or to hand off larger projects at the rotation end. The sign-out process is described in [Supplementary Appendix B]> (available in the online version).

Deployment and Project Close

A successful and complete implementation is a mark of success for a clinical informatician. Faculty continuously reinforce the need to put solutions into practice, with successful rollouts serving as a basis for improvements. Fellows support go-lives through good communication with stakeholders when planning go-live dates, executing the announcement strategy, and providing/teaching to implementation tools (identified during change management planning). Fellows are available to answer technical and workflow questions for go-lives.

Our faculty are physician IT leaders with a vested interest in evaluation who work with fellows to evaluate clinical, efficiency, and well-being-related outcomes. These measures are broad and may include effects on decision-making and clinical outcomes, process measures like adoption rates, and balancing measures monitoring for unintended consequences. Measures are commonly trended over time in run charts to evaluate outcomes for significance. Since projects typically address organizational priorities, fellows get exposure to measures of importance to the health system, some of which are reported to external entities like Leapfrog or accrediting bodies while others relating to business targets are reported to payors to inform risk-based contracts. Fellows commonly update organizational sponsors and informatics leaders, which provides experience in measuring and communicating success.

Following deployment, fellows are encouraged to formally conclude the project with the project team. The process varies with the size and nature of the project. It may include a “postmortem” analysis of the project experience, an assessment of whether the final product reached its SMART aims, and a determination of whether IT or other stakeholders have ongoing maintenance tasks. After this, fellows sign off and encourage operational and IT stakeholders to reengage as needed.

Dissemination

Project findings are discussed at least quarterly (at supervised fellow sign-out) and at closing with the CCIO/CMIO group. Fellows are also encouraged to present their work at informatics grand rounds, virtual case conferences, and in the community (e.g., allied health departments and local vendors). The statewide CMIO consortium also encourages fellow updates.

Fellows work with faculty to determine if intermediate and final work products may merit further dissemination. Writing a manuscript is suggested when (1) fellows have performed systematic reviews and no recent systematic reviews on a topic are found, (2) quality improvement projects with KDD and run charts show improved outcomes, or (3) the topic is emergent and other organizations may benefit from guidance. For smaller efforts, fellows are encouraged to submit abstracts. Fellow travel is reimbursed if presenting informatics work at conferences.

Educational Results

Based on the Program Requirements and Milestones, there are several specific experiences that fellows must demonstrate and faculty must evaluate. [Supplementary Appendix A]> (available in the online version) shows how using this standard project-based learning process helps to address numerous subcompetencies and milestones. This is facilitated by the cultivation of a learning environment that provides graduated supervision leading to independence. Graduated supervision is determined by individual faculty and supported by semiannual Clinical Competency Committee evaluation.

After putting the Standard Process in place, we learned how best to support fellows in its use. In addition to supporting needs assessment, risk/change management, implementation, and evaluation/improvement skills as above, we also supported skills development in collaboration, leadership, and time management/managing up. The general educational outcomes from project-based learning serve as a foundation to support more topic-focused learning outcomes (e.g., Clinical Decision Support expertise).

We started project-based learning with our first cohort of fellows in 2016. Since 2021, we have tracked fellow projects more closely. Fellows typically contribute to 40 to 60 informatics projects, with larger projects typically involving multiple fellows. Supported by project-based learning using this standard approach, and with targeted project selection to meet topic-based learning objectives, fellows reached the ability to practice independently in 15 to 21 months as assessed by rotation preceptors who meet quarterly ([Supplementary Appendix B]>, available in the online version), and who are all part of our Clinical Competency Committee.

Collaboration

Interdisciplinary projects require fellows to develop real-world skills in collaborating with various stakeholders—from clinician colleagues to analysts and administrators. The ability to network and constructively partner with appropriate subject matter experts is developed over time.

Many fellows do not have prior experience working with health system colleagues outside of patient care contexts (e.g., IT analysts, health information management, compliance, billing). These colleagues may have little or no clinical training but have vast domain expertise. Fellows practice under close supervision of faculty preceptors as they develop these new relationships. Prior fellow experience with hierarchical structures in medicine may color these new collaborations and fellows may become disoriented when acclimating to the power dynamics of a newly formed project team. This is an ideal time for faculty to teach fellows about the dynamics of group formation[31] (thereby providing a vocabulary to discuss these challenges), coach them on how to communicate these challenges to leadership and offer guidance for determining when “storming” within a team is normal versus a problem.

Leadership

Before fellowship, fellows have experience leading clinical teams as senior residents or as clinical faculty. However, few trainees have led significant change initiatives within an organization. Selecting high-priority projects that offer faculty the ability to model leadership skills and fellows to participate in experiential learning has allowed fellows to take graduated leadership roles commensurate with their experience and skill set.

With the project-based curriculum, preceptors start fellows with fundamental tasks to engender trust from analysts and health system collaborators through quick wins, gradually transitioning fellows to lead design and implementation discussions. This provides experience with the fundamental role of physician interpreter—to be able to explain the latest technological advancements to clinicians while simultaneously explaining clinical workflows, requirements, and constraints to analysts. Over time, as leadership and technical skills mature, faculty encourage fellows to take increasingly independent roles in projects with decreasing amounts of hands-on daily supervision. By the end of fellowship, fellows are capable of independently running both small and large projects.

Time Management/Managing Up

One of the strengths of project-based learning is that it affords increasing levels of independence for fellows to participate and lead work efforts. Since the pace of work varies by project, fellows often have multiple projects simultaneously with different implementation timelines and amounts of change. However, with the culture of medicine having providers work until the work is completed, new fellows may at times become overloaded and overworked. Fellows may also be afraid to report when things are not going well, presenting a rosy picture until failure is imminent. These behaviors can be problematic in different ways and present an opportunity for fellows to learn how to “manage up.”

With the Quadruple Aim,[32] provider work–life balance is a newer consideration. Faculty can help balance fellow project workload by setting reasonable expectations, helping them to prioritize, and modeling how to communicate with leadership and collaborators regarding their capacity. They can encourage honest communication and provide a supportive environment of safety where fellows can understand how and when to ask for help, which ensures that projects remain on track.

Discussion

The approach described in this paper is intentionally designed to simultaneously deliver and demonstrate value to the organization for their fellowship investment while providing hands-on experience and education for the fellows. Project work complements fellow project management coursework and UWM ITS project intake, scoring, prioritization, and management structures.

To achieve these goals, we leveraged departmental expertise in adult learning theory and CI to create a fellowship where fellows learn theory through didactics then see the theory in practice through project-based learning. While implementing project-based learning supports progress toward ACGME requirements,[4] [5] [33] doing so in the context of actual operational projects has the additional benefits of training fellows to work effectively within IT departments, balancing many projects at different stages of the project lifecycle, and building their capability and confidence toward independent practice. Use of contemporary projects allows the curriculum to remain relevant in the quickly evolving field of informatics and having fellows work on many important projects has been an asset to them when interviewing for jobs or during subsequent employment.

Fellows value doing work that directly affects their colleagues, with observable outcomes (positive or otherwise). Completing projects reinforces their role in the IT team and highlights the impact that they can make both on patient care and on customer well-being/efficiency. Working on prioritized projects also helps develop their comfort with contemporary, mission-driven technologies.

To ensure continued financial support for the fellowship (including salaries and Master's degree tuition), it has been vital to demonstrate clear return on investment to the organization. This return has been through ensuring success of mission critical projects, augmented by rigorous evaluation for projected and actual project impacts. Quarterly and annual reporting of the aggregate fellow project portfolio and the impact of these projects to the organization at various levels has been vital to ongoing and expanding support for the fellowship. This may represent a path toward sustainable funding for CI fellowship programs.[34] [35]

Fellows bring a combination of clinical expertise and both formal didactic and experiential informatics knowledge that grows throughout the 2-year fellowship. By being 90% dedicated to operational CI work, they are more available than other clinicians to UWM ITS project teams for project-related work and support, giving them a unique opportunity to practice with increasing autonomy. Lastly, the shift in culture that an informatics fellowship brings cannot be overstated. The presence of fellows encourages a culture of education, innovation, and creativity within an IT department, providing fresh energy and ideas to overcome organizational inertia and resistance to change.

Limitations

Local cultural and administrative factors may make implementing the Standard Process more challenging elsewhere.

For example, programs funded by clinical departments may be less influenced by changes in IT department leadership, financial results, and priorities. Additionally, as a health system spanning multiple hospitals and clinics, there are a plethora of projects underway that fellows can join.

Our organization invested in improving IT performance by embedding academic informatics faculty into operational service roles, often with significant leadership responsibilities ([Supplementary Appendix C], available in the online version). Furthermore, faculty are dedicated to identifying and facilitating project experiences for fellows and supporting their participation in a way that is high touch with personalized one-on-one supervision. Our block schedule is designed so that there is a fellow on each core rotation at all times.

Guiding fellows in our experiential learning model requires robust faculty development related to leading and teaching quality improvement. To prepare for this role, they typically have expertise through CI board certification, which is supplemented by knowledge from UW's participation in the Centers for Medicare and Medicaid Services' Transforming Clinical Practice Initiative.[36] Further, some faculty have attended Ohno Leadership Training in Japan,[37] and our core faculty includes a director of a Quality Improvement methods certificate program.

Overall, our health system has resources such as a clinical librarian to support literature searches with a culture of ample learning opportunities provided by the Graduate Medical Education office for all trainees. Lastly, the CI Program and the Fellowship Director have funding available to support fellow conference travel and unforeseen or cutting-edge needs.

Future Directions

We have identified four areas for potential improvement to the Standard Process. First, project activities could be reported in ways that support automated tracking of educational milestones. We envision future systems that use artificial intelligence to examine project work products, continuously monitoring progress toward mastery of selected entrustable activities. Second, we hope to develop better measurements of the long-term financial and clinical impact of projects, lighting the path toward functioning as a learning health care system. Third, we seek to reliably measure health care equity effects and plan to incorporate equity impact analysis in all fellow projects. Lastly, we believe we can apply simulation science preimplementation to better anticipate deployment challenges. There may also be opportunities to improve the sign-out process and handoff tool.

Conclusion

Project-based learning is an approach that can ensure that noncontent-based CI educational requirements and milestones are supported inherently. From an educational perspective, it reinforces project management didactics, while providing fellows with a marketable portfolio of projects. From an enterprise perspective, it is key that these projects are tightly aligned with organizational priorities to demonstrate value to the organization of their ongoing investment.

Clinical Relevance Statement

Project-based experiential learning is an effective way to train clinical informatics fellows.

Multiple-Choice Questions

1. A first-year Clinical Informatics fellow was assigned to an ongoing project to evaluate the efficacy of a best practice advisory in their outpatient primary care clinic. One of the team members is protective of their expertise and resistant to giving some of the work on the project to a fellow. Additionally, they are not including the fellow in relevant emails or meeting invites. Which of Tuckman's Stages of Team Development best describes the situation?

   • Forming

   • Storming

   • Norming

   • Performing

   • Adjourning

   Correct Answer: The correct answer is option b. Tuckman's Stages of Team Development has five categories: forming, storming, norming, performing, and adjourning. Forming is the initial stage where the team members define their purpose and goals. Storming is when tension appears in conflicts and disagreements as team members establish their roles. Norming is when these conflicts are resolved and norms are established with a cohesive identity. Performing is when the team achieves efficiency and productivity toward their common goal. Adjourning is the stage of disbanding the team as the task has been achieved. Storming is most consistent with the described scenario.

2. A clinical informatics fellow has been working on building note templates for the inpatient medicine teams throughout a health care system. They measured a high adoption rate among providers for their template and also received quarterly feedback that there is dissatisfaction with the time it takes to write notes. The fellow organizes a presentation of the process of building the note and the feedback they have received since its go-live. The project is submitted to a national conference where it is accepted for presentation. Which phase of the project impact analysis best describes the current stage of the project?

   • Build and test

   • Deployment and project close

   • Dissemination

   • Literature review

   Correct Answer: The correct answer is option c. In the build and test phase, fellows collaborate with other informatics specialists and regularly meet with stakeholders to prototype and iterate on potential solutions. In the deployment phase, fellows support go-live dates and create implementation tools. In the dissemination phase, project findings are reviewed and fellows are encouraged to present their progress and findings nationally. Literature review is where a fellow investigates the current body of knowledge on a topic and is typically completed before the start of the project.

Conflict of Interest

None declared.

Acknowledgments

We would like to thank the former Dean of the School of Medicine, Dr. Paul Ramsey, for providing initial funding for our fellowship program; Dr. Thomas Norris for providing a departmental home for our fellowship program; and Dr. Zafar Chaudry and Seattle Children's leadership for providing additional funding. We thank Eric Neil, Dr. Todd Burstain, Dr. Tim Dellit, Dr. Anneliese Schleyer, Wendy Giles, Joy Grosser, Dr. Jim Fine, Dr. Dave Chou, Dr. Sundeep Desai, Dr. Byron Joyner and the UW and Seattle Children's Hospital Graduate Medical Education offices, and the many UW Medicine and Seattle Children's faculty, informaticians, and staff that have devoted their time to teaching and supporting our fellowship program. We thank all of our Program Administrators over the years for their help with supporting the fellows, faculty, and program over the years—Jen Albrecht, Shawn Banta, Lora Brewsaugh, Karyn Crow, Marla Hill, Heidi Krueger, Kayleen McGinley, and last but not least, Taraneh Moll. We also thank our prior and current faculty and prior and current fellows for their patience and understanding as we developed these teaching methods.

Protection of Human and Animal Subjects

This manuscript does not involve Human or Animal Subjects research.

• References

• 1 Safran C, Shabot MM, Munger BS. et al; AMIA Board of Directors. Program requirements for fellowship education in the subspecialty of clinical informatics. J Am Med Inform Assoc 2009; 16 (02) 158-166 Erratum in: J Am Med Inform Assoc. 2009;16(4):605
• 2 Longhurst CA, Pageler NM, Palma JP. et al. Early experiences of accredited clinical informatics fellowships. J Am Med Inform Assoc 2016; 23 (04) 829-834
• 3 ACGME. Advanced Program Search. Accessed January 18, 2024 at: https://apps.acgme.org/ads/Public/Programs/Search
• 4 ACGME. ACGME Program Requirements for Graduate Medical Education in Clinical Informatics. Accessed February 3, 2024 at: https://www.acgme.org/globalassets/pfassets/programrequirements/381_clinicalinformatics_2023.pdf
• 5 Baker C, Elkin PL, Lehmann CU. et al. Clinical Informatics Milestones. Accessed February 3, 2024 at: clinicalinformaticsmilestones.pdf (acgme.org).
• 6 ACGME. ACGME Common Program Requirements Section VI with Background and Intent. Accessed February 21, 2024 at: https://www.acgme.org/globalassets/PFAssets/ProgramRequirements/CPRs_Section-VI_with-Background-and-Intent_2017-01.pdf
• 7 Goldman J, Kuper A, Baker GR. et al. Experiential learning in project-based quality improvement education: questioning assumptions and identifying future directions. Acad Med 2020; 95 (11) 1745-1754
• 8 Goodman CW, Justo J, Merrow C, Prest P, Ramsey E, Ray D. An experiential learning collaborative on quality improvement for interprofessional learners. J Interprof Care 2022; 36 (02) 327-330
• 9 Patton GA, Gardner RM. Medical informatics education: the University of Utah experience. J Am Med Inform Assoc 1999; 6 (06) 457-465
• 10 Gardner RM, Overhage JM, Steen EB. et al; AMIA Board of Directors. Core content for the subspecialty of clinical informatics. J Am Med Inform Assoc 2009; 16 (02) 153-157
• 11 Mantas J, Ammenwerth E, Demiris G. et al; IMIA Recommendations on Education Task Force. Recommendations of the International Medical Informatics Association (IMIA) on education in biomedical and health informatics. Methods Inf Med 2010; 49 (02) 105-120
• 12 Silverman H, Lehmann CU, Munger B. Milestones: critical elements in clinical informatics fellowship programs. Appl Clin Inform 2016; 7 (01) 177-190
• 13 Ahonen O, Kinnunen U-M, Lejonqvist G-B, Apkalna B, Viitkar K, Saranto K. Identifying biomedical and health informatics competencies in higher education curricula. Stud Health Technol Inform 2018; 251: 261-264
• 14 Silverman HD, Steen EB, Carpenito JN, Ondrula CJ, Williamson JJ, Fridsma DB. Domains, tasks, and knowledge for clinical informatics subspecialty practice: results of a practice analysis. J Am Med Inform Assoc 2019; 26 (07) 586-593
• 15 Mize DE, Osterman TJ. A unified approach to clinical informatics education for undergraduate and graduate medical education. AMIA Annu Symp Proc 2023; 2022: 766-774
• 16 Moehr JR, Berenji GR, Green CJ, Kagolovsky Y. Project-based teaching in health informatics: a course on health care quality improvement. Stud Health Technol Inform 2001; 84 (Pt 2): 1061-1065
• 17 Vitek CR, Dale JC, Homburger HA, Bryant SC, Saenger AK, Karon BS. Development and initial validation of a project-based rubric to assess the systems-based practice competency of residents in the clinical chemistry rotation of a pathology residency. Arch Pathol Lab Med 2014; 138 (06) 809-813
• 18 Crabtree EA, Brennan E, Davis A, Squires JE. Connecting education to quality: engaging medical students in the development of evidence-based clinical decision support tools. Acad Med 2017; 92 (01) 83-86
• 19 Singer JS, Cheng EM, Baldwin K, Pfeffer MA. UCLA Health Physician Informaticist Committee. The UCLA Health Resident Informaticist Program - a novel clinical informatics training program. J Am Med Inform Assoc 2017; 24 (04) 832-840
• 20 Mai MV, Luo BT, Orenstein EW, Luberti AA. A model for clinical informatics education for residents: addressing an unmet need. Appl Clin Inform 2018; 9 (02) 261-267
• 21 Turer RW, Arribas M, Balgord SM. et al. Clinical informatics training during emergency medicine residency: the University of Michigan experience. AEM Educ Train 2020; 5 (03) e10518
• 22 Reeves JJ, Longhurst CA, Mekeel KL. et al. A novel fellowship in perioperative administration, quality and informatics: a pilot experience for training surgical leaders. J Surg Educ 2022; 79 (04) 839-844
• 23 University of Washington. Online Master of Science in Clinical Informatics & Patient Centered Technologies. Accessed February 21, 2024 at: https://www.clinical-informatics.uw.edu/
• 24 Project Management Institute. A Guide to the Project Management Body of Knowledge. 7th ed. Newton Square, PA: Project Management Institute; 2021: 42-43 : 189
• 25 Bernstein SL, Weiss J, Curry L. Visualizing implementation: contextual and organizational support mapping of stakeholders (COSMOS). Implement Sci Commun 2020; 1: 48
• 26 Mendelow AL. Stakeholder mapping. Proceedings from the Second International Conference on Information Systems; 1991. Cambridge, MA: : ICIS 1981 Proceedings 1981; 20
• 27 Wellman J, Hagan P, Jeffries H, Bailey C. Leading the Lean Healthcare Journey: Driving Culture Change to Increase Value. 2nd ed. Boca Raton, FL: CRC Press; 2017: 277
• 28 SyncDev. MVP - Minimum viable product. Accessed February 5, 2024 at: https://web.archive.org/web/20160525101214/http://www.syncdev.com:80/minimum-viable-product
• 29 Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey-Bass Publishers; 2009: 90-119
• 30 Grange ES, Neil EJ, Stoffel M. et al. Responding to COVID-19: the UW Medicine information technology services experience. Appl Clin Inform 2020; 11 (02) 265-275
• 31 Tuckman BW. Developmental sequence in small groups. Psychol Bull 1965; 63 (06) 384-399
• 32 Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med 2014; 12 (06) 573-576
• 33 Lingham V, Chandwarkar A, Miller M. et al. A systematic approach to the design and implementation of clinical informatics fellowship programs. Appl Clin Inform 2023; 14 (05) 951-960
• 34 Kannry J, Smith J, Mohan V, Levy B, Finnell J, Lehmann CU. Clinical Informatics Program Directors Group-AMIA. Policy statement on clinical informatics fellowships and the future of informatics-driven medicine. Appl Clin Inform 2020; 11 (05) 710-713
• 35 Patel TN, Chaise AJ, Hanna JJ. et al. Structure and funding of clinical informatics fellowships: a national survey of program directors. Appl Clin Inform 2024; 15 (01) 155-163
• 36 Centers for Medicare & Medicaid Services. Transforming clinical practice initiative. Accessed February 15, 2024 at: https://www.cms.gov/priorities/innovation/innovation-models/transforming-clinical-practices
• 37 Enna. Ohno leadership training in Japan. Accessed January 22, 2024 at: https://japantrip.enna.com/
• 38 Starmer AJ, Spector ND, O'Toole JK. et al; I-PASS SHM Mentored Implementation Study Group. Implementation of the I-PASS handoff program in diverse clinical environments: a multicenter prospective effectiveness implementation study. J Hosp Med 2023; 18 (01) 5-14

Publication History

Received: 15 March 2024

Accepted: 21 July 2024

Article published online:
09 October 2024