CC BY 4.0 · ACI open 2023; 07(02): e79-e86
DOI: 10.1055/s-0043-1775971
Case Report

Curriculum for Early Exposure to Clinical Informatics and Data Science for Noninformatics Trainees to Promote Interest and Inclusion in Informatics

Akshay Ravi
1   Department of Medicine, University of California, San Francisco, San Francisco, California, United States
,
Benjamin Weia
1   Department of Medicine, University of California, San Francisco, San Francisco, California, United States
,
Matthew Sakumoto
1   Department of Medicine, University of California, San Francisco, San Francisco, California, United States
2   Department of Medicine, Sutter Health, San Francisco, California, United States
,
Aris Oates
3   Department of Pediatrics, University of California, San Francisco, San Francisco, California, United States
,
Xinran Liu
1   Department of Medicine, University of California, San Francisco, San Francisco, California, United States
› Institutsangaben
Funding None declared.
 

Abstract

Background Curricula aimed at increasing exposure to informatics and practical data analytics among medical trainees could increase their effectiveness in clinical research, quality improvement, and clinical operations.

Objectives The Clinical Informatics Data Science (CI-DS) pathway is a cross-disciplinary curriculum aimed at improving informatics exposure among medical trainees. We describe the development of this novel curriculum, the inaugural cohort, and lessons learned.

Methods The CI-DS pathway is framed around upfront informatics didactics followed by a longitudinal, experiential training focused on mentorship, clinical data extraction/machine learning, and health technology governance. The curriculum was evaluated based on pre- and postpathway surveys completed by learners and logs of the elective activities selected by learners.

Results The CI-DS pathway attracted 19 learners across 12 medical subspecialties, from medical students to fellows. Baseline surveys showed limited exposure to informatics across learners. The top three longitudinal activities completed were participating in electronic health record (EHR) governance meetings, data science supplemental courses, and designated mentorship meetings. Comparison of baseline with postpathway surveys demonstrated significant improvements in learner self-reported confidence in appraising an EHR modification ticket, accessing UCSF's deidentified data, exploring a database with basic structured query language (SQL), extracting data using SQL, and interpreting machine learning models.

Conclusion An early exposure curriculum in clinical informatics with training in data extraction and governance can successfully recruit a diverse array of learners and improve confidence in practical informatics skills. We reflect on the strengths and weaknesses of this curriculum, and summarize the lessons learned to guide others in creating similar curricula for noninformatics clinicians.


#

Background and Significance

With health information systems becoming integral to health care delivery, there is an increasing need for clinical informatics to be a part of medical education.[1] [2] [3] Advanced training programs such as clinical informatics fellowships,[4] [5] master's degree programs,[6] and certificate programs exist but are geared for those seeking careers in informatics. However, courses aimed at engaging medical trainees with unexplored interests in clinical informatics are still early in development. These types of curricula may serve two purposes: one, to introduce the breadth of the field of informatics to trainees and encourage them to pursue advanced training and two, to provide those who may not be interested in advanced training with the practical skills and informatics proficiencies needed for noninformaticists.

Several institutions have implemented elective clinical informatics curriculums at varying levels of training. At the undergraduate medical education (UME) and graduate medical education (GME) level, these courses typically incorporate structured didactics in the form of synchronous or asynchronous lectures/case studies and projects oriented around the build and implementation of new electronic health record (EHR) functionality.[6] [7] [8] [9] [10] These curricula vary in format and duration, ranging from 2 to 4 weeks' electives to multimonth longitudinal programs. However, across all of these curricula there have been no learning activities oriented toward EHR data extraction and data science competencies besides predictive analytics.[6] [9] [10]

While these EHR-focused curricula may be a useful for aspiring informaticists, they may not be ideal for noninformatics clinicians. For these physicians, skills in practical data analytics are vital in the era of the modern EHR because of the centrality of EHR data in several key nonclinical roles for physicians including clinical research,[11] quality improvement,[12] and clinical operations. Many scholarly and operational endeavors of noninformaticists would benefit from practical skills of how to query, interpret, use, and understand the limitations of EHR data.


#

Objectives

We describe the development and delivery of a Clinical Informatics-Data Science (CI-DS) curriculum aimed at UME and GME learners that incorporates practical data science competencies and reflect on lessons learned from the inaugural cohort.


#

Methods

The CI-DS curriculum was tailored for a tertiary, multisite academic medical center with a clinical informatics fellowship, but no other informatics curricula at the institution. The curriculum was developed based on the Kern six-step framework[13] with three major needs identified through observations and informal discussions: (1) appeal of the curriculum among informaticists and noninformaticists; (2) hands-on experiential training that enabled them to access and extract real-world data from EHR databases; and (3) accessible mentorship to promote diversity in levels of training and professional discipline.

We refined and specified competencies that would have broad utility for both informatics- and noninformatics-bound trainees based on a list of knowledge and skills compiled from surveying clinical informatics-certified physicians.[3] Existing clinical informatics curricula for UME and GME learners were referenced for the competencies focused on health information systems, while we created original learning activities for the data science competencies.[1] [10] These competencies are described in [Table 1].

Table 1

Distilled clinical informatics competencies for Clinical Informatics Data Science pathway

Domain[a]

Tasks/knowledge and skill area identifier[a]

Description

Fundamental knowledge and skills

K001

The discipline of informatics (e.g., definitions, history, careers, professional organizations)

K005

Health information technology (HIT) principles and science

K006

Computer programming fundamentals and computational thinking

K007

Basic systems and network architectures

K008

Basic database structure, data retrieval, and analytics techniques and tools

K025

The flow of data, information, and knowledge within the health system

Improving care delivery and outcomes

2.01

Develop, implement, evaluate, monitor, and maintain clinical decision support (CDS), in alignment with the Five Rights of CDS (information, person, intervention formats, channel, and point/time in workflow).

K027

CDS standards and processes for development, implementation, evaluation, and maintenance

K028

Five Rights of CDS (i.e., information, person, intervention formats, channel, and point/time in workflow)

K039

Predictive analytic techniques, indications, and limitations

K049

Prediction models

Enterprise information systems

K054

Institutional governance of clinical information systems

K060

Enterprise architecture (databases, storage, application, interface engine)

Data governance and data analytics

4.06

Create, enhance, and/or leverage information architecture to efficiently store, query, and retrieve data from large datasets

4.07

Interpret information and apply knowledge gained from datasets using descriptive, diagnostic, predictive, and prescriptive analytic approaches to derive actionable insights

K086

Stewardship of data

K088

Metadata and data dictionaries

K090

Transactional and reporting/research databases

K093

Data associated with workflow processes and clinical context

K094

Data management and validation techniques

K099

Query tools and techniques

K103

Advanced modeling and algorithms

K104

Artificial intelligence

K105

Machine learning (e.g., neural networks, support vector machines, Bayesian network)

K107

Natural language processing

Leadership and professionalism

K127

Theory and methods for promoting the adoption and effective use of clinical information systems

a Domains and tasks/knowledge and skill area identifiers from Silverman et al.


The overarching educational strategy was one initial week of structured didactics followed by a longitudinal program spanning 5 to 9 months ([Fig. 1]). This structure fit within the framework of an existing GME pathways program, which is a multidisciplinary program of supplementary curricula aimed at building specialized knowledge and skills.[14] Enrollment in the CI-DS pathway was open to medical students, residents, and fellows to afford broad accessibility of the curriculum.

Zoom Image
Fig. 1 Schema depicting core elements of CI-DS pathway. All learners participate in 1 week of core didactics. Seminar-only learners stop here, but longitudinal learners continue in the pathway for 5 to 6 months. This followed by two required activities: an electronic health record governance activity, and introductory data science exercises. Learners are then required to complete 10 to 15 other elective activities driven by personalized interests. Finally, mentorship is incorporated throughout each phase of the curriculum. CI-DS, Clinical Informatics Data Science.

Based on our targeted needs assessment, the structured didactics portion of the curriculum were designed to consolidate foundational concepts in informatics for learners across levels of training and experience in informatics to establish a baseline for all trainees.[13] Didactics consisted of a combination of readings, lectures, workshops, and panels facilitated by practicing clinical informaticians at the University of California, San Francisco (UCSF). Topics ranged from the Impact of Informatics on the Field of Medicine to Electronic Health Records and Clinical Decision Support to panels on training in clinical informatics and industry as well as introductory lectures on data science concepts ([Table 2]).

Table 2

Seminar lectures, foundational activities, and elective activities in the Clinical Informatics Data Science pathway

Areas of pathway

Activity title

Seminar series

What is Clinical Informatics: the Faces of Informatics

History of Clinical Informatics

Impact of Informatics on the Field of Medicine, Present and Future

Electronic Health Records, Clinical Decision Support, App Ecosystem at UCSF

Center for Digital Health Innovation and Telehealth Resource Center at UCSF

Center for Clinical Informatics Improvement Research at Bakar Computational Health Sciences Institute at UCSF

Panel on Informatics at UCSF

What is Data Science and what does EHR Data Look Like?

How is EHR Data Stored, Processed, and Accessed through SQL

Defining Cohorts in Data, Challenges and Best Practices

Information Commons Presentation and Access

Introduction to Machine Learning

Introduction to Natural Language Processing

Implementing Machine Learning in Clinical Practices and Bias

Machine Learning PICO

Clinical Informatics Fellowship Training

Panel on Clinical Informatics Fellowship

Panel on Industry Informatics

Longitudinal pathway: foundational activities

Attend Governance Committee Meeting & Complete Associated Reflections

Complete SQL data science and machine learning exercises

Write Independent SLQ Query for Research or Quality Improvement Project

Attend Monthly Mentorship Check-in meetings

Longitudinal pathway: elective activities

Harvard Clinical Informatics Lectures

Clinical Informatics Case Conference

Submit EHR modification ticket and complete reflection

Informatics Grand Rounds

Attend national informatics conference meetings: AMIA, AMDIS, HIMSS, BAMIS symposium

Interview Informatics Faculty Member

Listen to ACIF “Go Live” Podcast and Complete Reflection

Listen to “Informatics in the Round” Podcast and Complete Reflection

UCSF Library Data Science Initiative Course

Berkeley D-Lab Courses

Stanford Human-Centered AI Seminars

UCSF Data Colloquium/Research Data Series

UCSF LinkedIn Learning Courses

Attend/Participate in Bakar Computational Health Sciences Institute Lab Meeting

Attend and Participate in Information Commons User Group Meeting

Submit Informatics Related Abstract/Poster/Manuscript

Present Informatics Related Project at Informatics Work in Progress Session

Abbreviation: SQL, structured query language.


The CI-DS pathway's longitudinal component was developed using behaviorist and cognitivist theories of education emphasizing two primary loci for learning: skill acquisition through practice and real-world experience paired with self-reflection.[15] Activities were categorized as “Clinical Informatics” or “Data Science.” “Clinical Informatics” emphasized learning and understanding the workings of EHR design and governance. Learners were encouraged to attend EHR governance meetings and capstone the experience by shepherding an EHR modification through the process. “Data Science” offered activities designed by faculty and clinical informatics fellows that walked learners through gaining access to and directly querying from a deidentified clinical data warehouse at UCSF. We created self-guided tutorials for learners to perform structured query language (SQL) tasks and extract clinical data. Learners were also encouraged to define a question related to an ongoing or new quality improvement or research project that they could answer using the clinical data warehouse. The full list of lectures and activities is shown in [Table 2].

The CI-DS pathway was advertised to learners through emails by UCSF GME and online information. Learners completed baseline surveys about their knowledge in clinical informatics and exposure to informatics mentorship prior to start of the curriculum. During the longitudinal pathway, learners submitted informal reflections for each foundational or elective activity completed. After pathway completion, longitudinal learners were asked to complete a postsurvey to evaluate for changes in knowledge and confidence. Survey data were compared using a Fisher's exact test given the small sample size. All learner surveys and responses were collected through Qualtrics and data were analyzed using R, version 4.1.2.


#

Results

The inaugural cohort of the CI-DS pathway included 19 learners, 11 of whom completed the longitudinal curriculum. Among the 19 learners, there was enrollment from fourth year medical student to clinical fellows ([Table 3]), gender parity, with approximately 47% women enrolled ([Table 3]), and representation from 12 subspecialties of medicine.

Table 3

Demographic details of pathway learners

Feature

Number of learners (n = 19)

Gender identity[a]

 Male

10 (52.6%)

 Female

9 (47.4%)

Level of training

 MS4

4 (21.1%)

 PGY1

3 (15.8%)

 PGY2

1 (5.3%)

 PGY3

3 (15.8%)

 PGY4

3 (15.8%)

 Fellow

5 (26.3%)

a No learners identified as nonbinary.


Baseline survey response rate was 95% ([Table 4]). Learners reported limited knowledge and exposure to EHR governance, data querying and extraction, and machine learning prior to enrollment in the pathway.

Table 4

Learner self-assessment of skills and interest in informatics before and after Clinical Informatics Data Science pathway

Question

Pre (n = 18)

Post (n = 5)

Not confident

Neutral

Confident

Not confident

Neutral

Confident

p-Value

Appraising an EHR modification ticket

17 (94.4%)

0 (0.0%)

1 (5.6%)

2 (40.0%)

2 (40.0%)

1 (20.0%)

0.02[a]

Submitting an EHR modification ticket

15 (83.3%)

2 (11.1%)

1 (5.6%)

2 (40.0%)

1 (20.0%)

2 (40.0%)

0.09

Accessing UCSF de-identified data

15 (83.3%)

0 (0.0%)

3 (16.7%)

0 (0.0%)

3 (60.0%)

2 (40.0%)

<0.01[b]

Exploring a database with basic SQL

13 (72.2%)

2 (11.1%)

3 (16.7%)

1 (20.0%)

3 (60.0%)

1 (20.0%)

0.04[a]

Pulling deidentified data using SQL to answer a research/QI questions

15 (83.3%)

0 (0.0%)

3 (16.7%)

2 (40.0%)

2 (40.0%)

1 (20.0%)

0.04[a]

Interpreting the results of machine learning models

12 (66.7%)

1 (5.6%)

5 (27.8%)

0 (0.0%)

2 (40.0%)

3 (60.0%)

0.01[a]

Using machine learning on a dataset

11 (61.1%)

3 (16.7%)

4 (22.2%)

2 (40.0%)

1 (20.0%)

2 (40.0%)

0.8

Yes

Yes

Currently working on informatics project

8 (44.4%)

4 (80.0%)

0.2

Pre (n = 17)

Post (n = 5)

Unlikely

Neutral

Likely

Unlikely

Neutral

Likely

Interest in clinical informatics fellowship training

7 (41.2%)

5 (29.4%)

5 (29.4%)

1 (20.0%)

0 (0.0%)

4 (80.0%)

0.2

Abbreviations: EHR, electronic health record; SQL, structured query language; QI, quality improvement.


Note: p-Value computed using Fisher's exact test.


a p < 0.05.


b p < 0.01.


Postpathway surveys were completed by 5 out of 11 learners (45%; [Table 4]). Learners reported a shift in their confidence about all activities ranging from EHR governance to data querying and extraction to machine learning. Specifically, there was a significant difference in confidence in appraising an EHR modification ticket, accessing UCSF's de-identified data, exploring a database with basic SQL, extracting data using SQL, and interpreting machine learning models. There was no statistically significant difference in the percentage of learners working on an informatics project before and after the pathway, nor a difference in interest in clinical informatics fellowship training. In unstructured comments, learners reported a strong interest in learning more about querying data through SQL and applying machine learning to datasets.

Finally, the elective activities completed by longitudinal learners is summarized in [Table 5]. The top three activities completed by learners were the EHR governance committee activity, the UCSF Library Data Science Initiative course (synchronous virtual workshops offered quarterly on topics ranging from SQL, python, R, natural language processing, and machine learning), and monthly mentorship meetings.

Table 5

Longitudinal pathway learner reported elective activity completion

Area of pathway

Category of activity

Number of activities completed

Foundational activities

Attend governance committee meeting and complete associated pre-post reflections

6

Complete SQL data science and machine learning exercises

0

Write independent SQL query for research or quality improvement project

0

Attend monthly mentorship check-in meetings

4

Elective activities

Harvard clinical informatics lectures

0

Clinical informatics case conference

2

Submit EHR modification ticket and complete reflection

2

Informatics grand rounds

1

Attend national informatics conference meetings: AMIA, AMDIS, HIMSS, BAMIS symposium

1

Interview informatics faculty member

0

Listen to ACIF “Go Live” podcast and complete reflection

1

Listen to “Informatics in the Round” podcast and complete reflection

0

UCSF Library Data Science Initiative Course

5

Berkeley D-Lab Courses

0

Stanford Human-Centered AI Seminars

0

UCSF Data Colloquium/Research Data Series

0

UCSF LinkedIn Learning Courses

0

Attend and participate in Bakar Computational Health Sciences Institute Lab Meeting

0

Attend and participate in information commons user group meeting

1

Submit informatics related abstract/poster/manuscript

1

Present informatics related project at informatics work in progress session

1

Other custom activities

Extramural informatics project work

2

Abbreviation: SQL, structured query language.



#

Discussion

We describe the development of a new informatics curriculum for noninformatics clinicians at a tertiary care academic medical center as well as lessons learned from its first cohort. Unique elements of the curriculum include a strong emphasis on practical skills tailored to physicians in clinical data extraction/modeling from the EHR, availability of the curriculum across disciplines and UME and GME levels, and longitudinal mentorship. The CI-DS pathway attracted an initial cohort diverse in training level, specialty, and gender. We reflect on the strengths and weaknesses of this curriculum and summarize the lessons learned to continue improving the CI-DS pathway and to guide others in creating similar clinical informatics and data science curricula for noninformatics clinicians.

First, we found that learners reported a significant difference in their ability to interpret the results of machine learning models but did not show a significant difference in using machine learning on a clinical dataset. This is supported by elective activity completion logs, which show that learners completed supplemental data science online didactics but did not complete the SQL and machine learning exercises. This is notable given that end of pathway learners reported an interest in learning more about the data science and machine learning curricula. While these survey data are limited by response rate and sampling bias, these differences may still be useful in deriving insight into the best ways to teach similar curricula to physicians. When designing the data science exercises, we followed a flipped classroom approach,[16] by allowing independent, adult learners to review and complete the activity at their own pace, with several interactive, explanatory examples for learners to work with in the deidentified data warehouse. However, this structure may not have been adequate for medical trainees who may not have a computer science background. Accompanying didactics or other asynchronous audiovisual material could provide the framing and structure needed for future interactive data science curricula aimed at clinicians. This is further supported by the popularity of the UCSF Library Data Science Initiative courses, which allow for learners to enroll in additional didactic material about specific data science skills. Independent learning activities like these, which rely on a flipped classroom-esque homework, may also benefit from small group discussion or office hour sessions to consolidate knowledge, address questions, and clarify the material.

While EHR ticket appraisal and submission may be more useful for informaticists than noninformatics clinicians, having insight, context, and expectations around the governance process may still be a useful skill for these learners. Among these skills, learners reported a significant difference in skill with appraising an EHR modification ticket but showed no significant difference in ability to submit their own ticket. This is further supported by elective activity completion logs which confirm high rates of governance committee meeting attendance, but limited EHR modification ticket submissions. The reason for this discrepancy may be due to the availability and frequency of EHR governance committee meetings compared with the number of opportunities for learners to submit a novel EHR modification request. Others considering similar curricula may consider inverting the concept of this idea: instead of submitting a new modification request, learners can apply their governance knowledge and skills to remove unnecessary or harmful elements in the EHR akin to a “Clickbusters” program.[17] Such programs could allow noninformaticists to leverage their clinical domain expertise and to better understand the importance of the governance process in vetting future requests.

We also noted attrition from seminar learners (n = 19) to longitudinal learners (n = 11). As others have found that protecting research time for trainees may be correlated with academic output,[18] [19] one potential hypothesis for this discrepancy is that our trainees, who did not have protected time for this elective curriculum may have been limited in their ability to participate because of their clinical workload. Although some residency programs afford flexibility through X + Y systems, in which residents switch between 4 weeks of an inpatient service followed by 2 to 4 weeks of an elective rotation with flexibility for protected academic time, this may not be true for all residencies or fellowships, particularly procedural subspecialties.[20] To reach these learners, others designing similar curricula for noninformaticists may need to target advertisement and outreach to medical students, residents, and fellows who have a period of protected time in their training. For example, fourth year medical students or third year general surgery or cardiology learners starting their research years. More broadly, this issue will require demonstrating the curriculum's value to and coordinating with these clinical departments to facilitate participation.

Our study has several limitations. First, the small initial cohort limits comparative power. Second, given limited response rate of the postpathway survey (95% response rate before and 45% response rate after), there may be sampling bias from respondents. Future studies could include reevaluating the CI-DS pathway or similar curricula after further iteration as well as a follow-up of learners' ultimate career decision and involvement in informatics in the future.


#

Conclusion

A novel clinical informatics and data science pathway with experiential training in SQL for clinical data extraction, machine learning models, and EHR governance aimed at noninformatics learners across levels of training and professional discipline can successfully recruit a diverse array of learners. Similar curricula may be important for both training a modern workforce of clinical informatics leaders and informatics-fluent noninformaticists.


#

Clinical Relevance Statement

Our study describes the development and implementation of a targeted informatics curricula for noninformaticists. Key aspects of this curricula include focused core didactics, experiential training in SQL for clinical data extraction, machine learning models, and EHR governance for noninformaticists. This curriculum can attract a diverse group of learners, although important lessons to consider for future curricula include generating further buy-in from clinical departments for additional learner time and adding more structure for aspects of the curriculum less familiar to clinicians like computer science or health IT governance.


#
#

Conflict of Interest

None declared.

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles for Medical Research Involving Human Subjects and was reviewed and exempted by the UCSF Institutional Review Board.


  • References

  • 1 Hersh WR, Gorman PN, Biagioli FE, Mohan V, Gold JA, Mejicano GC. Beyond information retrieval and electronic health record use: competencies in clinical informatics for medical education. Adv Med Educ Pract 2014; 5: 205-212
  • 2 Fridsma DB. Health informatics: a required skill for 21st century clinicians. BMJ 2018; 362: k3043
  • 3 Silverman H, Cohen T, Fridsma D. The evolution of a novel biomedical informatics curriculum for medical students. Acad Med 2012; 87 (01) 84-90
  • 4 Gardner RM, Friedman NA, Carlson M, Bradham TS, Barrett TW. Impact of revised triage to improve throughput in an ED with limited traditional fast track population. Am J Emerg Med 2018; 36 (01) 124-127
  • 5 Longhurst CA, Pageler NM, Palma JP. et al. Early experiences of accredited clinical informatics fellowships. J Am Med Inform Assoc 2016; 23 (04) 829-834
  • 6 Hare AJ, Soegaard Ballester JM, Gabriel PE, Adusumalli S, Hanson III CW. Training digital natives to transform healthcare: a 5-tiered approach for integrating clinical informatics into undergraduate medical education. J Am Med Inform Assoc 2022; 30 (01) 139-143
  • 7 Breil B, Fritz F, Thiemann V, Dugas M. Multidisciplinary education in medical informatics – a course for medical and informatics students. Stud Health Technol Inform 2010; 160 (Pt 1): 581-584
  • 8 Mai MV, Luo BT, Orenstein EW, Luberti AA. A model for clinical informatics education for residents: addressing an unmet need. Appl Clin Inform 2018; 9 (02) 261-267
  • 9 Turer RW, Arribas M, Balgord SM. et al. Clinical informatics training during emergency medicine residency: the University of Michigan experience. AEM Educ Train 2020; 5 (03) e10518
  • 10 Singer JS, Cheng EM, Baldwin K, Pfeffer MA. UCLA Health Physician Informaticist Committee. The UCLA Health Resident Informaticist Program - a novel clinical informatics training program. J Am Med Inform Assoc 2017; 24 (04) 832-840
  • 11 Andriole DA, Wolfson RK. The physician-investigator workforce: looking ahead. Acad Med 2021; 96 (04) 486-489
  • 12 Deilkås ET, Rosta J, Baathe F. et al. Physician participation in quality improvement work- interest and opportunity: a cross-sectional survey. BMC Prim Care 2022; 23 (01) 267
  • 13 Thomas PA, Kern DE, Hughes MT, Chen BY. Curriculum Development for Medical Education: A Six-Step Approach. 3rd ed.. Johns Hopkins University Press; 2016. . Accessed May 2, 2022 at: https://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=979875&site=ehost-live
  • 14 Kohlwes RJ, Shunk RL, Avins A, Garber J, Bent S, Shlipak MG. The PRIME curriculum. Clinical research training during residency. J Gen Intern Med 2006; 21 (05) 506-509
  • 15 Torre DM, Daley BJ, Sebastian JL, Elnicki DM. Overview of current learning theories for medical educators. Am J Med 2006; 119 (10) 903-907
  • 16 Hew KF, Lo CK. Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Med Educ 2018; 18 (01) 38
  • 17 McCoy AB, Russo EM, Johnson KB. et al. Clinician collaboration to improve clinical decision support: the Clickbusters initiative. J Am Med Inform Assoc 2022; 29 (06) 1050-1059
  • 18 Onyekaba G, Mauch JT, Barrero C. et al. Does dedicated research time during residency predict resident academic productivity and a future academic career?. Ann Plast Surg 2022; 88 (03, Suppl 3): S293-S295
  • 19 Williams BR, Agel JA, Van Heest AE. Protected time for research during orthopaedic residency correlates with an increased number of resident publications. J Bone Joint Surg Am 2017; 99 (13) e73
  • 20 Shalaby M, Yaich S, Donnelly J, Chippendale R, DeOliveira MC, Noronha C. X + Y scheduling models for internal medicine residency programs-a look back and a look forward. J Grad Med Educ 2014; 6 (04) 639-642

Address for correspondence

Akshay Ravi, MD
Department of Medicine, University of California
San Francisco, 521 Parnassus Avenue, Box 0131, San Francisco, CA 94143
United States   

Publikationsverlauf

Eingereicht: 30. März 2023

Angenommen: 16. August 2023

Artikel online veröffentlicht:
01. November 2023

© 2023. The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution License, permitting unrestricted use, distribution, and reproduction so long as the original work is properly cited. (https://creativecommons.org/licenses/by/4.0/)

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

  • References

  • 1 Hersh WR, Gorman PN, Biagioli FE, Mohan V, Gold JA, Mejicano GC. Beyond information retrieval and electronic health record use: competencies in clinical informatics for medical education. Adv Med Educ Pract 2014; 5: 205-212
  • 2 Fridsma DB. Health informatics: a required skill for 21st century clinicians. BMJ 2018; 362: k3043
  • 3 Silverman H, Cohen T, Fridsma D. The evolution of a novel biomedical informatics curriculum for medical students. Acad Med 2012; 87 (01) 84-90
  • 4 Gardner RM, Friedman NA, Carlson M, Bradham TS, Barrett TW. Impact of revised triage to improve throughput in an ED with limited traditional fast track population. Am J Emerg Med 2018; 36 (01) 124-127
  • 5 Longhurst CA, Pageler NM, Palma JP. et al. Early experiences of accredited clinical informatics fellowships. J Am Med Inform Assoc 2016; 23 (04) 829-834
  • 6 Hare AJ, Soegaard Ballester JM, Gabriel PE, Adusumalli S, Hanson III CW. Training digital natives to transform healthcare: a 5-tiered approach for integrating clinical informatics into undergraduate medical education. J Am Med Inform Assoc 2022; 30 (01) 139-143
  • 7 Breil B, Fritz F, Thiemann V, Dugas M. Multidisciplinary education in medical informatics – a course for medical and informatics students. Stud Health Technol Inform 2010; 160 (Pt 1): 581-584
  • 8 Mai MV, Luo BT, Orenstein EW, Luberti AA. A model for clinical informatics education for residents: addressing an unmet need. Appl Clin Inform 2018; 9 (02) 261-267
  • 9 Turer RW, Arribas M, Balgord SM. et al. Clinical informatics training during emergency medicine residency: the University of Michigan experience. AEM Educ Train 2020; 5 (03) e10518
  • 10 Singer JS, Cheng EM, Baldwin K, Pfeffer MA. UCLA Health Physician Informaticist Committee. The UCLA Health Resident Informaticist Program - a novel clinical informatics training program. J Am Med Inform Assoc 2017; 24 (04) 832-840
  • 11 Andriole DA, Wolfson RK. The physician-investigator workforce: looking ahead. Acad Med 2021; 96 (04) 486-489
  • 12 Deilkås ET, Rosta J, Baathe F. et al. Physician participation in quality improvement work- interest and opportunity: a cross-sectional survey. BMC Prim Care 2022; 23 (01) 267
  • 13 Thomas PA, Kern DE, Hughes MT, Chen BY. Curriculum Development for Medical Education: A Six-Step Approach. 3rd ed.. Johns Hopkins University Press; 2016. . Accessed May 2, 2022 at: https://search.ebscohost.com/login.aspx?direct=true&db=nlebk&AN=979875&site=ehost-live
  • 14 Kohlwes RJ, Shunk RL, Avins A, Garber J, Bent S, Shlipak MG. The PRIME curriculum. Clinical research training during residency. J Gen Intern Med 2006; 21 (05) 506-509
  • 15 Torre DM, Daley BJ, Sebastian JL, Elnicki DM. Overview of current learning theories for medical educators. Am J Med 2006; 119 (10) 903-907
  • 16 Hew KF, Lo CK. Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Med Educ 2018; 18 (01) 38
  • 17 McCoy AB, Russo EM, Johnson KB. et al. Clinician collaboration to improve clinical decision support: the Clickbusters initiative. J Am Med Inform Assoc 2022; 29 (06) 1050-1059
  • 18 Onyekaba G, Mauch JT, Barrero C. et al. Does dedicated research time during residency predict resident academic productivity and a future academic career?. Ann Plast Surg 2022; 88 (03, Suppl 3): S293-S295
  • 19 Williams BR, Agel JA, Van Heest AE. Protected time for research during orthopaedic residency correlates with an increased number of resident publications. J Bone Joint Surg Am 2017; 99 (13) e73
  • 20 Shalaby M, Yaich S, Donnelly J, Chippendale R, DeOliveira MC, Noronha C. X + Y scheduling models for internal medicine residency programs-a look back and a look forward. J Grad Med Educ 2014; 6 (04) 639-642

Zoom Image
Fig. 1 Schema depicting core elements of CI-DS pathway. All learners participate in 1 week of core didactics. Seminar-only learners stop here, but longitudinal learners continue in the pathway for 5 to 6 months. This followed by two required activities: an electronic health record governance activity, and introductory data science exercises. Learners are then required to complete 10 to 15 other elective activities driven by personalized interests. Finally, mentorship is incorporated throughout each phase of the curriculum. CI-DS, Clinical Informatics Data Science.