Competency-Based Teaching in Radiology – Implementation and Evaluation of Interactive Workstation-Based Learning to Apply NKLM-Based Content

Abstract

Purpose New teaching formats are required to implement competency-based teaching in radiology teaching. Therefore, we have established and evaluated two practical competency-based radiological courses.

Materials and Methods The courses were held in a multimedia room with 25 computers and a professional DICOM viewer. Students were taught basic image analysis and presented clinical cases with a DICOM viewer under supervision of an instructor using desktop monitoring software. Two courses (elective course and obligatory course) were evaluated by the students (n = 160 and n = 100) and instructors (n = 9) using an anonymized online survey.

Results Courses were evaluated positively by the students and instructors. From the perspective of the students, the courses increased understanding of cross-sectional anatomy (elective/obligatory course: 97 %/95 %) and radiologic findings (97 %/99 %). Furthermore, the course increased the students’ interest in radiology (61 %/65 %). The students considered this way of teaching to be relevant to their future occupation (92 % of students in the obligatory course). The higher incidence of teacher-student interaction and the possibility of independent image analysis were rated positively. The majority of instructors did not observe increased distractibility due to the computers (67 %) or notice worse preparation for MC tests (56 %). However, 56 % of instructors reported greater preparation effort.

Conclusion Practical competency-based radiological teaching using a DICOM viewer is a feasible innovative approach with high acceptance among students and instructors. It fosters competency-based learning as proposed by the model curriculum of the German Radiological Society (DRG) and the National Competency-based Catalogue of Learning Objectives for Undergraduate Medical Education (NKLM).

Key Points 

• Practical competency-based radiological teaching is highly accepted by students and instructors.

• Students report improved understanding of imaging anatomy and radiological findings.

• Interactive case presentation with a DICOM viewer fosters competency-based learning.

Citation Format

• Koestner W, Otten W, Kaireit T et al. Competency-Based Teaching in Radiology – Implementation and Evaluation of Interactive Workstation-Based Learning to Apply NKLM-Based Content. Fortschr Röntgenstr 2017; 189: 1076 – 1085

Key words

medical education - radiological teaching - curriculum - competency-based learning - national competency-based catalogue of learning objectives for undergraduate medical education - DICOM viewer

Introduction

Medical education has changed greatly in the last century. After implementation of the basic scientific subjects in the medical curricula at the start of the 20th century, problem-oriented learning and integrative teaching in which the separation between basic subjects and clinical subjects was largely eliminated began in the middle of the 20th century [1]. The most recent reform focuses on competency-based learning. The National Competency-Based Catalogue of Learning Objectives for Undergraduate Medical Education published in June 2015 defines different competencies to be taught to medical students [2]. According to Weinert, competencies are "cognitive skills that are possessed or can be learned to resolve certain problems and the associated motivational, volitional, and social readiness and ability to use these skills to successfully and responsibly solve problems in a variety of situations" [3]. The level of learning is defined by levels. Three different competency levels that build on one another are defined in the National Competency-Based Catalogue of Learning Objectives for Undergraduate Medical Education ([Fig. 1]). Factual knowledge (level 1) and practical and technical knowledge (level 2) must be achieved before practical competence (level 3).

Fig. 1 Competency levels (National Competency-based Catalogue of Learning Objectives for Undergraduate Medical Education).

The European Society of Radiology (ESR) and German Radiological Society jointly developed model curricula to implement the recommendations regarding competency-based teaching in the National Competency-Based Catalogue of Learning Objectives for Undergraduate Medical Education in radiological teaching [4] [5]. However, the extent to which these changes can be implemented in the daily routine is not yet clear.

Lectures using transparencies have become established for teaching various content but have their limits when it comes to teaching content on levels 2 and 3. This format lacks active student participation and practical training. Practical teaching has long been established in subjects such as anatomy and pathology. As a result of increasing digitalization, the technical requirements for practical teaching are now also met in radiology. Therefore, it makes sense to integrate a radiological DICOM viewer into teaching thereby making radiological teaching more practical. This type of teaching is not yet common and has only been reported in a few studies [6] [7] [8] [9] [10] [11] [12].

We established two interactive competency-based radiology courses in which students independently and systematically view, analyze, and present radiological examinations on computers with a DICOM viewer under supervision of an instructor.

The goal of this study was to evaluate the newly established courses regarding practical feasibility, acceptance by students and instructors, and implementation of competency-based teaching as required by the model curriculum of the German Radiological Society and the National Competency-Based Catalogue of Learning Objectives for Undergraduate Medical Education.

Materials and Methods

Both evaluated courses were conducted at Hannover Medical School as part of the model course HannibaL (Hannoveraner integrierter, berufsorientierter und adaptiver Lehrplan). Beginning in academic year 2015/16, an elective course was established and incorporated in the usual radiology lecture in the 5th year. The average class size was approximately 20 students per course. The course included three 90-minutes sessions per quarter. Due to the success of this course, it was included in the curriculum as a regular course starting in academic year 2016/17 and an obligatory course for all students in the 4th academic year was additionally created. This course had a class size of 25 students and included two 180-minute sessions. Both courses were conducted in a multimedia room equipped with 25 PCs ([Fig. 2]). The server-based radiology DICOM viewer (Visage Imaging^®, Pro Medicus Limited, Richmond, Australia) that we used is also used in the clinical routine. The viewer allows students to use all techniques available to clinical radiologists, e. g. interactive multiplanar reconstruction, distance measurements, measurements in Hounsfield units, and maximum/minimum intensity projections ([Fig. 3]). A special login gives students access to anonymized teaching cases but not to the entire PACS. The cases are transferred to this restricted teaching area and anonymized directly with the DICOM viewer. Since this is a server-client solution, complicated transfer or import of cases is not necessary. In addition to pathological cases, annotated teaching cases were used study normal anatomy ([Fig. 3a]). Desktop monitoring software (Netop Vision^®, Netop, Birkerod, Denmark) made it possible to project the screen of a student's PC so that the case could be discussed jointly as a group. 

Fig. 2 The course was conducted in a multimedia room equipped with 25 computers.

Fig. 3 Desktop of the DICOM Viewer (Visage Imaging^®) a Cross-sectional CT images with anatomical annotations; b Multiplanar reconstruction for generation of a "virtual" chest X-ray. Left: axial sections of a lung carcinoma in the left lower lobe (3 mm slice thickness). Coronal (middle) and sagittal (right) reconstructions (100 mm slice thickness). Arrows point to lung carcinoma in the left lower lobe.

Plain radiographs and/or CT scans of the skull, thorax, abdomen, and skeleton or CT scans of the thorax and abdomen were analyzed with DICOM viewer software in the elective and obligatory course, respectively. To represent the principle of practical competency-based teaching, an exemplary selection of treated topics with the corresponding competency level and the learning objectives of the model curriculum and the National Competency-Based Catalogue of Learning Objectives for Undergraduate Medical Education is presented in [Table 1]. The students interactively studied 5 – 8 cases (elective course) and 5 – 6 cases (obligatory course) per course using DICOM viewer and desktop monitoring software. The ratio of classroom teaching to practical teaching was approx. 1:1. The format is explained using chest X-rays as an example: PowerPoint was initially used to teach the basic principles of the acquisition technique and of the systematic evaluation of a chest X-ray ([Table 1], level 1). This was followed by a short introduction to the DICOM viewer and its basic functions ([Table 1], level 2). To learn cross-sectional anatomy, the students used the DICOM viewer to independently view a chest CT with annotated anatomical structures ([Table 1], level 2, [Fig. 3a]). In the subsequent demonstration of pathological cases, the students applied their acquired image analysis skills and used the DICOM viewer to present pathological findings on the chest X-ray. Students then presented each case individually to the group via desktop monitoring software ([Table 1], level 3a). The differential diagnoses were evaluated and the final diagnosis was determined during case demonstration with the help of the instructor. The students then presented the corresponding CT scan ([Table 1], level 3a) and applied previously acquired skills such as measurement of Hounsfield units ([Table 1], level 3a). To be able to better understand projection radiography, coronal and sagittal reconstructions were performed by all students to generate „virtual“ X-rays from the CT scan ([Table 1], level 2, [Fig. 3b]). Following joint analysis of the cases, the instructor used PowerPoint to teach additional theoretical content regarding the disease ([Table 1], level 1).

Table 1 Selective topics of the practical competency-based radiological teaching with respect to the learning objectives of the model curriculum for radiological teaching (DRG) and of the national competency-based catalogue (NKLM).

Topics	PowerPoint	DICOM viewer	Desctop monitoring software	Competency level	Learning objectives model Curriculum of the German Radiological Society	Learning objectives National Competency-Based Catalogue of Learning Objectives for Undergraduate Medical Education
Anatomy, thorax:					R-1.2 – In-depth knowledge of the normal anatomy of various organs and structures on X-ray images – knowledge of the normal cross-sectional anatomy on CT – reliable identification of normal anatomical structures on X-ray images and CT scans R-2.2 – Knowledge of the normal anatomy of the heart and vessels on X-ray images and CT scans – reliable identification of normal anatomical structures on chest X-ray images R-2.3 – Knowledge of the normal anatomy of the respiratory system and lungs, heart, large vessels, mediastinum, and thoracic wall on chest X-ray images and on CT scans – identification of the following structures on posterior-anterior (p. a.) and lateral chest X-ray images: Fissures and lobar boundaries of the lung, trachea, main bronchia, atria and ventricles of the right and left heart, pulmonary arteries, aorta, mediastinal parts and diaphragm	15.4.1.6 – die Absolventin und der Absolvent können relevante anatomische Strukturen in Röntgenaufnahmen erkennen und benennen (Lungenlappen und -grenzen, Trachea und Carina, Herz mit grenzdefinierenden Strukturen, Pulmonalarterien, Aorta, V. cava sup., V. azygos, Wirbelsäule, Rippen, Claviculae, Scapulae, Humeri, Sternum, Recessus) 15.4.1.10 – die Absolventin und der Absolvent können relevante anatomische Strukturen in computertomografischen Schnittbildern erkennen und benennen.
Anatomy of chest X-ray	x	-	-	1
Systematic evaluation of chest X-ray	x	-	-	1
Functioning of DICOM viewer	-	x	-	1, 2
Normal anatomy on chest CT scan	-	x	-	1, 2
Window settings for CT	x	x	-	1, 2
Multiplanar reconstructions	-	x	-	1, 2
Aortic dissection					R-1.3 – Understanding of the acquisition times after bolus administration of contrast agent and their relevance for the particular issue – ability to identify whether a CT scan is contrast-enhanced. R-2.2 – Knowledge of the normal dimensions of the aorta and the classification of aortic aneurysms and dissections.	15.4.1.7 – Graduates can identify and describe relevant pathological changes on X-ray images (mediastinal enlargement). 15.4.1.11 – Graduates can identify and describe relevant pathological changes on CT scans (aortic dissection).
Symptoms	x	-	-	1
Pathogenesis	x	-	-	1
Classification of aortic dissections	x	-	-	1
Visualization on X-ray image	-	x	x	1, 2, 3a
Visualization on CT scan	-	x	x	1, 2, 3a
Contrast agent phases on CT scan	x	x	x	1, 2, 3a
Treatment options	x	-	-	1
Mediastinal tumors					R-1.2 – Understanding of Hounsfield units and their scaling – Knowledge of the normal attenuation values in Hounsfield units for important tissue and pathologies – measurement in Hounsfield units on CT scan and identification of corresponding tissue properties. R-2.3 – Knowledge of the visualization of various mediastinal masses on chest X-ray images and CT scans	15.4.1.7 – Graduates can identify and describe relevant pathological changes on X-ray images (mediastinal enlargement). 15.4.1.11 – Graduates can identify and describe relevant pathological changes on CT scans (mediastinal lymph nodes).
Chest X-ray of thymoma	–	x	x	1, 2, 3a
CT scan of thymoma with measurement of Hounsfield units	x	x	x	1, 2, 3a
CT scan of bronchogenic cyst with measurement of Hounsfield units	–	x	x	1, 2, 3a
Chest X-ray image of mediastinal lymphoma	–	x	x	1, 2, 3a
CT scan of mediastinal lymphoma	–	x	x	1, 2, 3a
Differential diagnosis of mediastinal tumors	x	–	–	1
Intracerebral hemorrhage					R-2.9 – Understanding of typical radiological changes in hemorrhagic stroke on CT scan – Identification and differentiation of different types of intracranial hemorrhage on CT scan.	15.4.1.11 – Graduates can identify and describe relevant pathological changes on CT scans (differentiation of EDH, SDH, subarachnoid hemorrhage, intracranial hemorrhage on CT scan).
Classification of intracerebral hemorrhages	x	–	–	1
Basal ganglia hemorrhage on CCT	–	x	x	1, 2, 3a
Pathogenesis and symptoms	x	–	–	1
Subdural hemorrhage on CCT	–	x	x	1, 2, 3a
Pathogenesis and symptoms	x	–	–	1
Epidural hemorrhage on CCT	–	x	x	1, 2, 3a
Pathogenesis and symptoms	x	–	–	1
Subarachnoid hemorrhage on CCT	–	x	x	1, 2, 3a
Pathogenesis and symptoms	x	–	–	1

An anonymous online survey (SurveyMonkey^®, Dublin, Ireland) was performed immediately after the courses. Students were able to rate their overall satisfaction with the course on a scale of 1 to 6. In addition, students were asked to rate on a scale of 1 – 5 how easy it was to use the DICOM viewer, whether the course helped to improve their knowledge of cross-sectional anatomy and radiological findings, whether the combination of DICOM viewer and classroom teaching was superior to teaching via PowerPoint alone, and whether the course helped to increase their interest in radiology. Students were also able to write comments about the course and express ideas and criticisms. In the elective course, students were asked whether they would be in favor of a new practical competency-based course being established and in the obligatory course they were asked whether they considered the course to be relevant to their future occupation.

Instructors rated the course retrospectively on a scale of 1 – 6 via an online survey (SurveyMonkey^®). Moreover, the instructors were asked to rate possible advantages and disadvantages of practical competency-based teaching on a scale of 1 – 5. Improved teacher-student interaction, better understanding of the DICOM viewer and cross-sectional images, improved understanding of radiologic findings, greater understanding of course content, increased learning success due to active student involvement and higher motivation on the part of the students were included in the survey as advantages. Greater preparation effort, insufficient time for theoretical content, distraction due to the computers, worse preparation for MC tests, and worse teaching of structured content were included in the survey as disadvantages. Moreover, use of the DICOM viewer during teaching was rated on a scale of 1 to 5.

Results

A total of 260 students (160 students in the elective course and 100 students in the obligatory course) participated in the survey. Since attendance was only documented in the obligatory course, a response rate could only be determined for this course (86 %). The overall rating of the courses was positive: the elective course and the obligatory course were rated as excellent by 27 % and 10 %, very good by 41 % and 30 % and good by 28 % and 52 %, respectively ([Fig. 4a]). 94 % (elective course) and 99 % (obligatory course) of the students stated that use of the DICOM viewer in the course was feasible ([Fig. 4b]). Almost all students felt that the course improved their understanding of cross-sectional anatomy (97 % in elective course, 95 % in obligatory course) and radiologic findings (97 % in elective course, 98 % in obligatory course) ([Fig. 4c-d]). Teaching with a combination of PowerPoint and DICOM viewer is superior to teaching with PowerPoint alone (98 % in elective course and 95 % in obligatory course, [Fig. 4e]). 61 % of the students in the elective course and 65 % of the students in the obligatory course indicated that the course increased their interest in radiology, while 32 % in both courses were undecided ([Fig. 4f]). 98 % of the students in the elective course rated the establishing of a practical competency-based radiology course with a DICOM viewer as extremely or very useful ([Fig. 4 g]). 92 % of the students in the obligatory course felt that the course was very relevant or relevant for their future occupation ([Fig. 4 h]). The interaction with instructors and independent viewing and presentation of cases were rated positively in the comments. Suggestions for improvement included more time for the introduction to the program, a greater number of cases and the option to independently prepare and post-process cases on suitable workstations.

Fig. 4 Results of student evaluation of elective (n = 160) and obligatory course (n = 100): Results of MC questions a–h.

9 instructors participated in the instructor survey (response rate 75 %). The instructors also rated the competency-based radiology course using a DICOM viewer positively: 44 % as excellent, 33 % as very good, and 23 % as good ([Fig. 5a]). The majority of instructors (89 %) stated that it was feasible to use a DICOM viewer in the course, while one instructor (11 %) disagreed primarily due to unexpected, temporary technical difficulties (network failure) ([Fig. 5b]). The instructors indicated the following as advantages: interaction with students (100 %) and improved understanding of radiological findings (89 %) ([Fig. 5c]). The instructors spent approximately 50 % of the overall course time speaking compared to 90 % in a conventional lecture (not shown). The majority of the instructors (56 %) stated the following as a disadvantage of the course: greater preparation effort than in a conventional lecture. However, 44 % of the instructors did not share this opinion ([Fig. 5 d]). Only one instructor found the teaching of theoretical content insufficient. None of the instructors felt that the teaching of structured content was worse than in a conventional lecture. The majority of the instructors (67 %) did not observe increased distraction of the students due to the computers, while 33 % of the instructors were undecided. The majority of the instructors (56 %) did not notice worse preparation for MC tests, 33 % were undecided, and one instructor (11 %) was concerned about worse preparation.

Fig. 5 Results of evaluation of instructors (n = 9, a–d).

Discussion

The goal of our study was to evaluate the newly established practical competency-based radiology course with respect to practical feasibility, acceptance by students and lecturers, and competency-based radiological teaching with respect to the model curriculum of the German Radiological Society and the National Competency-Based Catalogue of Learning Objectives for Undergraduate Medical Education.

We were able to show 1) that practical competency-based radiological teaching is very feasible and has high acceptance among students and lecturers, 2) that the course aroused interest in radiology among the majority of the students, 3) that students report improved understanding of cross-sectional anatomy and radiological findings, and 4) that this type of teaching fosters competency-based teaching as proposed by the model curriculum of the German Radiological Society and the National Competency-based Catalogue of Learning Objectives for Undergraduate Medical Education.

While the learning objectives in the National Competency-based Catalogue of Learning Objectives for Undergraduate Medical Education and the model curriculum of the German Radiological Society are clearly defined, it is not yet clear how competency-based teaching can be efficiently implemented in radiology. The practical competency-based teaching presented in this study represents one possibility. Students learn how to use a DICOM viewer, implement their learned factual knowledge in practice, and present cases via desktop monitoring software in front of the group under the supervision of the instructor. This requires different levels. Students must acquire basic factual knowledge (level 1) as well as practical and technical knowledge (level 2) and independently demonstrate pathological and non-pathological radiological findings under supervision (level 3a). In particular, this addresses the role of the medical expert as defined in the National Competency-based Catalogue of Learning Objectives for Undergraduate Medical Education.

To date, there are only a few studies investigating the use of interactive image analysis on a DICOM viewer in the teaching of students even though simple DICOM viewers are now available as freeware for all operating systems and are used at almost all clinics, even by non-radiologists. Strickland used the free Mac-based DICOM viewer Osirix to set up a virtual workstation on which students could independently view cases from the musculoskeletal region [6]. The usability of Osirix as a teaching platform for radiologists was also emphasized by Rojas who generated a case collection of interstitial pulmonary diseases and implemented pathology reports, clinical findings, and radiological findings in the DICOM data set [10]. Additional studies used a DICOM viewer to improve understanding of anatomy and cross-sectional anatomy [7] [8] [9] [10] [11] [12]. Alvarez conducted an introductory course in radiologic anatomy using the freely available iPACS DICOM viewer. Anatomical structures were annotated in the CT data set in small groups and the results were discussed with the entire group via desktop monitoring software [7]. In a further study, donated bodies were examined with CT prior to being dissected by students in the macroscopic anatomy course [8]. Integrative classes in anatomy and radiology in which images are interactively analyzed are being offered as part of preclinical education at multiple medical schools in Germany [13]. For their study Schober et al. used the Osirix DICOM viewer which is also available free of charge for the iPad [12]. A study from Japan used the same method to compare the body anatomy to the CT anatomy during dissection in the anatomy course [11].

Interactive teaching using a DICOM viewer is more established in post-graduate medical training than in student education. DICOM-based standardized test methods have been evaluated in a number of studies to measure the competence of prospective radiologists on the basis of simulated emergency CT cases [14] [15] [16] [17]. In particular, the possibility to simulate emergency situations prior to being on call was considered useful by participants [17]. The interaction when viewing DICOM data sets [7] [10] [16] and the high acceptance by study participants were considered significant advantages in these studies as well as in our results.

Why isn't the use of DICOM viewers in student education more common? This is probably due to issues of the past, such as a lack of technical infrastructure, inflexible systems requiring resources for operating the DICOM viewer, and concern about preparation effort. Digital image storage systems in radiology have only been widely available for about 15 years and image export and anonymization for teaching purposes used to time-intensive. DICOM viewers are now readily available in radiology and are flexible as a result of the use of client-server solutions. In the case of the DICOM viewer that we used, anonymization and assignment to the teaching archive can be performed in one step without significant effort. By using iOS-based mobile devices, PDFs and photos (surgery images, pathological specimens) can also be integrated simply and intuitively. A further advantage is the use of the same DICOM viewer as in the clinical routine so that instructors do not require additional training and students have easier access during the clinical electives, subinternship and practical year.

An alternative to the software that we used is the free Mac-based DICOM viewer Osirix. For example, radiologists in Great Britain have already recognized the potential of this software for teaching. There is a user group of approx. 265 registered users who exchange anonymized DICOM teaching cases via Osirix and use them for teaching and examination purposes [18].

In our opinion a number of conditions must be met to be able to successfully establish practical competency-based teaching. A multimedia room with a sufficient number of PCs must be available. Desktop monitoring software allowing case presentation by individual students is also required. Moreover, the DICOM viewer must allow current functions such as multiplanar reconstruction and quick and uncomplicated generation of teaching cases to minimize preparation effort.

As a result of the reforms initiated by the National Competency-based Catalogue of Learning Objectives for Undergraduate Medical Education, examination formats will also change. Objective structured clinical examinations (OSCEs), which test skills regarding the patient interview and clinical examination in particular, have already been instituted at some universities. To date, radiology is rarely represented in these examinations. The interactive platform described in this study offers optimal conditions for conducting OSCEs in radiology under standardized conditions. Conceivable further developments of the course concept include blended learning and flipped classroom approaches in which students independently acquire theoretical content in advance, e. g. through online material, in order to be optimally prepared for practical teaching.

It is noteworthy that the majority of the students were in favor of the establishment of a new obligatory seminar for students despite the increased time requirement – a clear endorsement of the teaching format. Due to the positive feedback, image analysis training has been included in the curriculum as an obligatory course since the 2016/17 academic year.

Our study has some limitations, the most important being that a comparison group was not studied prospectively. However, a two-arm prospective design was not possible due to differences in treatment and a cross-over design was not possible due to the time limitations in the radiological curriculum.

In summary, our results show that practical radiological teaching on PCs using a DICOM viewer and desktop monitoring software is an innovative approach with high acceptance among students and instructors and good practical feasibility. The competency-based teaching proposed by the National Competency-Based Catalogue of Learning Objectives for Undergraduate Medical Education and the model curriculum of the German Radiological Society can be implemented in this way.

Interessenkonflikte

Die MHH hat grants von Siemens Healthineers, von Promedicus Ltd. und von Delcath Systems, Inc. erhalten. Prof. Wacker hat ein Honorar der Novartis Pharma GmbH und Delcath Systems, Inc. erhalten.

• References

• 1 MD PJF, MD DLC, PhD PZAB. et al. The Lancet Commissions Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. The Lancet 2010; 376: 1923-1958
• 2 Kooperation der Gesellschaft für medizinische Ausbildung (GMA) dMFeVMudVdHfZ-, Mund- und Kieferheilkunde (VHZMK). Nationaler Kompetenzbasierter Lernzielkatalog Medizin (NKLM) Medizinischer Fakultätentag 2015; Online publication. Downloaded from internet 6.1.2017 von der URL http://www.nklm.de/download.html
• 3 Weinert FE. (Hg.) Leistungsmessungen in Schulen. Weinheim u. Basel: Beltz; 2001: 27
• 4 http://www.myesr.org/sites/default/files/ESR_2017_ESR-EuropeanTrainingCurriculum_U-LEVEL-web.pdf web site. Downloaded from internet 20.4.2017
• 5 Ertl-Wagner BBJ, Mahnken AH. et al. White Paper: Radiological Curriculum for Undergraduate Medical Education in Germany. RoFo : Fortschritte auf dem Gebiete der Rontgenstrahlen und der Nuklearmedizin 2016; 188: 1017-1023
• 6 Strickland CD, Lowry PA, Petersen BD. et al. Introduction of a virtual workstation into radiology medical student education. Am J Roentgenol 2015; 204: W289-W292
• 7 Alvarez A, Gold GE, Tobin B. et al. Software tools for interactive instruction in radiologic anatomy. Academic radiology 2006; 13: 512-517
• 8 Lufler RS, Zumwalt AC, Romney CA. et al. Incorporating radiology into medical gross anatomy: does the use of cadaver CT scans improve students' academic performance in anatomy?. Anatomical sciences education 2010; 3: 56-63
• 9 Tam MDBS, Hart AR, Williams SM. et al. Evaluation of a computer program (‘disect’) to consolidate anatomy knowledge: A randomised-controlled trial. Medical teacher 2010; 32: e138-e142
• 10 Rojas CA, Jawad H, Chung JH. The new era of radiology teaching files. Am J Roentgenol 2012; 198: 773-776
• 11 Murakami T, Tajika Y, Ueno H. et al. An integrated teaching method of gross anatomy and computed tomography radiology. Anatomical sciences education 2014; 7: 438-449
• 12 Schober A, Pieper CC, Schmidt R. et al. "Anatomy and imaging": 10 years of experience with an interdisciplinary teaching project in preclinical medical education – from an elective to a curricular course. RoFo : Fortschritte auf dem Gebiete der Rontgenstrahlen und der Nuklearmedizin 2014; 186: 458-465
• 13 Dettmer S, Schmiedl A, Meyer S. et al. Radiological anatomy – evaluation of integrative education in radiology. RöFo : Fortschritte auf dem Gebiete der Röntgenstrahlen und der Nuklearmedizin 2013; 185: 838-843
• 14 Ganguli S, Camacho M, Yam C-S. et al. Preparing first-year radiology residents and assessing their readiness for on-call responsibilities: results over 5 years. Am J Roentgenol American journal of roentgenology 2009; 192: 539-544
• 15 Ganguli S, Pedrosa I, Yam C-S. et al. Part I: preparing first-year radiology residents and assessing their readiness for on-call responsibilities. Academic radiology 2006; 13: 764-769
• 16 Towbin AJ, Paterson B, Chang PJ. A computer-based radiology simulator as a learning tool to help prepare first-year residents for being on call. Academic radiology 2007; 14: 1271-1283
• 17 Towbin AJ, Paterson BE, Chang PJ. Computer-based simulator for radiology: an educational tool. Radiographics : a review publication of the Radiological Society of North America, Inc 2008; 28: 309-316
• 18 [Anonym]. http://www.osirix-ukusergroup.org/ Web site. publication. Downloaded from internet 7.1.2017