Fortschr Röntgenstr
DOI: 10.1055/s-0043-119037
Academic Radiology
© Georg Thieme Verlag KG Stuttgart · New York

Improvement of Radiological Teaching – Effects of Focusing of Learning Targets and Increased Consideration of Learning Theory Knowledge

Article in several languages: English | deutschStefan Wirth1, York-Alexander William1, Marco Paolini1, Kathrin Wirth1, Daniel Maxien1, Maximilian Reiser1, Martin R. Fischer2
  • 1Department of Radiology,University Hospital, LMU Munich, Germany
  • 2Institute for Medical Education,University Hospital, LMU Munich, Germany
Further Information

Correspondence

Prof. Dr. Dr. Stefan Wirth
Institut für Klinische Radiologie, Ludwigs-Maximilians-Universität München
Nußbaumstr. 20
80336 München
Germany   
Phone: ++ 49/89/4 40 05 92 01   
Fax: ++ 49/89/4 40 05 92 02   

Publication History

16 February 2017

20 July 2017

Publication Date:
20 September 2017 (eFirst)

 

Abstract

Purpose Based on evaluation and examination results of students, a necessity for improvement of so far purely instructor-based radiological teaching at the local institution was determined. Aim of our study was to use one out of eight seminars to exemplify adaptation of the teaching concept according to learning theory knowledge, to determine the resulting effects and to interpret them.

Materials and methods The institutional review board approved the prospective study of the seminar conversion, which was performed after the end of the winter semester 2015/2016. Didactically, this included a course split into online preparation, attendance phase and online follow-up with integration of interactive scaffolding, practice-oriented clinical teaching according to Stanford, Peyton skills transfer and extensive feedback into the attendance phase. At the beginning and at the end of each course, each student filled in identical, standardized questionnaires (n = 256 before and after conversion) using a 5-point Likert scale (1: very good; to 5: deficient) and additionally answered two randomly chosen written examination questions from a content-adapted questionnaire pool of the last five years. For statistical evaluation, the Mann-Whitney U-Test was used for evaluation data and Fisher's Exact test for exam questions.

Results Before/after conversion, the subjective total evaluation score of students was 3.22 (mean value) ± 1.51 (standard deviation) / 1.66 ± 0.78 (p < 0.001) and the objective proportion of correctly answered examination questions in the respective cohort at the beginning of the seminar 37.7/53.9 % and at the end of the seminar 55.1/84.6 % (p < 0.001).

Conclusion The conversion of the test seminar resulted in both a better evaluation of the teaching unit by the students (evaluation) and a considerably higher rate of correctly answered examination questions from past state examinations (learning success). This supports transferring the concept to comparable teaching units.

Key points

  • Radiological teaching allows integration of current learning theory concepts with reasonable effort.

  • In a test seminar this improved the evaluation results of the teaching unit by the students.

  • In addition, this also led to a higher rate of correctly answered examination questions from past state examinations.

  • This supports further steps towards excellent radiological teaching.

Citation Format

  • Wirth S, William Y, Paolini M et al. Improvement of Radiological Teaching – Effects of Focusing of Learning Targets and Increased Consideration of Learning Theory Knowledge. Fortschr Röntgenstr 2017; DOI: 10.1055/s-0043-119037


#

Introduction

Since the establishment of a Chair for Medical Didactics and Educational Research [1] at the studied university, the average score of the students in the state examinations increased from 3.11 in 2010 to 2.63 in 2014. Country-wide this corresponded to 11th place of 36 universities in the written part of the second state examination [2]. Nevertheless, an internal comparison of the correct solution rate of the written state examinations between 2011 and 2015 for questions with radiological content mainly resulted in a below-average rate of correct solutions.

The teaching units in radiology were not practical, with little structure and inadequate coordination. Likewise, the seminars and practica were almost exclusively instructor-centered courses and in the form of lectures. It was primarily up to the instructional staff to determine the depth of course content as well as the presentation method. In addition, substantial time was lost in the repetition of material from previous semesters. Students indicated that the goal of radiological teaching units was frequently unclear, that they desired more hands-on activity, and that mastery of the material could not be assessed.

On the whole, a fundamental need for the change of radiological coursework was determined. Since, for personnel reasons this could not be implemented across the board for all radiological teaching units, an initial step was undertaken in the form of a test course, the success of which could be a motivation to apply the changes to other teaching units. In consultation with the local medicine didactics organ, it appeared to be useful to orient teaching toward concrete clinical conditions and needs. Earlier publications have supported this and particularly report the success using interactive, case-based learning and e-learning [3] [4] as well as peer-to-peer feedback [5]. The following uses terminology relating to medicine didactics, therefore refer to [Table 1] for a short overview.

Table 1

Synopsis and brief explanation of medical-didactic terms.

term

explanation

goals

blended learning

Learning form and organization, which offer a didactically meaningful linking of traditional face-to-face instruction and modern forms such as E-learning. The concept combines the effectiveness and flexibility of electronic learning forms with the social aspects of personal communication as well as the practical learning of skills.

Suitable combination of different media and methods to enhance their advantages and minimize their disadvantages.

brainstorming

Method for coming up with ideas. A group to address influential parameters of a specific issue, mixed according to age, expertise, hierarchical position, ideally containing 6 to 8 persons, to collect as many ideas possible in a short amount of time. Free association and imagination are not only allowed but desired. On the other hand, evaluating and judging the ideas should not be permitted.

Generation of new ideas in a group with regard to a given issue or topic.

buzz group

Murmur group. Two to four students form a group and quietly exchange opinions on a specified topic. After expiration of a specified period of time, the results of the individual groups are shared with the plenary group. Frequently the buzz groups represents a transition between the instructor-centered format and the interactive format.

Use of prior knowledge, lowering of inhibitions to active participation by the student.

e-learning

Teaching forms that use electronic or digital media for the presentation or distribution of learning materials.

Independent of location or time.

feedback

Group dynamic method to respond to a person regarding behavior. This includes how other participants perceived, understood and experienced this person.

Awareness of weaknesses and strengths. Repetition of core content.

flipped classroom

Pedagogical model in which the typical lecture and homework elements of a course are reversed. Instead of learning during class time and practicing it at home, material is worked on at home and practiced in class.

Refreshing fundamentals, avoiding intervening questions.

peyton

Method of acquiring practical skills. First, instructors demonstrate the skill at a normal speed without commentary. Then instructors repeat the procedure slowly while explaining the individual procedural steps. In the third step students explain to the instructors the individual procedural steps. In the final step the students perform the procedure independently.

Transmission of practical skills, in this case diagnosis and assessment.

peer-to-peer

Communication at the same level, in this case, student-to-student.

Increase of acceptance, inhibition reduction.

scaffolding

Support of complex learning processes in the form of instructions, concepts and other help provided by instructors. In this way, the knowledge of the learner can be addressed individually and complex tasks can be divided into simple controllable segments.

Simplification of complex learning processes, individualization and time management.

swot

Strategic planning tool to analyze possible internal (in this case the concrete course of teaching units) and external effects (in this case radiology, the clinic or teaching as a whole) of project implementation. Strengths, Weaknesses, Opportunities and Threats are analyzed.

Project enhancement at the start of the planning phase

The description of radiography of the thorax (chest X-ray) is common, complex, typical and due to its relevance in the clinical routine, is a common element in medical certification examinations [6]. Unfortunately the diagnostic performance of students and the medical profession in the early stage of training is frequently unsatisfactory [7] [8]. This is particularly true under emergency conditions such as in the intensive care unit; therefore the seminar “Radiological Imaging in the Intensive Care Unit” appeared to be a suitable test course.


#

Goals

The goal was to improve the seminar “Radiological Imaging in the Intensive Care Unit” by intensifying the focus on content and orientation according to theoretical knowledge, to identify the resulting effects and to interpret them.

The issue was to determine the results of coordinated learning goals and increased focus as well emphasized orientation toward learning theory when compared to the semester prior to this reorientation. This might be measured by a significant improvement (p < 0.05) in the subjective assessment by the students by at least 0.5 points in a 5-point Likert scale and/or at least a five-percent objective improvement in the results of associated state examination questions.


#

Materials and Methods

The local ethics commission approved the project under number 17 – 036 UE as a prospective study.

General Learning Goals and Introduction in the Curriculum

The overarching learning objectives of all radiological coursework are (a) imparting knowledge regarding the representation of normal and altered anatomy in images produced by various radiological procedures, (b) proficiency in systematic descriptive diagnosis, (c) competence in prioritized assessment, and (d) competence in the evaluation of indications in certain radiological procedures with and without contrast media. At the studied institution, six clinical modules (semesters) follow two years of preclinical studies. Module 1 is devoted to the principles of clinical medicine covering the radiological basis of (a) with respect to normal anatomy, (b), as well as partially regarding (d). Based on this, the interdisciplinary basic year (modules 2 and 3 combined as a one-year module 23) contains eight radiological seminars and one practicum [9]. Beyond that, radiology is available only as an elective subject in the Practical Year and in the context of a review course. This division is also oriented, for example, toward the radiological curricula recommended by the German Radiological Society with a more modality-oriented basic section and a competence-oriented second part [10].


#

Topics, Target Group, Format and Framework

The original teaching unit “Radiological Imaging in the Intensive Care Unit” is devoted mainly to supine radiography of the chest, but also includes typical CT findings such as cerebral hemorrhage, pneumonia, abscess, and typical ultrasound findings such as cholestasis, cholecystitis, pleural effusion, pericardial effusion and free abdominal fluid.

The 90-minute seminar is a required course with one-time participation of students in the second or third clinical semester. The class takes place in a daylight-free, dimmable, small lecture theater with 40 seats and is offered by each of the four lecturers 4 times per semester and is attended by about 16 students each.


#

Conversion of the Selected Test Seminar

In order to be able to assess the impact of the seminar changeover, a SWOT analysis was carried out involving students, lecturers and decision-makers (Table A, B, Online Supplement).

Prior to the change, the course contents showed distinct overlap with other teaching units (e. g. abdominal CT, trauma seminar, interventional radiology, etc.). In principle, imaging in the intensive care unit can involve all modalities. Using matrix analyses [11] [12], those involved in the restructuring of the coursework developed a consensus regarding the most important radiological modality with the least overlap with other modalities (Tables [3a, b]). In addition, a comparison of the learning contents and objectives with other teaching units indicated that all parts outside “supine chest X-ray” were either already represented in other coursework or could be integrated there with minimum effort. Therefore the subject area was limited to radiography in supine position and the course was renamed accordingly.


#

The modified Seminar

The learning objectives of the redesigned seminar “Supine Chest X-ray” are given in [Table 2]. These are distributed on the basis of the online preparation, face-to-face segment and online follow-up, and are toward the German “Competence-based learning goal catalog for medicine” (NKLM) [13] and further classified according to knowledge (K), Skills (S), Competencies (C) and Attitudes (A).

Table 2

Learning goals.

phase

type[1]

learning goal

preparation

K

Can name the differences between a supine chest X-ray and 2 planes of an upright chest X-ray.

K

Can provide quality criteria and normal anatomy for a supine chest X-ray.

K

Can provide a scheme for the structured assessment of a supine chest X-ray.

K

Can identify the classification of the certainty of results (certain, suspicious, questionable, exclusion).

K

Has viewed an example of the manifestation of typical pathologies (see below).

face-to-face

K S

Can provide quality criteria for a supine chest X-ray and normal finding.

C

Can apply a scheme for the structured assessment of a supine chest X-ray.

C

Can apply the classification of the certainty of results from the online preparation.

For each pathology group: (1) pneumothorax, tension pneumothorax, (2) atelectasis, ventilation abnormality, (3) pneumonia, typical and atypical, (4) congestive heart failure, congestion, hyperhydration, pleural effusion, pulmonary edema, (5) pericardial effusion (6) pulmonary embolism, (7) misplaced implant (8) example of co-pathology.

S

With help identifies manifestation in supine chest X-ray.

S

With help can identify normal and diseased changes.

C

With help can estimate the extent and urgency of a finding.

C

With help can make acute diagnostic and therapeutic decisions.

C

With help prioritize and structure with indication of certainty.

S C A

Can accept feedback from instructor/peer and can provide feedback to peer.

follow-up

S

Can identify pathologies from the face-to-face segment in sample images

S

Can systematically describe sample image.

C

Can systematically assess sample images from a case.

1 K: Knowledge, S: Skill, C: Competence, A: Attitude.


The online preparation is a ten-page Word document, about half of which contains repetition of previous content and in the other half prepares for the face-to-face segment. The face-to-face and follow-up phases are based on PowerPoint slides. The face-to-face segment begins in the instructor-centered format for about 20 minutes and introduces the case-based learning through an example, which is used as an example by the lecturers. The students undertake the remaining seven cases in various roles, including: finding, feedback on finding, assessment, feedback on assessment. Finally the lecturers again provide feedback on the individual roles and provide supplemental examples of the same pathology. For the face-to-face segment of the new course format, [Fig. 1] provides a visual explanation of the changing activity levels of the students (sandwich structure [14]). Central elements are interactive, clinically-oriented case studies following the Stanford clinical teaching model [15], as well as a special focus on feedback [16]. During the teaching unit, instructors reduce their help while the students work out the problems in a process referred to as “scaffolding” [17]. In this way instructors support the learning process by providing an initial orientation in the form of instructions, concepts and other assistance. [Table 3] provides the new sequence, and [Fig. 2] provides a practical example.

Table 3

Course schedule.

phase

articulation point[1]

duration

method

content

ICAP[2]

preparation

25 min

online / moodle

20 min

self-study

Repetition of foundations (PDF, 10 pages)

P

5 min

video

Demonstration of normal findings of chest X-ray 1 plane

P

face-to-face

90 min

seminar

Getting started

A

1 min

instructor

Greeting and introduction

P

1 min

eye catcher

Emphasis on clinical relevance of topic

P

2 min

instructor

Introduction of learning goals, structure, content and sequence of course

P

Theoretical principles of thoracic X-ray in 1 plane

7 min

lecture

Technique, quality criteria, feedback rules, finding scheme

P

B1

2 min

buzz group

Form/content of a written finding report?

A

C1

3 min

synchronization

Query results, consolidation and resolution

C A

4 min

lecture

Normal finding, differences between supine and upright X-ray

P

Cased-based learning Case 1 according to Peyton steps 1 + 2

4 min

lecture

Example: Case 1 with explanation of further sequence

P

each

scaffolding

Case-based learning Cases 2 through 8 according to Peyton steps 3 + 4 (including 3 min half-time break)

2.5 min

activity S1

Structured finding by students

C A

1 min

activity S2

Prioritized assessment by other students

C A

1 min

activity S3

Peer-to-peer feedback (FB) on S1, S2

I

1 min

interaction

Resolution and selected instructor feedback on S1–S3

I

2 min

lecture

Further examples and explanations of the pathology

P

Finishing up

3 min

lecture

Summary

B2

2 min

buzz group

Take-home items

C A

C2

4 min

synchronization

Query results, consolidation and resolution

I

D

1 min

lecture

Explanation of follow-up, send-off

P

1 min

questionnaire

Evaluation

A

1 min

eye catcher

Reviewed emphasis on clinical relevance of topic

P

1.5 min

buffer

follow-up

35 min

online / moodle

30 min

self-study

Approx 2 – 4 examples for each of 8 pathologies (PPT[3])

C A

5 min

multiple-choice test

Achievement control without grading

A

1 The articulation point indicates the phase of group work (A: no group work, B: entry into group work, C: synchronization of the group, D: end of group work).


2 ICAP indicates the cognitive level of engagement of the students (I: interactive, C: constructive, A: active, P: passive).


3 The cases are provided as a PowerPoint file (PPT) in Moodle, a web-based institutional organization platform for students.


Zoom Image
Fig. 1 Course of the seminar "X-ray of the chest in supine position" during the presence time after adaption according to learning theory knowledge. Green components represent activity of the lecturers, whereas this is marked for the students in Apricot.
Zoom Image
Fig. 2 Sample pathology "Congestion", taken from the seminar after conversion. a Clinical indication with corresponding image to be processed of one particular pathology group. One student does the reading, a different one the interpretation. If required, they receive help in each task. Subsequently, two other students give feedback on both tasks. b Dissolution of the case by providing the correct interpretation together with a corresponding CT image, followed by a last feedback by the teacher. c further clinical course. e, f Further examples of the same pathology group presented by the teacher.

The means of acquiring diagnostic abilities and assessment are based on Payton’s methodology [18] [19] which involves moving through four steps. During the first step (“Demonstration”), instructors demonstrate the relevant skill at a normal speed without commentary. In the second step (“Deconstruction”), instructors repeat the procedure slowly while explaining the individual procedural steps. In the third step (“Comprehension”) students explain to the instructors the individual procedural steps with some help from the instructors. In the fourth and final step (“Performance”) the students perform the procedure independently.

Follow-up, just as preparation, is optional. Additional cases are presented as PowerPoint files, each case consisting of two slides. The initial slide presents clinical information, issues and an X-ray image. Here the students can check their abilities, and with the second slide can review the finding, assessment and additional explanations.

The instructors received general pedagogical training in their continuing education and were involved in the restructuring process. Prior to the start of the new format course, two joint test classes were held with Practical Year students. In addition, SW conducted the initial classes which were observed at least once by each additional instructor as silent observers.

The total time required to restructure the seminar about four person-weeks.


#

Data Collection, Processing and Evaluation

At the end of each face-to-face course, the students filled out identical, standardized questionnaires using a 5-point Likert scale (n = 256 in the semester before and after conversion). In addition, at the start and finish of the course they answered two randomly selected written questions from a group of 50 state examination questions from the previous five years. The studies directorship took into account those questions with a particular relevance for intensive care patients and the content of which was not a separate focus of other teaching units. The resulting pool of questions was identical before and after the seminar. The focus was on supine chest X-ray, under makeshift conditions as well as subsequent CT. Both modalities made up about 90 percent of the questions; the rest had to do with other modalities, generally including ultrasound.

The evaluation questionnaire consisted of 31 questions, 30 of which used a Likert scale where 1: “Completely applicable” and 5: “Completely inapplicable”. The 31st question asked for an overall evaluation following the grading system where 1: “Very good” and 5: “Insufficient”. [Table 4] lists the questions used in an abridged form and shows how the resulting blocks of topics “Personal motivation”, “Learning climate”, “Instructor”, “Concept and material”, “Understanding and learning success” and “Overall grade”. All 31 questions are shown in the results section of [Table 5].

Table 4

Thematic grouping of questionnaire items for the subjective evaluation.

item no.

item text (shortened)

topic group

 1

Personal motivation is high

personal motivation

 2

Content relevant and important

 5

Preparation for course content necessary

 9

Follow-up necessary for course

26

High interest in content

 3

My knowledge of content is high

understanding and learning success

27

Course developed understanding of content

28

Can provide course content

29

Course makes preparation for examination easier

12

Learning goals were clear

concept and material

13

Identifiable common theme in course

14

Course time sufficient for processing content

15

Level of difficulty exactly right

17

Examples/images helped in understanding

22

Sufficient opportunities for active participation

23

Remarks on critical analysis of content

25

Scope of interactivity exactly right

16

Instructors made complicated material understandable

instructor

18

Instructors’ explanations easy to follow

19

Instructors tried to share knowledge

20

Adequate interaction of instructors and students

21

Instructors responded well to questions

24

Constructive learning atmosphere

learning climate

30

Course was fun

31

Course grade

overall grade

Table 5

Detailed results of the subjective evaluation.

item no.

item text (somewhat shortened)

before conversion[1]

after conversion1

mean

standard deviation

mean

standard deviation

 1

I am highly motivated to continue learning the material after the course

3.32

1.49

2.29

0.98

 2

In my opinion, course content is relevant and important

2.00

0.91

1.39

0.61

 3

My level of knowledge of the content of the AINS block in Module 23 is high

3.67

1.20

2.61

0.92

 4

My level of knowledge of the necessary basic radiological principles is high

3.76

1.17

2.76

0.87

 5

In my opinion, you should prepare for the course

2.09

1.154

2.14

0.99

 6

The supplied learning material helped me to prepare

[2]

2

1.74

0.90

 7

Preparation for the course should mainly contain radiological basic principles

1.81

1.00

2.39

1.04

 8

Preparation for the course should mainly contain sample images

1.55

0.85

1.79

0.83

 9

In my opinion today’s course should be followed-up

2.06

1.19

2.75

1.19

10

Follow-up should mainly contain radiological basic principles

2.13

1.30

2.90

1.26

11

Follow-up should mainly contain sample images

1.64

0.93

1.87

0.85

12

The learning goals were clear

2.29

0.92

1.88

0.94

13

I perceived a clear common theme thought out the course

3.20

1.63

1.70

0.91

14

During the course I had sufficient time to completely work through all the content

3.41

1.49

1.93

0.93

15

The level of difficulty of the course was exactly right

3.11

1.62

1.83

0.82

16

I my opinion, the instructor made difficult material easier to understand

1.30

0.57

1.58

0.79

17

The examples, images or cases helped me understand the content

2.07

0.871

1.48

0.66

18

I could easily follow the instructor’s explanations

1.32

0.60

1.63

0.93

19

It was important to the instructor that the students learn something

1.20

0.49

1.38

0.70

20

Behavior of the instructor was professional and appropriate

1.16

0.41

1.31

0.63

21

Instructor responded well to questions and comments

1.17

0.45

1.34

0.63

22

There was sufficient opportunity to get actively involved

2.77

1.82

1.36

0.66

23

I was encouraged to critically analyze the course content

3.09

1.64

1.81

0.83

24

I found the learning atmosphere in the course to be constructive

2.15

0.87

1.89

0.95

25

The scope of interactivity in the course was on the whole exactly right

2.95

1.71

1.88

0.94

26

I was interested in the content of the course

1.48

0.77

1.68

0.76

27

The course increased my understanding of the content

2.97

1.67

1.75

0.79

28

I can provide an overview of the course content

3.01

1.64

1.82

0.81

29

Attending the course makes preparation for the examination easier

3.31

1.46

1.99

0.92

30

On the whole the course was fun

2.43

0.80

1.85

0.93

31

On the whole I give the course the grade of

3.22

1.51

1.66

0.78

1 Questions 1 through 30: Likert scale where 1: “Completely applicable” and 5: “Completely inapplicable”. Question 31: school grading system where 1: “Very good” and 5: “Insufficient”.


2 Question 6 was posed only after the conversion, since previously no leaning material had been made available.


For the assessment of the evaluation sheets, mean value differences of more than 0.5 were defined as “relevant” and those with more than 1.0 as “highly relevant”. On the other hand, with respect to the state examination questions, the percentage of correct and incorrect answers in the cohort were determined, and changes of more than five-percent were considered relevant and highly relevant if greater than ten percent. The background of both considerations was that for ratings based on the school grading scale, changes of 0.5 would lead to a change in grades for half of the cohort, and this would be the case for changes of 1.0 for the entire cohort. Similarly, in the case of the state medical examinations, the standard case of the 60 per cent passing grade threshold (that is, without indexing correction to avoid a greater than 22 percent failure rate) would result in grade escalation in ten percent increments. The Mann-Whitney U test was used for independent samples for the statistical testing of differences in data of the evaluation sheets and, in the case of the test results, Fisher's Exact Test was used. With regard to the requirements for multiple tests, question 31 and the evaluation of the test results were considered to be not formally related to the other evaluation, and the Bonferroni method was used to adjust the threshold significances [20]. This resulted in a threshold of P < 0.05 for the overall score, P < 0.01 for each of the five other subject blocks of the evaluation surveys, and P < 0.01 for the state examination questions.


#
#

Results

The results are shown in Tables [ 5], [6].

Modification of the teaching unit resulted in a slight improvement in the rating of “Personal motivation” from 2.19 to 2.05; there was a significant improvement for “Understanding and learning success” from 3.24 to 2.01 as well as for “Concept and material” from 2.85 to 1.69. Assessment of “Instructors” showed a slight downgrade from 1.23 to 1.44. “Learning climate” improved from 2.29 to 1.87 as well for “Overall grade” which demonstrated a highly-significant improvement from 3.22 to 1.66. The results of the individual questionnaire items are shown in [Table 5].

Table 6

Summary of results.

topic group

statistics[1]

time point seminar

before conversion

after conversion

p-value

questionnaires

(subjective)

n = 256

n = 256

Mann-Whitney U test

personal motivation

MV (SD)

End

2.19 (0.46)

2.05 (0.62)

0.001

MD (25 % / 75 %)

2 (2 / 2)

2 (2 / 2)

understanding and learning success

MV (SD)

End

3.24 (1.41)

2.04 (0.63)

< 0.001

MD (25 % / 75 %)

2,5 (2 / 5)

2 (1,5 / 2,5)

concept and material

MV (SD)

End

2.85 (1.39)

1.69 (0.56)

< 0.001

MD (25 % / 75 %)

2 (2 / 4)

1.5 (1 / 2)

instructor

MV (SD)

End

1.23 (0.40)

1.44 (0.60)

< 0.001

MD (25 % / 75 %)

1 (1 / 2)

1 (1 / 2)

learning climate

MV (SD)

End

2.29 (0.73)

1.87 (0.85)

< 0.001

MD (25 % / 75 %)

2.5 (1.5 / 3)

2 (1 / 2.5)

overall grade

MV (SD)

End

3.22 (1.51)

1.66 (0.78)

< 0.001

MD (25 % / 75 %)

3 (2 / 5)

2 (1 / 2)

examination questions

n = 512

n = 512

Fisher’s Exact-Test

Correct (%)

Start

37.7

53.91

< 0.001

Correct (%)

End

55.11

84.6

< 0.001

< 0.001

< 0.001

p-value

1 A cross-wise comparison of correct answers of exam questions at the end of the seminar before conversion to the results at the beginning of the course in the new format revealed a p-value of 0.754 for double-sided and 0.377 for single-sided testing when applying Fisher's Exact Test. MV: Mean Value; SD: Standard Deviation.


As a consequence of the seminar restructuring, the rate of correctly answered state examination questions improved significantly at the start of the face-to-face segment from 37.7 % to 53.9 %, and at the end of the teaching unit substantially from 55.1 % to 84.6 % ([Fig. 3]).

Zoom Image
Fig. 3 Proportion of test questions correctly answered by the students before and after conversion of the test seminar. After conversion, the course already starts at the level with which this ended before conversion. In addition, the change resulted in a significant improvement during the event and raised the level of the collective right at the end of the event to the grade of a "two", which is usually given for ranks between 80 and 90 percent (data from a total of 1056 answers to questions, two per student at each time, see also Table 6).

[Table 6] contains an overview of all important results.


#

Discussion

The student protests at the end of the 1990 s called attention to problems in clinical teaching. The core of the criticism centered on theory-focused instruction and little hands-on orientation [21]. From that point on there was rethinking of current practices and nowadays almost every medical program has a separate organization for medicine didactics. The effort to improve teaching affects the course of study in general and therefor also concerns the individual departments that are specifically responsible for the implementation of changes. In addition, university clinics are responsible for a very large part of medical training.

The institution in question had itself determined a need for change, and in view of fixed resource allocation and presumed overall expenditure, first decided to restructure a radiological teaching unit as a test. Overlapping courses posed a particular challenge as well as a different sequence of seminars relating to the “AINS” blocks (Anesthesia, Intensive and Emergency medicine, Pain therapy) containing the test seminar. In addition, the radiological teaching units cover interdisciplinary topics in a relatively early stage of medical education. In addition there was a desired orientation to the white paper “Radiological Curriculum for Undergraduate Medical Education in Germany” by the German Radiological Society (DRG) and the Association of Chairmen in Academic Radiology (KLR) [10]. Taking into account proven learning-theory-based knowledge, first a preparatory part was developed to provide or refresh key knowledge in order to make the face-to-face time more effective (inverted or “flipped classroom” [22] [23], a model designed to make face-to-face learning more efficient). With the exception of these concepts, which rely on technical requirements and therefore comparatively up-to-date areas of knowledge in pedagogy and didactics, only established and known methods have been used. The results demonstrate the potential these proven and often simple means can have if applied concretely and consistently. At the studied institution, in consultation with the local medicine didactics organ, it appeared to be useful to orient teaching toward concrete clinical conditions and needs. Although publications on teaching radiology which go beyond an interdisciplinary approach while involving anatomy are only available in comparatively small number, such studies particularly found positive effects of interactive case-based learning and e-learning [3] [4] as well as peer-to-peer feedback [5].

In the new format, targeted communication [24] [25] takes place in several phases during the face-to-face time. Right at the start, an “eye catcher” illustrates the particular clinical significance of the topic. The introductory instructor-centered section stipulates concisely-formulated learning objectives, emphasizes personal benefit to the student and presents the agenda. Within the framework of the first activity phase, the students developed a “buzz group” in which two to four students briefly discuss a question, an image, a problem, etc. and thus become independently active while lowering the inhibition threshold for cooperation. The activity was a generally-applicable structure of a finding report, itself a structured application of the learning objectives.

Two-and-a-half minutes for structured diagnosis report by students is not much time, but still significantly more than is available in the clinical routine. One of the strengths of the concept after conversion is scaffolding which allows the instructor to step in at any time when time is running out. At the start of the class, there is no expectation of completeness of the findings report; instead the instructor intervenes and provides assistance, and continues to do so in the course of the class. This is possible since the exercise and the repetitions via feedback make this decreasingly necessary.

The main goal of promoting understanding and retention was to improve the conception, content and workflow as well as the promotion of active learning. The online preparation reinforces basic principles, thus saving time through elimination of unnecessary repetitions and supports the prerequisites for achieving the learning goals. The face-to-face segment improves the learning climate, and the more effective targeted communication increases attentiveness. The deliberate limitation of the introductory instructor-centered section is based as much on learning psychology as the subsequent, consistent activity changes with clear work instructions. The focus was on the group work approach so that it could be mastered by different instructors. The group phase at the beginning helps to determine meaningfulness, and at the end supports retention by students individually working out take-home items. The core of the face-to-face segment is the effective, recursive structure based on cases with scaffolding-supported skills mediation based on Peyton [17] [18] [19]. The third step of the approach in which students direct the instructors who respond accordingly can be only somewhat reproduced. Feedback tries to compensate for this, provides additional activity, supports the estimation by the students, enforces repetition of core content while allowing learning from observable errors [5] [16] [26]. The online follow-up work offers more depth with alternative case examples.

The training measures for instructors also proved to be exemplary. With the given limitation of the number of methods, training required little additional effort. In general, this consisted of medical-didactic continuing education for all instructors, a one-hour explanation of the new methods followed by a test run-through of the course for all instructional staff as well as participation as observers during the initial seminars in the new format.

The method according to Kern [27] has been established for curricula, and quality control standards have been defined by the World Federation for Medical Education [28]. After each semester, the student evaluations should be used for continuous improvement, evaluated and reviewed in the relevant radiological curriculum committee during the following quarter, discussed at an institutional conference also with regard to remarks by the instructors. Finally, responses should be chosen, documented and monitored.

The overall assessment of the modified teaching unit improved significantly by 1.56 points, thus placing it in the range of the top seventh of the scale. With regard to the individual results of different subject groups of further subjective evaluation, they were also all significant but differed in their relevance. The evaluation of the instructors declined by 0.19 point. This could be explained by a very good baseline level of 1.22 and possible uncertainties regarding the initial seminar implementation in the new format. The “Personal motivation” score of the students improved by 0.14 points as well as “Learning climate” which increased by 0.42 points just below the defined relevance threshold which was already at a good level. Highly relevant improvements were found for “Concept and material” (1.16) points as well as for “Understanding and learning success” (1.2 points); both are now at a good level.

Since multiple-choice questions do not test skills or competencies within the framework of the NKLM, their value in examinations is controversial, even though multiple choice questions have been standardized and used in the state examinations. For this reason, questions from written medical examinations were used in this study to obtain a more objective measure of the impact of the seminar conversion. The increase in the rate of 29.5 % of correctly answered examination questions confirmed the subjective results of learning success. Measured against the grade thresholds of the last written medical state examination [29], this would mean that both cohorts prior to the start of the seminar were rated “5” and the semester group prior to the conversion remained at this grade level after the seminar, whereas the group after the modification was rated as “2” directly after taking the seminar. Sustainability of this success should be promoted through the provision of follow-up, which had been little used to date, however, possibly due to a start-up effect. Although “supine chest X-ray” questions were the main focus, more than a third of the questions addressed CT. We can explain the success rate of questions dealing with radiography due to the course oriented toward the new format. With regard to CT as the second most important modality, this is presumably due to the extensive use to illustrate actively compiled pathologies and also as a result of supplementary examples.

After establishment of the conversion, a further improvement could be intensification of the self-learning phase in the online preparation to provide factual knowledge via blended learning concepts such as the flipped classroom [22] [23]. In this case, and especially in the online follow-up, an increased integration of the well-known advantages of e-learning [4] [30] would probably be varied and promising. To allow each student to assume each role at least once (finding, assessment, accepting and giving feedback on the finding and assessment), a limitation on the number of cases to be worked on would be necessary. Concretely this would mean eight cases, one of which would be handled by the instructor, thus limiting participation to seven students.

Limitations

This study is based on one seminar of an institution, therefore there are no reliable findings that allow a transfer of concept to other coursework or institutions. SW, one of four instructors, undertook the restructuring of the teaching unit; additionally, SW is the direct supervisor of the other instructors. Although effort has been made to avoid resulting bias, it cannot be fully ruled out

Multiple-choice questions were used to determine the student’s success; these queried knowledge but no skills or competencies according to the NKLM, and provided only a limited testing modality for the new format seminar.

The composition of the cohorts was also not checked for comparability. Although the number of 256 students per semester could support the presumption of a large balanced random sample, this cannot be assured. The same applies to seasonal effects, since there was both a summer and winter cohort.

Provision of follow-up is presumably supportive, but the extent of a sustained positive effect is still unknown.


#
#

Conclusions

Restructuring a test seminar in radiology with coordinated and focused learning targets as well as an increased inclusion of concepts of learning theory resulted in a valuable improvement and provided the motivation to apply the concept to comparable teaching units.

Possible financial advantages could result in the event of performance-oriented allocation of funding. Although this is certainly not the purpose of university teaching, restructuring courses might at least be effected without additional costs.

Further potential can be realized from increased integration of e-learning in the online preparation and follow-up, from a departmental or even institutional quality management approach to continuous improvement and in a reduction of group size.

Clinical Relevance

Radiology is directly integrated into diagnostics or course of therapy in up to 80 percent of cases, especially in hospitals. Consequently much radiological content is highly relevant outside the specialty. A measurable and valuable improvement resulted from focusing on learning goals and integrating learning theory in the studied seminar. Therefore it is a clear advantage to motivate the persons involved in teaching radiology to make an even greater contribution to improving medical education, resulting ultimately in even better physicians.


#
#

No conflict of interest has been declared by the author(s).


Correspondence

Prof. Dr. Dr. Stefan Wirth
Institut für Klinische Radiologie, Ludwigs-Maximilians-Universität München
Nußbaumstr. 20
80336 München
Germany   
Phone: ++ 49/89/4 40 05 92 01   
Fax: ++ 49/89/4 40 05 92 02   


Zoom Image
Fig. 1 Course of the seminar "X-ray of the chest in supine position" during the presence time after adaption according to learning theory knowledge. Green components represent activity of the lecturers, whereas this is marked for the students in Apricot.
Zoom Image
Fig. 2 Sample pathology "Congestion", taken from the seminar after conversion. a Clinical indication with corresponding image to be processed of one particular pathology group. One student does the reading, a different one the interpretation. If required, they receive help in each task. Subsequently, two other students give feedback on both tasks. b Dissolution of the case by providing the correct interpretation together with a corresponding CT image, followed by a last feedback by the teacher. c further clinical course. e, f Further examples of the same pathology group presented by the teacher.
Zoom Image
Fig. 3 Proportion of test questions correctly answered by the students before and after conversion of the test seminar. After conversion, the course already starts at the level with which this ended before conversion. In addition, the change resulted in a significant improvement during the event and raised the level of the collective right at the end of the event to the grade of a "two", which is usually given for ranks between 80 and 90 percent (data from a total of 1056 answers to questions, two per student at each time, see also Table 6).
Zoom Image
Abb. 1 Lerntheoretisch verbesserte Ablaufstruktur des Seminars „Röntgen-Thorax im Liegen“ während der Präsenzzeit. Grüne Bestandteile stellen Aktivität der Dozierenden dar, wogegen dies für die Studierenden in Apricot markiert ist.
Zoom Image
Abb. 2 Beispielpathologie „Stauung“ aus dem Seminar nach Umstellung. a Klinische Angabe mit zu bearbeitendem Bild einer Pathologiegruppe. Ein/e Studierende/r führt die Befundung, ein/e weiter/e die Beurteilung durch. Falls benötigt, so erhalten sie hierbei jeweils Hilfestellung. Im Anschluss daran geben zwei weitere Studierende Feedback zu Befund und Beurteilung. b Auflösung des Falles durch Angabe einer Musterbeurteilung und eines zugehörigen CT-Bildes, gefolgt von einem abschließenden Dozentenfeedback. c Weiterer Verlauf. e, f Weitere Beispiele derselben Pathologiegruppe, welche frontal durch Dozierende vorgestellt werden.
Zoom Image
Abb. 3 Anteil der durch die Studierenden richtig beantworteten Prüfungsfragen vor und nach Umstellung des Testseminars. Nach Umstellung startet die Lehrveranstaltung bereits auf dem Niveau, mit dem dies vor Umstellung endete. Zudem bewirkte die Veränderung eine deutliche Verbesserung während der Veranstaltung und hebt das Niveau des Kollektivs direkt am Ende der Veranstaltung auf das Notenniveau einer „zwei“, die üblicherweise bei Rängen zwischen 80 und 90 % vergeben wird (Daten von insgesamt 1056 Antworten zu Prüfungsfragen, je zwei pro Studierende und Zeitpunkt, siehe auch [Tab. 6].