Improvement of Radiological Teaching – Effects of Focusing of Learning Targets and Increased Consideration of Learning Theory Knowledge

Anmerkung

Teile der Arbeit entstanden im Rahmen einer Projektarbeit von Stefan Wirth im Rahmen seines Studiums zum Master of Medical Education.

Abstract

Purpose Based on evaluation and examination results of students, a necessity for improvement of so far purely instructor-based radiological teaching at the local institution was determined. Aim of our study was to use one out of eight seminars to exemplify adaptation of the teaching concept according to learning theory knowledge, to determine the resulting effects and to interpret them.

Materials and methods The institutional review board approved the prospective study of the seminar conversion, which was performed after the end of the winter semester 2015/2016. Didactically, this included a course split into online preparation, attendance phase and online follow-up with integration of interactive scaffolding, practice-oriented clinical teaching according to Stanford, Peyton skills transfer and extensive feedback into the attendance phase. At the beginning and at the end of each course, each student filled in identical, standardized questionnaires (n = 256 before and after conversion) using a 5-point Likert scale (1: very good; to 5: deficient) and additionally answered two randomly chosen written examination questions from a content-adapted questionnaire pool of the last five years. For statistical evaluation, the Mann-Whitney U-Test was used for evaluation data and Fisher's Exact test for exam questions.

Results Before/after conversion, the subjective total evaluation score of students was 3.22 (mean value) ± 1.51 (standard deviation) / 1.66 ± 0.78 (p < 0.001) and the objective proportion of correctly answered examination questions in the respective cohort at the beginning of the seminar 37.7/53.9 % and at the end of the seminar 55.1/84.6 % (p < 0.001).

Conclusion The conversion of the test seminar resulted in both a better evaluation of the teaching unit by the students (evaluation) and a considerably higher rate of correctly answered examination questions from past state examinations (learning success). This supports transferring the concept to comparable teaching units.

Key points 

• Radiological teaching allows integration of current learning theory concepts with reasonable effort.

• In a test seminar this improved the evaluation results of the teaching unit by the students.

• In addition, this also led to a higher rate of correctly answered examination questions from past state examinations.

• This supports further steps towards excellent radiological teaching.

Citation Format

• Wirth S, William Y, Paolini M et al. Improvement of Radiological Teaching – Effects of Focusing of Learning Targets and Increased Consideration of Learning Theory Knowledge. Fortschr Röntgenstr 2018; 190: 161 – 174

Key words

radiological and medical education - curricular improvement - emergency radiography of the chest

Introduction

Since the establishment of a Chair for Medical Didactics and Educational Research [1] at the studied university, the average score of the students in the state examinations increased from 3.11 in 2010 to 2.63 in 2014. Country-wide this corresponded to 11th place of 36 universities in the written part of the second state examination [2]. Nevertheless, an internal comparison of the correct solution rate of the written state examinations between 2011 and 2015 for questions with radiological content mainly resulted in a below-average rate of correct solutions.

The teaching units in radiology were not practical, with little structure and inadequate coordination. Likewise, the seminars and practica were almost exclusively instructor-centered courses and in the form of lectures. It was primarily up to the instructional staff to determine the depth of course content as well as the presentation method. In addition, substantial time was lost in the repetition of material from previous semesters. Students indicated that the goal of radiological teaching units was frequently unclear, that they desired more hands-on activity, and that mastery of the material could not be assessed.

On the whole, a fundamental need for the change of radiological coursework was determined. Since, for personnel reasons this could not be implemented across the board for all radiological teaching units, an initial step was undertaken in the form of a test course, the success of which could be a motivation to apply the changes to other teaching units. In consultation with the local medicine didactics organ, it appeared to be useful to orient teaching toward concrete clinical conditions and needs. Earlier publications have supported this and particularly report the success using interactive, case-based learning and e-learning [3] [4] as well as peer-to-peer feedback [5]. The following uses terminology relating to medicine didactics, therefore refer to [Table 1] for a short overview.

Table 1 Synopsis and brief explanation of medical-didactic terms.

term	explanation	goals
blended learning	Learning form and organization, which offer a didactically meaningful linking of traditional face-to-face instruction and modern forms such as E-learning. The concept combines the effectiveness and flexibility of electronic learning forms with the social aspects of personal communication as well as the practical learning of skills.	Suitable combination of different media and methods to enhance their advantages and minimize their disadvantages.
brainstorming	Method for coming up with ideas. A group to address influential parameters of a specific issue, mixed according to age, expertise, hierarchical position, ideally containing 6 to 8 persons, to collect as many ideas possible in a short amount of time. Free association and imagination are not only allowed but desired. On the other hand, evaluating and judging the ideas should not be permitted.	Generation of new ideas in a group with regard to a given issue or topic.
buzz group	Murmur group. Two to four students form a group and quietly exchange opinions on a specified topic. After expiration of a specified period of time, the results of the individual groups are shared with the plenary group. Frequently the buzz groups represents a transition between the instructor-centered format and the interactive format.	Use of prior knowledge, lowering of inhibitions to active participation by the student.
e-learning	Teaching forms that use electronic or digital media for the presentation or distribution of learning materials.	Independent of location or time.
feedback	Group dynamic method to respond to a person regarding behavior. This includes how other participants perceived, understood and experienced this person.	Awareness of weaknesses and strengths. Repetition of core content.
flipped classroom	Pedagogical model in which the typical lecture and homework elements of a course are reversed. Instead of learning during class time and practicing it at home, material is worked on at home and practiced in class.	Refreshing fundamentals, avoiding intervening questions.
peyton	Method of acquiring practical skills. First, instructors demonstrate the skill at a normal speed without commentary. Then instructors repeat the procedure slowly while explaining the individual procedural steps. In the third step students explain to the instructors the individual procedural steps. In the final step the students perform the procedure independently.	Transmission of practical skills, in this case diagnosis and assessment.
peer-to-peer	Communication at the same level, in this case, student-to-student.	Increase of acceptance, inhibition reduction.
scaffolding	Support of complex learning processes in the form of instructions, concepts and other help provided by instructors. In this way, the knowledge of the learner can be addressed individually and complex tasks can be divided into simple controllable segments.	Simplification of complex learning processes, individualization and time management.
swot	Strategic planning tool to analyze possible internal (in this case the concrete course of teaching units) and external effects (in this case radiology, the clinic or teaching as a whole) of project implementation. Strengths, Weaknesses, Opportunities and Threats are analyzed.	Project enhancement at the start of the planning phase

The description of radiography of the thorax (chest X-ray) is common, complex, typical and due to its relevance in the clinical routine, is a common element in medical certification examinations [6]. Unfortunately the diagnostic performance of students and the medical profession in the early stage of training is frequently unsatisfactory [7] [8]. This is particularly true under emergency conditions such as in the intensive care unit; therefore the seminar “Radiological Imaging in the Intensive Care Unit” appeared to be a suitable test course.

Goals

The goal was to improve the seminar “Radiological Imaging in the Intensive Care Unit” by intensifying the focus on content and orientation according to theoretical knowledge, to identify the resulting effects and to interpret them.

The issue was to determine the results of coordinated learning goals and increased focus as well emphasized orientation toward learning theory when compared to the semester prior to this reorientation. This might be measured by a significant improvement (p < 0.05) in the subjective assessment by the students by at least 0.5 points in a 5-point Likert scale and/or at least a five-percent objective improvement in the results of associated state examination questions.

Materials and Methods

The local ethics commission approved the project under number 17 – 036 UE as a prospective study.

General Learning Goals and Introduction in the Curriculum

The overarching learning objectives of all radiological coursework are (a) imparting knowledge regarding the representation of normal and altered anatomy in images produced by various radiological procedures, (b) proficiency in systematic descriptive diagnosis, (c) competence in prioritized assessment, and (d) competence in the evaluation of indications in certain radiological procedures with and without contrast media. At the studied institution, six clinical modules (semesters) follow two years of preclinical studies. Module 1 is devoted to the principles of clinical medicine covering the radiological basis of (a) with respect to normal anatomy, (b), as well as partially regarding (d). Based on this, the interdisciplinary basic year (modules 2 and 3 combined as a one-year module 23) contains eight radiological seminars and one practicum [9]. Beyond that, radiology is available only as an elective subject in the Practical Year and in the context of a review course. This division is also oriented, for example, toward the radiological curricula recommended by the German Radiological Society with a more modality-oriented basic section and a competence-oriented second part [10].

Topics, Target Group, Format and Framework

The original teaching unit “Radiological Imaging in the Intensive Care Unit” is devoted mainly to supine radiography of the chest, but also includes typical CT findings such as cerebral hemorrhage, pneumonia, abscess, and typical ultrasound findings such as cholestasis, cholecystitis, pleural effusion, pericardial effusion and free abdominal fluid.

The 90-minute seminar is a required course with one-time participation of students in the second or third clinical semester. The class takes place in a daylight-free, dimmable, small lecture theater with 40 seats and is offered by each of the four lecturers 4 times per semester and is attended by about 16 students each.

Conversion of the Selected Test Seminar

In order to be able to assess the impact of the seminar changeover, a SWOT analysis was carried out involving students, lecturers and decision-makers ([Table A, B], Online Supplement).

Prior to the change, the course contents showed distinct overlap with other teaching units (e. g. abdominal CT, trauma seminar, interventional radiology, etc.). In principle, imaging in the intensive care unit can involve all modalities. Using matrix analyses [11] [12], those involved in the restructuring of the coursework developed a consensus regarding the most important radiological modality with the least overlap with other modalities ([Tables 3a, b]). In addition, a comparison of the learning contents and objectives with other teaching units indicated that all parts outside “supine chest X-ray” were either already represented in other coursework or could be integrated there with minimum effort. Therefore the subject area was limited to radiography in supine position and the course was renamed accordingly.

The modified Seminar

The learning objectives of the redesigned seminar “Supine Chest X-ray” are given in [Table 2]. These are distributed on the basis of the online preparation, face-to-face segment and online follow-up, and are toward the German “Competence-based learning goal catalog for medicine” (NKLM) [13] and further classified according to knowledge (K), Skills (S), Competencies (C) and Attitudes (A).

Table 2 Learning goals.

phase	type[1]	learning goal
preparation
	K	Can name the differences between a supine chest X-ray and 2 planes of an upright chest X-ray.
	K	Can provide quality criteria and normal anatomy for a supine chest X-ray.
	K	Can provide a scheme for the structured assessment of a supine chest X-ray.
	K	Can identify the classification of the certainty of results (certain, suspicious, questionable, exclusion).
	K	Has viewed an example of the manifestation of typical pathologies (see below).
face-to-face
	K S	Can provide quality criteria for a supine chest X-ray and normal finding.
	C	Can apply a scheme for the structured assessment of a supine chest X-ray.
	C	Can apply the classification of the certainty of results from the online preparation.
	For each pathology group: (1) pneumothorax, tension pneumothorax, (2) atelectasis, ventilation abnormality, (3) pneumonia, typical and atypical, (4) congestive heart failure, congestion, hyperhydration, pleural effusion, pulmonary edema, (5) pericardial effusion (6) pulmonary embolism, (7) misplaced implant (8) example of co-pathology.
	S	With help identifies manifestation in supine chest X-ray.
	S	With help can identify normal and diseased changes.
	C	With help can estimate the extent and urgency of a finding.
	C	With help can make acute diagnostic and therapeutic decisions.
	C	With help prioritize and structure with indication of certainty.
	S C A	Can accept feedback from instructor/peer and can provide feedback to peer.
follow-up
	S	Can identify pathologies from the face-to-face segment in sample images
	S	Can systematically describe sample image.
	C	Can systematically assess sample images from a case.

¹ K: Knowledge, S: Skill, C: Competence, A: Attitude.

The online preparation is a ten-page Word document, about half of which contains repetition of previous content and in the other half prepares for the face-to-face segment. The face-to-face and follow-up phases are based on PowerPoint slides. The face-to-face segment begins in the instructor-centered format for about 20 minutes and introduces the case-based learning through an example, which is used as an example by the lecturers. The students undertake the remaining seven cases in various roles, including: finding, feedback on finding, assessment, feedback on assessment. Finally the lecturers again provide feedback on the individual roles and provide supplemental examples of the same pathology. For the face-to-face segment of the new course format, [Fig. 1] provides a visual explanation of the changing activity levels of the students (sandwich structure [14]). Central elements are interactive, clinically-oriented case studies following the Stanford clinical teaching model [15], as well as a special focus on feedback [16]. During the teaching unit, instructors reduce their help while the students work out the problems in a process referred to as “scaffolding” [17]. In this way instructors support the learning process by providing an initial orientation in the form of instructions, concepts and other assistance. [Table 3] provides the new sequence, and [Fig. 2] provides a practical example.

Table 3 Course schedule.

phase	articulation point[1]	duration	method	content	ICAP[2]
preparation		25 min	online / moodle
		20 min	self-study	Repetition of foundations (PDF, 10 pages)	P
		5 min	video	Demonstration of normal findings of chest X-ray 1 plane	P
face-to-face		90 min	seminar
				Getting started
	A	1 min	instructor	Greeting and introduction	P
		1 min	eye catcher	Emphasis on clinical relevance of topic	P
		2 min	instructor	Introduction of learning goals, structure, content and sequence of course	P
				Theoretical principles of thoracic X-ray in 1 plane
		7 min	lecture	Technique, quality criteria, feedback rules, finding scheme	P
	B1	2 min	buzz group	Form/content of a written finding report?	A
	C1	3 min	synchronization	Query results, consolidation and resolution	C A
		4 min	lecture	Normal finding, differences between supine and upright X-ray	P
				Cased-based learning Case 1 according to Peyton steps 1 + 2
		4 min	lecture	Example: Case 1 with explanation of further sequence	P
		each	scaffolding	Case-based learning Cases 2 through 8 according to Peyton steps 3 + 4 (including 3 min half-time break)
		2.5 min	activity S1	Structured finding by students	C A
		1 min	activity S2	Prioritized assessment by other students	C A
		1 min	activity S3	Peer-to-peer feedback (FB) on S1, S2	I
		1 min	interaction	Resolution and selected instructor feedback on S1–S3	I
		2 min	lecture	Further examples and explanations of the pathology	P
				Finishing up
		3 min	lecture	Summary
	B2	2 min	buzz group	Take-home items	C A
	C2	4 min	synchronization	Query results, consolidation and resolution	I
	D	1 min	lecture	Explanation of follow-up, send-off	P
		1 min	questionnaire	Evaluation	A
		1 min	eye catcher	Reviewed emphasis on clinical relevance of topic	P
		1.5 min	buffer
follow-up		35 min	online / moodle
		30 min	self-study	Approx 2 – 4 examples for each of 8 pathologies (PPT[3])	C A
		5 min	multiple-choice test	Achievement control without grading	A

¹ The articulation point indicates the phase of group work (A: no group work, B: entry into group work, C: synchronization of the group, D: end of group work).

² ICAP indicates the cognitive level of engagement of the students (I: interactive, C: constructive, A: active, P: passive).

³ The cases are provided as a PowerPoint file (PPT) in Moodle, a web-based institutional organization platform for students.

Fig. 1 Course of the seminar "X-ray of the chest in supine position" during the presence time after adaption according to learning theory knowledge. Green components represent activity of the lecturers, whereas this is marked for the students in Apricot.

Fig. 2 Sample pathology "Congestion", taken from the seminar after conversion. a Clinical indication with corresponding image to be processed of one particular pathology group. One student does the reading, a different one the interpretation. If required, they receive help in each task. Subsequently, two other students give feedback on both tasks. b Dissolution of the case by providing the correct interpretation together with a corresponding CT image, followed by a last feedback by the teacher. c further clinical course. e, f Further examples of the same pathology group presented by the teacher.

The means of acquiring diagnostic abilities and assessment are based on Payton’s methodology [18] [19] which involves moving through four steps. During the first step (“Demonstration”), instructors demonstrate the relevant skill at a normal speed without commentary. In the second step (“Deconstruction”), instructors repeat the procedure slowly while explaining the individual procedural steps. In the third step (“Comprehension”) students explain to the instructors the individual procedural steps with some help from the instructors. In the fourth and final step (“Performance”) the students perform the procedure independently.

Follow-up, just as preparation, is optional. Additional cases are presented as PowerPoint files, each case consisting of two slides. The initial slide presents clinical information, issues and an X-ray image. Here the students can check their abilities, and with the second slide can review the finding, assessment and additional explanations.

The instructors received general pedagogical training in their continuing education and were involved in the restructuring process. Prior to the start of the new format course, two joint test classes were held with Practical Year students. In addition, SW conducted the initial classes which were observed at least once by each additional instructor as silent observers.

The total time required to restructure the seminar about four person-weeks.

Data Collection, Processing and Evaluation

At the end of each face-to-face course, the students filled out identical, standardized questionnaires using a 5-point Likert scale (n = 256 in the semester before and after conversion). In addition, at the start and finish of the course they answered two randomly selected written questions from a group of 50 state examination questions from the previous five years. The studies directorship took into account those questions with a particular relevance for intensive care patients and the content of which was not a separate focus of other teaching units. The resulting pool of questions was identical before and after the seminar. The focus was on supine chest X-ray, under makeshift conditions as well as subsequent CT. Both modalities made up about 90 percent of the questions; the rest had to do with other modalities, generally including ultrasound.

The evaluation questionnaire consisted of 31 questions, 30 of which used a Likert scale where 1: “Completely applicable” and 5: “Completely inapplicable”. The 31st question asked for an overall evaluation following the grading system where 1: “Very good” and 5: “Insufficient”. [Table 4] lists the questions used in an abridged form and shows how the resulting blocks of topics “Personal motivation”, “Learning climate”, “Instructor”, “Concept and material”, “Understanding and learning success” and “Overall grade”. All 31 questions are shown in the results section of [Table 5].

Table 4 Thematic grouping of questionnaire items for the subjective evaluation.

item no.	item text (shortened)	topic group
1	Personal motivation is high	personal motivation
2	Content relevant and important
5	Preparation for course content necessary
9	Follow-up necessary for course
26	High interest in content
3	My knowledge of content is high	understanding and learning success
27	Course developed understanding of content
28	Can provide course content
29	Course makes preparation for examination easier
12	Learning goals were clear	concept and material
13	Identifiable common theme in course
14	Course time sufficient for processing content
15	Level of difficulty exactly right
17	Examples/images helped in understanding
22	Sufficient opportunities for active participation
23	Remarks on critical analysis of content
25	Scope of interactivity exactly right
16	Instructors made complicated material understandable	instructor
18	Instructors’ explanations easy to follow
19	Instructors tried to share knowledge
20	Adequate interaction of instructors and students
21	Instructors responded well to questions
24	Constructive learning atmosphere	learning climate
30	Course was fun
31	Course grade	overall grade

Table 5 Detailed results of the subjective evaluation.

item no.	item text (somewhat shortened)	before conversion[1]		after conversion1
		mean	standard deviation	mean	standard deviation
1	I am highly motivated to continue learning the material after the course	3.32	1.49	2.29	0.98
2	In my opinion, course content is relevant and important	2.00	0.91	1.39	0.61
3	My level of knowledge of the content of the AINS block in Module 23 is high	3.67	1.20	2.61	0.92
4	My level of knowledge of the necessary basic radiological principles is high	3.76	1.17	2.76	0.87
5	In my opinion, you should prepare for the course	2.09	1.154	2.14	0.99
6	The supplied learning material helped me to prepare	[2]	2	1.74	0.90
7	Preparation for the course should mainly contain radiological basic principles	1.81	1.00	2.39	1.04
8	Preparation for the course should mainly contain sample images	1.55	0.85	1.79	0.83
9	In my opinion today’s course should be followed-up	2.06	1.19	2.75	1.19
10	Follow-up should mainly contain radiological basic principles	2.13	1.30	2.90	1.26
11	Follow-up should mainly contain sample images	1.64	0.93	1.87	0.85
12	The learning goals were clear	2.29	0.92	1.88	0.94
13	I perceived a clear common theme thought out the course	3.20	1.63	1.70	0.91
14	During the course I had sufficient time to completely work through all the content	3.41	1.49	1.93	0.93
15	The level of difficulty of the course was exactly right	3.11	1.62	1.83	0.82
16	I my opinion, the instructor made difficult material easier to understand	1.30	0.57	1.58	0.79
17	The examples, images or cases helped me understand the content	2.07	0.871	1.48	0.66
18	I could easily follow the instructor’s explanations	1.32	0.60	1.63	0.93
19	It was important to the instructor that the students learn something	1.20	0.49	1.38	0.70
20	Behavior of the instructor was professional and appropriate	1.16	0.41	1.31	0.63
21	Instructor responded well to questions and comments	1.17	0.45	1.34	0.63
22	There was sufficient opportunity to get actively involved	2.77	1.82	1.36	0.66
23	I was encouraged to critically analyze the course content	3.09	1.64	1.81	0.83
24	I found the learning atmosphere in the course to be constructive	2.15	0.87	1.89	0.95
25	The scope of interactivity in the course was on the whole exactly right	2.95	1.71	1.88	0.94
26	I was interested in the content of the course	1.48	0.77	1.68	0.76
27	The course increased my understanding of the content	2.97	1.67	1.75	0.79
28	I can provide an overview of the course content	3.01	1.64	1.82	0.81
29	Attending the course makes preparation for the examination easier	3.31	1.46	1.99	0.92
30	On the whole the course was fun	2.43	0.80	1.85	0.93
31	On the whole I give the course the grade of	3.22	1.51	1.66	0.78

¹ Questions 1 through 30: Likert scale where 1: “Completely applicable” and 5: “Completely inapplicable”. Question 31: school grading system where 1: “Very good” and 5: “Insufficient”.

² Question 6 was posed only after the conversion, since previously no leaning material had been made available.

For the assessment of the evaluation sheets, mean value differences of more than 0.5 were defined as “relevant” and those with more than 1.0 as “highly relevant”. On the other hand, with respect to the state examination questions, the percentage of correct and incorrect answers in the cohort were determined, and changes of more than five-percent were considered relevant and highly relevant if greater than ten percent. The background of both considerations was that for ratings based on the school grading scale, changes of 0.5 would lead to a change in grades for half of the cohort, and this would be the case for changes of 1.0 for the entire cohort. Similarly, in the case of the state medical examinations, the standard case of the 60 per cent passing grade threshold (that is, without indexing correction to avoid a greater than 22 percent failure rate) would result in grade escalation in ten percent increments. The Mann-Whitney U test was used for independent samples for the statistical testing of differences in data of the evaluation sheets and, in the case of the test results, Fisher's Exact Test was used. With regard to the requirements for multiple tests, question 31 and the evaluation of the test results were considered to be not formally related to the other evaluation, and the Bonferroni method was used to adjust the threshold significances [20]. This resulted in a threshold of P < 0.05 for the overall score, P < 0.01 for each of the five other subject blocks of the evaluation surveys, and P < 0.01 for the state examination questions.

Results

The results are shown in [ Tables 5], [6].

Modification of the teaching unit resulted in a slight improvement in the rating of “Personal motivation” from 2.19 to 2.05; there was a significant improvement for “Understanding and learning success” from 3.24 to 2.01 as well as for “Concept and material” from 2.85 to 1.69. Assessment of “Instructors” showed a slight downgrade from 1.23 to 1.44. “Learning climate” improved from 2.29 to 1.87 as well for “Overall grade” which demonstrated a highly-significant improvement from 3.22 to 1.66. The results of the individual questionnaire items are shown in [Table 5].

Table 6 Summary of results.

topic group	statistics[1]	time point seminar	before conversion	after conversion	p-value
questionnaires (subjective)			n = 256	n = 256	Mann-Whitney U test
personal motivation	MV (SD)	End	2.19 (0.46)	2.05 (0.62)	0.001
	MD (25 % / 75 %)		2 (2 / 2)	2 (2 / 2)
understanding and learning success	MV (SD)	End	3.24 (1.41)	2.04 (0.63)	< 0.001
	MD (25 % / 75 %)		2,5 (2 / 5)	2 (1,5 / 2,5)
concept and material	MV (SD)	End	2.85 (1.39)	1.69 (0.56)	< 0.001
	MD (25 % / 75 %)		2 (2 / 4)	1.5 (1 / 2)
instructor	MV (SD)	End	1.23 (0.40)	1.44 (0.60)	< 0.001
	MD (25 % / 75 %)		1 (1 / 2)	1 (1 / 2)
learning climate	MV (SD)	End	2.29 (0.73)	1.87 (0.85)	< 0.001
	MD (25 % / 75 %)		2.5 (1.5 / 3)	2 (1 / 2.5)
overall grade	MV (SD)	End	3.22 (1.51)	1.66 (0.78)	< 0.001
	MD (25 % / 75 %)		3 (2 / 5)	2 (1 / 2)
examination questions			n = 512	n = 512	Fisher’s Exact-Test
	Correct (%)	Start	37.7	53.91	< 0.001
	Correct (%)	End	55.11	84.6	< 0.001
			< 0.001	< 0.001	p-value

¹ A cross-wise comparison of correct answers of exam questions at the end of the seminar before conversion to the results at the beginning of the course in the new format revealed a p-value of 0.754 for double-sided and 0.377 for single-sided testing when applying Fisher's Exact Test. MV: Mean Value; SD: Standard Deviation.

As a consequence of the seminar restructuring, the rate of correctly answered state examination questions improved significantly at the start of the face-to-face segment from 37.7 % to 53.9 %, and at the end of the teaching unit substantially from 55.1 % to 84.6 % ([Fig. 3]).

Fig. 3 Proportion of test questions correctly answered by the students before and after conversion of the test seminar. After conversion, the course already starts at the level with which this ended before conversion. In addition, the change resulted in a significant improvement during the event and raised the level of the collective right at the end of the event to the grade of a "two", which is usually given for ranks between 80 and 90 percent (data from a total of 1056 answers to questions, two per student at each time, see also Table 6).

[Table 6] contains an overview of all important results.

Discussion

The student protests at the end of the 1990 s called attention to problems in clinical teaching. The core of the criticism centered on theory-focused instruction and little hands-on orientation [21]. From that point on there was rethinking of current practices and nowadays almost every medical program has a separate organization for medicine didactics. The effort to improve teaching affects the course of study in general and therefor also concerns the individual departments that are specifically responsible for the implementation of changes. In addition, university clinics are responsible for a very large part of medical training.

The institution in question had itself determined a need for change, and in view of fixed resource allocation and presumed overall expenditure, first decided to restructure a radiological teaching unit as a test. Overlapping courses posed a particular challenge as well as a different sequence of seminars relating to the “AINS” blocks (Anesthesia, Intensive and Emergency medicine, Pain therapy) containing the test seminar. In addition, the radiological teaching units cover interdisciplinary topics in a relatively early stage of medical education. In addition there was a desired orientation to the white paper “Radiological Curriculum for Undergraduate Medical Education in Germany” by the German Radiological Society (DRG) and the Association of Chairmen in Academic Radiology (KLR) [10]. Taking into account proven learning-theory-based knowledge, first a preparatory part was developed to provide or refresh key knowledge in order to make the face-to-face time more effective (inverted or “flipped classroom” [22] [23], a model designed to make face-to-face learning more efficient). With the exception of these concepts, which rely on technical requirements and therefore comparatively up-to-date areas of knowledge in pedagogy and didactics, only established and known methods have been used. The results demonstrate the potential these proven and often simple means can have if applied concretely and consistently. At the studied institution, in consultation with the local medicine didactics organ, it appeared to be useful to orient teaching toward concrete clinical conditions and needs. Although publications on teaching radiology which go beyond an interdisciplinary approach while involving anatomy are only available in comparatively small number, such studies particularly found positive effects of interactive case-based learning and e-learning [3] [4] as well as peer-to-peer feedback [5].

In the new format, targeted communication [24] [25] takes place in several phases during the face-to-face time. Right at the start, an “eye catcher” illustrates the particular clinical significance of the topic. The introductory instructor-centered section stipulates concisely-formulated learning objectives, emphasizes personal benefit to the student and presents the agenda. Within the framework of the first activity phase, the students developed a “buzz group” in which two to four students briefly discuss a question, an image, a problem, etc. and thus become independently active while lowering the inhibition threshold for cooperation. The activity was a generally-applicable structure of a finding report, itself a structured application of the learning objectives.

Two-and-a-half minutes for structured diagnosis report by students is not much time, but still significantly more than is available in the clinical routine. One of the strengths of the concept after conversion is scaffolding which allows the instructor to step in at any time when time is running out. At the start of the class, there is no expectation of completeness of the findings report; instead the instructor intervenes and provides assistance, and continues to do so in the course of the class. This is possible since the exercise and the repetitions via feedback make this decreasingly necessary.

The main goal of promoting understanding and retention was to improve the conception, content and workflow as well as the promotion of active learning. The online preparation reinforces basic principles, thus saving time through elimination of unnecessary repetitions and supports the prerequisites for achieving the learning goals. The face-to-face segment improves the learning climate, and the more effective targeted communication increases attentiveness. The deliberate limitation of the introductory instructor-centered section is based as much on learning psychology as the subsequent, consistent activity changes with clear work instructions. The focus was on the group work approach so that it could be mastered by different instructors. The group phase at the beginning helps to determine meaningfulness, and at the end supports retention by students individually working out take-home items. The core of the face-to-face segment is the effective, recursive structure based on cases with scaffolding-supported skills mediation based on Peyton [17] [18] [19]. The third step of the approach in which students direct the instructors who respond accordingly can be only somewhat reproduced. Feedback tries to compensate for this, provides additional activity, supports the estimation by the students, enforces repetition of core content while allowing learning from observable errors [5] [16] [26]. The online follow-up work offers more depth with alternative case examples.

The training measures for instructors also proved to be exemplary. With the given limitation of the number of methods, training required little additional effort. In general, this consisted of medical-didactic continuing education for all instructors, a one-hour explanation of the new methods followed by a test run-through of the course for all instructional staff as well as participation as observers during the initial seminars in the new format.

The method according to Kern [27] has been established for curricula, and quality control standards have been defined by the World Federation for Medical Education [28]. After each semester, the student evaluations should be used for continuous improvement, evaluated and reviewed in the relevant radiological curriculum committee during the following quarter, discussed at an institutional conference also with regard to remarks by the instructors. Finally, responses should be chosen, documented and monitored.

The overall assessment of the modified teaching unit improved significantly by 1.56 points, thus placing it in the range of the top seventh of the scale. With regard to the individual results of different subject groups of further subjective evaluation, they were also all significant but differed in their relevance. The evaluation of the instructors declined by 0.19 point. This could be explained by a very good baseline level of 1.22 and possible uncertainties regarding the initial seminar implementation in the new format. The “Personal motivation” score of the students improved by 0.14 points as well as “Learning climate” which increased by 0.42 points just below the defined relevance threshold which was already at a good level. Highly relevant improvements were found for “Concept and material” (1.16) points as well as for “Understanding and learning success” (1.2 points); both are now at a good level.

Since multiple-choice questions do not test skills or competencies within the framework of the NKLM, their value in examinations is controversial, even though multiple choice questions have been standardized and used in the state examinations. For this reason, questions from written medical examinations were used in this study to obtain a more objective measure of the impact of the seminar conversion. The increase in the rate of 29.5 % of correctly answered examination questions confirmed the subjective results of learning success. Measured against the grade thresholds of the last written medical state examination [29], this would mean that both cohorts prior to the start of the seminar were rated “5” and the semester group prior to the conversion remained at this grade level after the seminar, whereas the group after the modification was rated as “2” directly after taking the seminar. Sustainability of this success should be promoted through the provision of follow-up, which had been little used to date, however, possibly due to a start-up effect. Although “supine chest X-ray” questions were the main focus, more than a third of the questions addressed CT. We can explain the success rate of questions dealing with radiography due to the course oriented toward the new format. With regard to CT as the second most important modality, this is presumably due to the extensive use to illustrate actively compiled pathologies and also as a result of supplementary examples.

After establishment of the conversion, a further improvement could be intensification of the self-learning phase in the online preparation to provide factual knowledge via blended learning concepts such as the flipped classroom [22] [23]. In this case, and especially in the online follow-up, an increased integration of the well-known advantages of e-learning [4] [30] would probably be varied and promising. To allow each student to assume each role at least once (finding, assessment, accepting and giving feedback on the finding and assessment), a limitation on the number of cases to be worked on would be necessary. Concretely this would mean eight cases, one of which would be handled by the instructor, thus limiting participation to seven students.

Limitations

This study is based on one seminar of an institution, therefore there are no reliable findings that allow a transfer of concept to other coursework or institutions. SW, one of four instructors, undertook the restructuring of the teaching unit; additionally, SW is the direct supervisor of the other instructors. Although effort has been made to avoid resulting bias, it cannot be fully ruled out

Multiple-choice questions were used to determine the student’s success; these queried knowledge but no skills or competencies according to the NKLM, and provided only a limited testing modality for the new format seminar.

The composition of the cohorts was also not checked for comparability. Although the number of 256 students per semester could support the presumption of a large balanced random sample, this cannot be assured. The same applies to seasonal effects, since there was both a summer and winter cohort.

Provision of follow-up is presumably supportive, but the extent of a sustained positive effect is still unknown.

Conclusions

Restructuring a test seminar in radiology with coordinated and focused learning targets as well as an increased inclusion of concepts of learning theory resulted in a valuable improvement and provided the motivation to apply the concept to comparable teaching units.

Possible financial advantages could result in the event of performance-oriented allocation of funding. Although this is certainly not the purpose of university teaching, restructuring courses might at least be effected without additional costs.

Further potential can be realized from increased integration of e-learning in the online preparation and follow-up, from a departmental or even institutional quality management approach to continuous improvement and in a reduction of group size.

Online Supplement

Table A SWOT analysis* of strengths, weaknesses, opportunities and threats with respect to changing the seminar "Radiological Imaging on Intensive Care Units" in "X-ray of the Chest at the Intensive Care Unit", including intended counter-measures.

	helpful	damaging	measures / consequences
internal factors	Strengths S2: Broad clinical relevance S2: Strong practice orientation S3: Optimized learning theory orientation S4: Grateful for images as visual stimuli S5: Inclusion of critique (evaluations) S6: Inclusion of students (questionnaire) S7: Improved learning climate S8: Motivated instructors S9: Equipment of one of the largest radiology departments in EU	Weaknesses W1: Poor acceptance by students W2: Invested time missing elsewhere W3: Meaningfulness for students not apparent W4: Low learning success W5: Basic principle issues hinder process flow W6: Limited examination component W7: Anxiety/bashfulness of students W8: Too narrow focus on X-ray W9: Pathologies too typical	Strategy W1: Use of S1-S8, O1, O3, O8, O9 W2: S8, S9, O6 W3: S1-S8, O1-O3, effective comm. of learning goals W4: S3, O1, individual aspects of 3 rd conception W5: Online preparation W6: O1 negotiation with module block team W7: S7, S8, O2, O8 W8: Module team learning goal coordination W9: S3, O3, online follow-up
external factors	Opportunities O1: Better long term learning goal achievement O2: Promotion of general teamwork O3: Increased interest/understanding of RAD O4: Motivation of subsequent applicants O5: Enhancing radiology in teaching O6: Motivation for teaching on direction level O7: Format application to other courses O8: Increase student self-esteem O9: Diverting course	Threats T1: Change not ready in time T2: No transfer to routine T3: Students doing worse on examinations T4: Too little material taught T5: Demotivation of investment in teaching RAD T6: Demotivation of instructors T7: Drop in applicants for RAD as specialty T8: No funding T9: No time	Strategy T1: S8, O5, O6, project work by SW T2: S1-S9, O3, O4 T3: S1-S8, O1, Pareto principle, QM integr. T4: S3, O3, online preparation/follow-up T5: S1-S8, O1-O9 T6: S1-S3, S7, S8, O1-O7, O9 T7: see T5. T8: S8, O6 T9: see T8.

*Prioritized listing of the nine most valued items; Abbreviations: RAD: Radiology, EU: Europe, S: students, TU: teaching unit, QM: quality management.

Table B Overview of key challenges and corresponding solution strategies for the intended change of the test seminar.

Problem (status pre-conversion)	Solution (goal post-conversion)
Much overlap with other teaching units	Limitation to one modality, balance of learning goals
Basic questions interrupt flow	Opportunity for a review online preparation prior to the actual lecture-based teaching unit
Acquisition of the skill of fundamental diagnosis and assessment are necessary competences, the acquisition of which can be presumed at best	Practice orientation with Clinical Teaching following Stanford; provide example of normal finding and evaluation; encourage student activity; provide reduced help in competence acquisition during the course of the seminar following Peyton (scaffolding)
No support of “better” anchoring of important learning content	Multiple repetition of important learning content, including multidimensional feedback from students and lecturers
Concrete cases are motivating, but have been too diffuse and unrelated to the clinic and clinical operation	Increased integration of case-based and problem-based learning
Lecture component heavily emphasized	Horten lecture component, initially max. 20 minutes, followed by alternating activities of students and instructors with scattered explanatory components lasting up to 5 minutes each
Encourage students but don’t over-encourage Paying attention to individual students can consume different amounts of time	Simple and unified flow chart with scaffolding allows instructor intervention at any time
There are numerous variants of typical pathologies	Interactive consideration of a typical variant per pathology, followed by an explanation of additional examples of the same pathology as needed
There is no support for of sustainable knowledge, especially beyond the end of the seminar	Provision of online follow-up with further case studies on the pathologies considered during the lecture
There is no support of alternating activity level of the students	In addition to practical skills and peer-to-peer feedback, also inclusion of group elements
There is no support for method mastery and uniform application by all participating instructors	Limitations on methods. Selection of buzz group as sole method with respect to group element. Personal training of instructors and passive participation of instructors in the first class after introduction of the new format

Table C Matrix analyses for comparative assessment of the respective suitability of radiological examination modalities with respect to importance a and minor overlaps with other teaching units b.

a	US	CR	CT	MR	DL	Total
US	–	4	3	9	8	24
CR	6	–	5	9	8	28
CT	7	5	–	9	7	28
MR	1	1	1	–	4	7
DL	2	2	3	6	–	13
b	US	CR	CT	MR	DL	Total
US	–	2	3	5	7	17
CR	8	–	7	8	7	30
CT	7	3	–	3	4	17
MR	5	2	7	–	6	20
DL	3	3	6	4	–	16

In the direct cell-by-case comparison of modalities (US: ultrasound, CR: radiography, CT: computed tomography, MR: magnetic resonance tomography, FL: fluoroscopy, angiography and intervention), 10 points were split according to the respective question, a part between 0 and 10 assigned as the relative value of the modality in a row compared to the modality in a column (bold-face values). The corresponding modality comparison of the symmetrical matrix automatically contains values, which add to a total of 10 (modal-related switch of row and column, non-bold-face values). Finally, a total score per modality is formed by adding up values of one line (here with a maximum value of 40) which interprets as good (> 25, marked in green), medium (20 – 25, marked in yellow), poor (< 20, marked in red).

No conflict of interest has been declared by the author(s).

• References

• 1 Klinikum der Universität München. Institut für Didaktik und Ausbildungsforschung in der Medizin . 2014 Abruf von http://www.klinikum.uni-muenchen.de/Institut-fuer-Didaktik-und-Ausbildungsforschung-in-der-Medizin/de/impressum/index.html am 25.01.2017
• 2 Institut für medizinische und pharmazeutische Prüfungsfragen. Archiv: Lösungen und Ergebnisse der Prüfungen nach der Approbationsordnung für Ärzte. 2014 Abgerufen von https://www.impp.de/internet/de/archiv.html am 26.01.2017
• 3 Braun LT, Zottmann JM, Adolf C. et al. Representation scaffolds improve diagnostic efficiency in medical students. Med Educ 2017; Epub ahead of print
• 4 Heye T, Kurz P, Eiers M. et al. Eine radiologische Fallsammlung mit interaktivem Charakter als neues Element in der studentischen Ausbildung. Fortschr Röntgenstr 2008; 180: 337-344
• 5 Elshami W, Abdalla ME. Diagnostic radiography students' perceptions of formative peer assessment within a radiographic technique module. Radiography (Lond) 2017; 23: 9-13
• 6 Fotenos AF, Safdar NM, Nagy PG. et al. Unbiased review of digital diagnostic images in practice: informatics prototype and pilot study. Acad Radiol 2013; 20: 238-242
• 7 Christiansen JM, Gerke O, Karstoft J. et al. Poor interpretation of chest X-rays by junior doctors. Dan Med J 2014; 61: A4875
• 8 Salajegheh A, Jahangiri A, Dolan-Evans E. et al. A combination of traditional learning and e-learning can be more effective on radiological interpretation skills in medical students: a pre- and post-intervention study. BMC Medical Education 2016; 16: 46
• 9 Studiendekanat der Medizinischen Fakultät der LMU München. Medizinisches Curriculum München – MeCuM. 2014 Abgerufen von http://www.mecum-online.de/de/studium/mecum/index.html am 25.01.2017
• 10 Ertl-Wagner B, Barkhausen J, Mahnken AH. et al. White Paper: Radiological Curriculum for Undergraduate Medical Education in Germany. Fortschr Röntgenstr 2016; 188: 1017 - 1023
• 11 Charantimath PM. Total Quality Management. Dorling Kindersley; Pearson: 2011: 260
• 12 Rösch FM. Nephrologischer Unterricht an den 36 Universitäten in Deutschland – ein interfakultärer Vergleich. Dissertation Humanmedizin. Hamburg-Eppendorf: 2016
• 13 Hahn EG, Fischer MR. Nationaler Kompetenzbasierter Lernzielkatalog Medizin (NKLM) für Deutschland: Zusammenarbeit der Gesellschaft für Medizinische Ausbildung (GMA) und des Medizinischen Fakultätentages (MFT). GMS Z Med Ausbild 2009; 26: 2009-2026
• 14 Kadmon M, Strittmatter-Haubold V. et al. Das Sandwich-Prinzip--Einfuhrung in Lerner zentrierte Lehr-Lernmethoden in der Medizin. Z Evid Fortbild Qual Gesundhwes 2008; 102: 628-633
• 15 Ramani S, Leinster S. AMEE Guide no. 34: Teaching in the clinical environment. Med Teach 2008; 30: 347-364
• 16 Cantillon P, Sargeant J. Teaching Rounds: Giving Feedback in Clinical Settings. BMJ 2008; 337: 1292-1294
• 17 Pea RD. The Social and Technological Dimensions of Scaffolding and Related Theoretical Concepts for Learning, Education, and Human Activity. JLS 2004; 13: 423-451
• 18 Walker M, Peyton JWR. Teaching in the theatre. In: Peyton JWR. (editor) Teaching and learning in medical practice. Rickmansworth: Manticore Publishers Europe Ltd; 1998: 171-180
• 19 Nikendei C, Huber J, Stiepak J. et al. Modification of Peyton’s four-step approach for small group teaching – a descriptive study. BMC Med Educ 2014; 14: 68
• 20 Shaffe JP. Multiple Hypothesis Testing. Ann Rev Psych 1995; 46: 561-584
• 21 Jox RJ, Galambos P. Munich-Harvard-Alliance for Medical Education: Im Mekka der Medizin. Dtsch Arztebl International 2002; 99: A-1078/B-898/C-840
• 22 Leibniz-Institut für Wissensmedien (IWM) in Tübingen. Inverted Classroom. 2015 Abgerufen von http://www.e-teaching.org/lehrszenarien/vorlesung/inverted_classroom am 23.01.2016
• 23 Bösner S, Pickert J, Stibane T. Teaching differential diagnosis in primary care using an inverted classroom approach: student satisfaction and gain in skills and knowledge. BMC Medical Education 2015; 15: 63
• 24 Roff S, Aleer MC. What is educational climate ?. Med Teach 2009; 23: 333-334
• 25 Mullan PB, Lypson ML. Communicating your Program’s Goals and Objectives. J Grad Med Educ 2011; 3: 574-576
• 26 Kopp V, Stark R, Fischer MR. Fostering diagnostic knowledge through computer-supported, case-based worked examples: effects of erroneous examples and feedback. Med Educ 2008; 42: 823-829
• 27 Kern DE, Thomas PA, Howard DM. et al. Curriculum Development for Medical Education: A Six-step Approach. Baltimore: Johns Hopkins Press; 1998
• 28 World Federation for Medical Education. Globale Standards der WMFE zur Qualitätsverbesserung. 2003 Abgerufen von http://wfme.org/standards/bme/6-quality-improvement-in-basic-medical-education-german/file am 25.01.2017
• 29 Institut für medizinische und Pharmazeutische Prüfungsfragen (IMPP). Schriftlicher Teil des Zweiten Abschnitts der Ärztlichen Prüfung nach ÄAppO 2002: Lösungen und Ergebnisse 2016. Abgerufen von https://www.impp.de/internet/de/loesungen-und-ergebnisse/articles/bestehens-und-notengrenzen.html?file=files/internet_files/PDF/aktuelle%20Loesungen%20und%20Ergebnisse/AktM2H16APPO2012.pdf am 28.01.2017
• 30 Heiman HL, Uchida T, Adams C. et al. E-learning and deliberate practice for oral case presentation skills: A randomized trial. Med Teach 2012; 34: e820-e826