Using Formative Video Competencies and Summative In-Person Competencies to Examine Preparedness for Entry-Level Professional Practice

Abstract

A key element of competency-based education is assessment. Effective assessment requires access to a core set of expectations that match a learner's level of preparation. Miller's triangle provides a framework for establishing appropriate expectations that move learners from novice to entry-level clinicians. Formative assessment and feedback are a crucial part of facilitating learning in this context. A pilot investigation was conducted to examine the effects of a formative, video competency on performance in a summative, live competency. Rubrics were used to score performance on two competencies, an oral mechanism exam (OME) and a clinical bedside swallowing examination (CBSE). Performance on the OME was significantly improved in the summative competency, compared with the formative, video competency. Performance on the CBSE did not change from formative to summative competency. Assessment in competency-based education is important as a measure of readiness for entry-level practice. Formative assessment and feedback can improve preparedness and performance on summative competencies. Detailed, criterion-referenced assessment tools are crucial to identifying performance. While the OME rubric used in this investigation appears to meet that standard, it is likely that the CBSE rubric was not specific enough to detect changes.

Keywords

formative assessment - feedback - summative assessment - Miller's pyramid

Financial

No relevant financial relationships exist for either author.

Nonfinancial

The first author is an editorial board member for Teaching and Learning in Communication Sciences and Disorders. No other relevant nonfinancial relationships exist for either author.

• References

• 1 McAllister L, Lincoln M, McLeod S, Maloney D. Facilitating Learning in Clinical Settings. Cheltenham, UK: Nelson Thornes; 1997
• 2 McAllister S, Lincoln M, Ferguson A, McAllister L. A systematic program of research regarding the assessment of speech-language pathology competencies. Int J Speech Lang Pathol 2011; 13 (06) 469-479
• 3 Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990; 65 (9, Suppl): S63-S67
• 4 Lockyer J, Carraccio C, Chan MK. , et al; ICBME Collaborators. Core principles of assessment in competency-based medical education. Med Teach 2017; 39 (06) 609-616
• 5 Sadler DR. Formative assessment and the design of instructional systems. Instr Sci 1989; 18 (02) 119-144
• 6 Mercaitis PA. A supervisory method for prepracticum skill development. Clin Supervisor 1993; 11 (01) 179-187
• 7 Ho DW, Whitehill TL, Ciocca V. Performance of speech-language pathology students in problem-based learning tutorials and in clinical practice. Clin Linguist Phon 2014; 28 (1-2): 102-116
• 8 Council for Clinical Certification in Audiology and Speech-Language Pathology of the American Speech-Language-Hearing Association. 2020 Standards for the Certificate of Clinical Competence in Speech-Language Pathology. Available: https://www.asha.org/certification/2020-SLP-Certification-Standards 2018 . Accessed May 31, 2020
• 9 Tekian A, Watling CJ, Roberts TE, Steinert Y, Norcini J. Qualitative and quantitative feedback in the context of competency-based education. Med Teach 2017; 39 (12) 1245-1249
• 10 Hoepner JK, Zigler E. Collaborative counseling: using course-embedded clinical experiences to train motivational interviewing. 2020
• 11 Hoepner JK, Sather TW, Pakanich L, Hansen B. The effects of course-embedded clinical experiences on classroom learning outcomes. 2020
• 12 Hoepner JK, Sather TW. Teaching and mentoring students in the LPAA service delivery perspective. Perspectives of the ASHA Special Interest Groups (SIG 2). 2020 5. 397-413
• 13 Hoepner JK. Course embedded practical experiences: a reflection on innovation. Journal for Research and Practice in College Teaching: Special Issue: Innovative Teaching Personal Essays 2018; 3 (02) 185-190
• 14 Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. , International CBME Collaborators. The role of assessment in competency-based medical education. Med Teach 2010; 32 (08) 676-682
• 15 Hattie J, Timperley H. The power of feedback. Rev Educ Res 2007; 77 (01) 81-112
• 16 Hattie J. Influences on student learning (Inaugural professorial address, University of Auckland, New Zealand). Available at: www.arts.aukland.ac.nz/staff/index.cfm?P=8650 . Accessed May 31,2020
• 17 Swing SR. ; International CBME Collaborators. Perspectives on competency-based medical education from the learning sciences. Med Teach 2010; 32 (08) 663-668
• 18 McAllister S, Lincoln M, Ferguson A, McAllister L. COMPASS(R): Competency Assessment in Speech Pathology. Melbourne: Speech Pathology Australia; 2006
• 19 Ferguson A, McAllister S, Lincoln M, McAllister L, Owen S. Becoming familiar with competency-based student assessment: an evaluation of workshop outcomes. Int J Speech Lang Pathol 2010; 12 (06) 545-554