Translation and Transcultural Adaptation of the Milestones Instrument to Assess Teaching in Medical Residency Services on Orthopedics and Traumatology^[*]

Abstract

Objective Orthopedics is not very common in many Brazilian medical schools, and there is no questionnaire to assess the teaching of musculoskeletal disorders during medical training. The Orthopedic Surgery Milestone Project is an assessment tool for orthopedic residents in programs or fellowships recognized by the Accreditation Council for Graduate Medical Education (ACGME) and the American Board of Orthopedic Surgery (ABOS). This study aims to translate the Orthopedic Surgery Milestone Project into Portuguese and to perform its transcultural adaptation.

Methods The translation and transcultural adaptation consisted of the initial translation into Portuguese, back-translation into English, preparation of a pretest consensual text, and the subsequent elaboration of a final text.

Results The final text was deemed adequate and equivalent to the original one for the evaluation of orthopedics residents throughout their program.

Conclusions Given the lack of instruments for the assessment of orthopedic residents, the translation and transcultural adaptation of the Orthopedic Surgery Milestone Project were compatible; this can be an instrument for improved medical education.

Keywords

translating - transcultural adaptation - medical education - surveys and questionnaires

Introduction

Musculoskeletal disorders, including lumbalgia, lumbosciatalgia, and arthrosis, are frequent complaints in primary clinics, accounting for up to 25% of the reasons to seek medical care. The World Health Organization declared the period from 2000 to 2010 as the Bone and Joints Decade, thus confirming the importance of this topic in medicine.

Therefore, it is essential to train adequately the professionals who will meet this high demand. The development of skills in multiple care dimensions during the training of orthopedists and traumatologists is paramount to assure the proper treatment of patients. To do so, tools assessing the development of these skills are protagonists both to encourage individuals to acquire abilities they still do not possess and to provide data to improve the educational quality of residency programs.[1]

No instrument in Portuguese evaluates the teaching of musculoskeletal disorders throughout medical training. However, similar questionnaires, available for several medical specialties, have been created in other countries, including the Orthopedic Surgery Milestone Project. This instrument evaluates physicians from residency or fellowship programs recognized by the Accreditation Council for Graduate Medical Education (ACGME), a non-profit private institution that assesses and endorses all graduate medical training programs, including internships, residencies, fellowships, and subspecialty programs, in the United States.[2]

The Orthopedic Surgery Milestone Project started in 2013 to evaluate the skills and knowledge of residents from a specialty within the same set of competencies. It does not replace existing assessments in residency programs but provides a common framework to analyze acquired skills.[3] The Milestones instrument generates detailed information on the strengths and weaknesses of each residency service and identifies syllabus gaps to be addressed, improving the educational quality of residency programs.

The ACGME requires an annual evaluation, including systematic analysis of curriculum, resident performance, faculty development, and program quality. It also suggests that assessment instruments include written examinations, global ratings, and case/procedure records. The teaching analysis model recommended by the ACGME is based on the skills deemed fundamental for professional practice.

Residents are evaluated through milestones, that is, medical knowledge, patient care, professionalism, communication skills, and system practice, in addition to learning based on such practices and improvements. During the assessment, a preceptor selects the milestone that best describes the resident's current performance, from levels 1 to 5. Level 1 represents the milestones expected from a 1st-year resident, and level 5 refers to the performance goals defined for the residency, potentially describing the performance of a professional practicing for several years.[4] However, it is worth mentioning that these levels do not correspond to the year of residency; moreover, the selection of a certain level implies that the resident substantially demonstrates the characteristics related to it, as well as those from lower levels. All residents, from the 1st to the last year of medical residency, will be evaluated using the Milestones instrument.

Thus, the translation and transcultural adaptation of the Orthopedic Surgery Milestone Project will help in the evaluation and, consequently, training of professionals specializing in orthopedics.

Methods

After the ACGME authorized the translation of the questionnaire, two Brazilian translators fluent in English performed the first translation into Portuguese, and both texts were compared to formulate a consensus text. This text was back translated into English by two bilingual translators whose main language is English and who had no knowledge of the original text of the instrument. This translation was evaluated and compared to the original text by a review committee to correct potential discrepancies and enable the preparation of a pretest version of the questionnaire.

The last step consisted of evaluating the clarity and understanding of the questionnaire regarding its intended use. Since the pretest version had the desired characteristics, it was confirmed as the final text.

Results

[Figures 1] [2] [3] [4] to [5] describe the translation into Portuguese and transcultural adaptation of the Milestones 2.0 instrument. It is noteworthy that the back translation process showed that the initial Portuguese text corresponded to the English text, with no semantic divergence between translators.

Fig. 1. Diagram of the protocol used for translation and transcultural adaptation of the Milestones evaluation instrument.

Fig. 2 Translation of the Milestones instrument. Clinical case 1-original text.

Fig. 3 Translation of the Milestones instrument. Clinical case 2-original text.

Fig. 4 Translation of the Milestones instrument. Clinical case 3-original text.

Fig. 5 Translation of the Milestones instrument. Clinical case 4-original text.

The first two texts were the initial translations into Portuguese. Both presented semantic similarity, with no modification of the analyzed proposition. However, sometimes, different words with similar meanings were used to translate the same term (e.g., basic imaging exams versus basic imaging studies). A specialized committee reviewed the two translations and reconciled them into a single, easier-to-understand text. The final text consisted of the back translation of the reviewed versions.

The original Milestones instrument has 54 pages and 20 clinical situations from different areas of orthopedics; its questions encompass the doctor-patient relationship, in addition to theoretical and practical medical knowledge ([Table 1] and [Figures 1]–[2]).

Table 1

Original text	Initial text	Consensual text	Final text
Appropriately orders basic imaging studies	Solicita adequadamente os exames básicos de imagem	Solicita adequadamente os estudos básicos de imagem	Solicita adequadamente os estudos básicos de imagem
Provides procedure and patient specific postoperative management and rehabilitation	Fornece gerenciamento pós-operatório e reabilitação específicos para cada procedimento e paciente	Fornece procedimentos e gerenciamento pós-operatório e reabilitação paciente – específicos	Fornece gerenciamento pós-operatório e reabilitação específicos para cada procedimento e paciente
Prescribes and manages nonoperative treatment (e.g., nonsteroidal antiinflammatory drugs [NSAIDs], steroid injections, brace, rocker bottom shoes)	Prescreve e gerencia o tratamento não cirúrgico (por exemplo, anti-inflamatórios não-esteroides [AINEs], injeções de corticosteroides, imobilizadores, sapatos com a entressola curva)	Prescreve e gere o tratamento não cirúrgico (por exemplo, anti-inflamatórios não-esteroides [AINEs], injeções de corticosteroides, órteses, calçado ortopédico com sola curva (do inglês rocker bottom shoes)	Prescreve e gere o tratamento não cirúrgico (por exemplo, anti-inflamatórios não-esteroides [AINEs], injeções de corticosteroides, órteses, calçado ortopédico com sola curva (do inglês rocker bottom shoes)
Capable of surgical reduction and fixation of a full range of fractures and dislocations	Capaz de reduzir e fixar cirurgicamente uma gama completa de fraturas e luxações	Capaz de reduzir e fixar cirurgicamente uma série completa de fraturas e luxações	Capaz de reduzir e fixar cirurgicamente uma gama completa de fraturas e luxações
Performs surgical reduction and fixation of a simple fracture	Reduz cirurgicamente e fixa uma fratura simples	Realiza a redução cirúrgica e a fixação de uma fratura simples	Realiza a redução cirúrgica e a fixação de uma fratura simples

Discussion

There are no instruments to assess medical skills regarding musculoskeletal disorders in Portuguese, but the Orthopedic Surgery Milestone Project fulfills this purpose in English. The method of translation and transcultural adaptation used here was recommended by Guillemin et al.[4] It consists of four stages: initial translation, back translation, elaboration of a consensus text, and preparation of a commented pretest version and the final text.

The present study followed the guidelines recommended by Guillemin et al.[4] to minimize bias and biased results. This methodology makes the translated Milestones questionnaire suitable for application in Brazilian orthopedic residency services to assess the skills and abilities of these professionals.

Translation and adaptation of this instrument occurred after ACGME approval. The entire process was carried out properly, following the standards of cultural adaptation, and there were no difficulties in understanding any question. The Milestones instrument is widely recognized and used in the United States in different medical services and subspecialties, including clinical and surgical areas (e.g., Milestones for Plastic Surgery, Internal Medicine, Urology). It is applied easily through clinical situations that assess professionalism, communication skills, and medical knowledge. Residents are analyzed and graded on competency scales and developmental milestones throughout the year. It is a way for the student to recognize their limitations and deficiencies and improve them. The goal is to make specialist training programs more adequate to the current demand so that the specialized physician can work in public and private sectors, either in large or small centers.[5]

A study based on competencies and milestones aims to identify and analyze any deficiency in the training process of resident physicians, allowing preceptors to reinforce the teaching/learning of that competency.[6] In addition, competency-based training allows for more uniform curricular development in medical residency schools.

The adaptation of this assessment instrument to the Brazilian context required minor grammatical and cultural adjustments. In preparing the pretest version and the final text, we tried to maintain the form that most closely resembled the original to achieve equivalent semantics between them.

It is also worth mentioning that no items from the original questionnaire were included or excluded throughout the development of the final text, preventing further changes in psychometric properties to allow texts comparison.

The potential use of Milestones as a method for evaluation and improvement of the quality of teaching in medical residency services will result in physicians better prepared to exercise their profession. The translation of this instrument and its application will allow the assessment of the main deficiencies in Brazilian Orthopedic Services, which may improve medical education.

The Portuguese version of the Milestones instrument, adapted to the Brazilian culture, is applied easily.

This instrument should be used once a year in residents of the 3 years, similar to the Assessment Test of Residents in Orthopedics (TARO) by the Brazilian Society of Orthopedics (SBOT). We initially propose the assessment of competency 1 in the 1st year of residency, competencies 2 and 3 in the 2nd year, and competencies 4 and 5 in the 3rd year of residency. However, instrument application will eventually determine the equivalence. The final text of this questionnaire was the result of the corrections made by all translators and review teams involved. Although there is no evidence against the adequate translation and transcultural adaptation of the Milestones instrument to Brazil, future studies will establish the accuracy and reliability of this tool in Brazilian orthopedic and traumatology residency programs.

Conclusion

The translation into Portuguese and the transcultural adaptation of The Orthopedic Surgery Milestone Project provide an instrument for assessing competencies suited to the Brazilian reality. This instrument can evaluate different aspects of knowledge and practice over the 3 years of specialization, seeking a better medical education. The translation of the Milestones assessment instrument into Portuguese was consistent with the original texts.

Conflito de Interesses

Os autores declaram não haver conflito de interesses.

Authors' Contribution

Each author contributed individually and significantly to the development of this article. Castro, U. B.: review and intellectual concept of the article. Simão, K. F. R.: data analysis and interpretation and data writing. Gomes, G. R.: data analysis and interpretation as well as data writing. Egito, L. J. C.: translation and data writing. Figueiredo, S. D. F. A.: translation and data writing. Bispo Júnior, R. Z.: translation and article review.

^* Study developed at Hospital das Clínicas, Universidade Federal de Minas Gerais, Belo Horizonte, MG, Brazil, and Centro de Ciências Médicas, Universidade Federal da Paraíba, João Pessoa, PB, Brazil.

• Referências

• 1 Ames SE, Ponce BA, Marsh JL, Hamstra SJ. Orthopaedic Surgery Residency Milestones: Initial Formulation and Future Directions. J Am Acad Orthop Surg 2020; 28 (01) e1-e8
• 2 Accreditation Council for Graduate Medical Education (ACGME).. Accreditation Council for Graduate Medical Education (ACGME) − About Us [access in 2021 Mar 19]. Disponível em: http://www.acgme.org/About-Us/Overview
• 3 Hobgood CD, Riviello RJ, Jouriles N, Hamilton G. Assessment of communication and interpersonal skills competencies. Acad Emerg Med 2002; 9 (11) 1257-1269
• 4 Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 1993; 46 (12) 1417-1432
• 5 Fernandes CR, Farias Filho A, Gomes JMA, Pinto Filho WA, Cunha GKF, Maia FL. Currículo baseado em competências na residência médica. Rev Bras Educ Med 2012; 36 (01) 129-136
• 6 Stodel EJ, Wyand A, Crooks S, Moffett S, Chiu M, Hudson CC. Designing and Implementing a Competency-Based Training Program for Anesthesiology Residents at the University of Ottawa. Anesthesiol Res Pract 2015; 2015: 713038

Publication History

Received: 12 August 2021

Accepted: 14 March 2022

Article published online:
22 July 2022