Evaluation of the Intraobserver and Interobserver Agreements of the New AO/OTA Classification for Fractures of the Trochanteric Region and the Femoral Neck^[*]

Abstract

Objective In the present study, we investigated the intra and interobserver agreement of the new Arbeitsgemeinschaft für Osteosynthesefragen/Orthopaedic Trauma Association (AO/OTA) classification for fractures of the proximal extremity of the femur.

Methods One hundred hip radiographs were selected from patients who suffered fractures of the trochanteric region or femoral neck. Four orthopedists, fellowship trained hip surgeons, and four orthopedic residents evaluated and classified fractures according to the new AO/OTA system on two separate occasions. The kappa (k) coefficient was used to evaluate intra and interobserver agreement in the different steps of the classification, namely: type, group, subgroup, and qualifier.

Results Hip surgery experts obtained almost perfect intraobserver agreement of type, substantial for group and, only moderate, for subgroup and qualifiers. The residents had lower performance, with substantial agreement for type, moderate for group, and reasonable for subgroup and qualifier. In the specialists' interobserver evaluation, there was also a gradual decrease in the agreement between type (almost perfect) and group (moderate), which was even lower for subgroup and qualifiers. Residents had a substantial interobserver agreement for type, moderate for group, and reasonable in the other branches.

Conclusion The new AO/OTA classification for fractures of the trochanteric region and femoral neck showed intra and interobserver agreements considered appropriate for type and group, with a drop in the subsequent branches, that is, for subgroup and qualifier. Still, in relation to the old AO/OTA classification, there was an improvement in the agreements for subgroup.

Keywords

femur neck - hip fractures - classification

Introduction

Fractures of the proximal extremity of the femur cause great morbidity and mortality in the short and medium term in the elderly,[1] [2] [3] with one third of patients progressing to death within 1 year, and half becoming dependent for locomotion.[4] [5] These fractures can also affect young patients victimized by high-energy trauma.[4] The treatment of these lesions requires the interaction of a multidisciplinary team.[6] These lesions recquire eminently surgical treatment and, to define the best treatment, among other data, it is necessary to classify the fracture.

An ideal classification system should allow communication between physicians, standardize terminology for research, and guide treatment decision.[6] Many attempts to create a classification system for fractures of the proximal femur have been described, with the classifications of Garden,[7] Evans,[8] Boyd and Griffin,[9] Tronzo[10] and the Arbeitsgemeinschaft für Osteosynthesefragen/Orthopaedic Trauma Association (AO/OTA)[11] as the best-known ones. The AO/OTA classification has favorable evidence of reliability in the trochanteric region[12] [13] [14] and in the femoral neck,[15] when compared to the other commonly used classifications. However, its additional ramifications tend to decrease interobserver confidence[13] [16] and require great practice.[16] [17]

The validation of a classification occurs with the demonstration of some criteria: good clinical correlation, adequacy in terms of agreement and accuracy, and also constructive validation (relevance).[18] Concerns with the terminology and complex flow line of the previous AO/OTA classification[19] motivated the modernization of the classification. However, we have not located studies in the literature on the reliability of this new version.

In this sense, the present study aimed to evaluate the degree of intra and interobserver agreement in each sequential subdivision of the new AO/OTA classification for fractures of the proximal extremity of the femur, in observers considered experienced (adult hip surgeons) and inexperienced (orthopedic residents).

Materials and Methods

The present retrospective study included radiographic records of patients who suffered fractures of the proximal extremity of the femur between 2015 and 2019, treated at a reference center in orthopedic trauma. A total of 100 consecutive cases were selected for this evaluation. The sample size was arbitrated based on previous studies[4] [5] [6] [12] [13] [18] that used smaller samples to validate classifications (between 40 and 70 cases). The project was previously submitted and approved by the Research Ethics Committee of the Institution (CAAE: 30754120.7.0000.5226).

The inclusion criteria were presence of fracture of the proximal extremity of the femur (bone 3, anatomical region 1), the trochanteric region (group A) or the femoral neck (group B), in skeletically mature individuals. Fractures of the femoral head (which are best evaluated by computed axial tomography) and pathological fractures were not included in this study.

Each participant received the digital radiography images in anteroposterior and lateral views for analysis. There was no patient or treatment information on the images used for classification. Four orthopedists specialized in adult hip surgery and 42^nd-year orthopedic residents classified all cases (sequentially and uninterruptedly, without time limitation) at 2 distinct moments, with an interval of 4 weeks. At the beginning of the evaluations, the detailed description of the new classification and its illustrative images were made available to the evaluators for learning the system. Each evaluation was performed individually, and neither the answers were allowed to be kept nor were the results discussed among them.

Interobserver reliability was determined through the first response between the evaluators and the intraobserver through a new evaluation 4 weeks after the initial one. This interval was used to reduce the risk of memory bias. Data were collected and stored in spreadsheets for statistical analysis. The Cohen kappa coefficient was used to evaluate the intraobserver agreement, and the Fleiss kappa coefficient was calculated to evaluate the inter-observer agreement. The SPSS Statistcs for Windows, Version 20.0 software (IBM Corp., Armonk, NY, USA) and the Online kappa Calculator (www.statisticssolutions.com) were used for the analyses. The agreement assessment included 4 stages: type of fracture (A - trochanteric or B - neck), group (1, 2, or 3), subgroups (1, 2, or 3 - except in type B3, which does not have subgroups), and also their qualifiers when available in the classification, that is, only in A.1.1 (N or O) or B2 1.2 or 3 (P, Q, or R). [Table 1] and [Table 2] illustrate the differences between the old and new AO/OTA classifications.

Table 1

	Old AO/OTA		New AO/OTA
Group A1	Simple transtrochanteric		Simple transtrochanteric
A1.1	Simple undisplaced		Isolated fracture of trochanter *Qualifiers: n: greater trochanter o: lesser trochanter
A1.2	Simple displaced		Two-part fracture
A1.3	Simple with distal to calcar extension		Fracture with intact lateral wall (> 20.5 mm)
Group A2	Transtrochanteric cominution		Multifragmentary transtrochanteric, incompetent lateral wall (<20.5 mm)
A2.1	Comminuta undisplaced
A2.2	Displaced with cominution		Fracture with 1 intermediate fragment
A2.3	Multifragmented (> 3 fragments)		Fracture with 2 or more fragments intermediate
Group A3	Reverse transtrochanteric		Reverse transtrochanteric
A3.1	Reverse obliquity		Simple, reverse obliquity
A3.2	Transverse reverse obliquity		Simple transverse, reverse obliquity
A3.3	Reverse obliquity with fracture of lesser trochanter		Reverse obliquity with wedge or multifragmentary

Table 2

	Old AO/OTA		New AO/OTA
Group B1	Subcapital fracture with minimal displacement		Subcapital fracture
B1.1	Valgus impaction > 15 degrees		Valgus impaction
B1.2	Valgus impaction < 15 degrees		Undisplaced
B1.3	Not impacted		Displaced
Group B2	Transcervical fracture		Transcervical fracture
B2.1	Basicervical		Simple Qualifiers: p < 30° q = 30–70° r > 70°
B2.2	Adducted mid-cervical		Multifragmented Qualifiers: p < 30° q = 30–70° r > 70°
B2.3	Shear mid-cervical		With shear Qualifiers: p < 30° q = 30–70° r >70°
Group B3	Subcapital displaced fracture, not impacted		Basicervical fracture
B3.1	Moderate varus displacement and lateral rotation
B3.2	Moderate vertical displacement and lateral rotation
B3.3	Significant displacement

Although the degree of agreement has distinct forms of interpretation,[19] the classic proposal of Landis and Koch was adopted,[20] with values between 0.00 and 0.20 considered as mild agreement; 0.21 and 0.40 reasonable agreement; 0.41 and 0.60 moderate agreement; 0.61 and 0.80 substantial agreement, and 0.81 and 1.00 almost perfect agreement (or excellent).

In the population sample studied, the mean age was 77.71 years (ranging from 57–98 years, standard deviation of 10.12). The female gender was predominant, with 63% of the cases, and the right side had one more case of fracture (51%).

Results

Intraobserver agreement

In the repeated evaluation with an interval of 4 weeks, the intraobserver agreement of type was almost perfect for specialists, with a mean k of 0.92, while that of residents was substantial (mean k 0.77). In the group classification, the specialists presented a substantial agreement (mean k 0.68), and residents a moderate agreement (mean k 0.44). For subgroup, the agreement of the specialists was moderate (mean k 0.52), and for the residents it was reasonable (mean k 0.28). Finally, in relation to qualifiers, the agreement of the specialists was also moderate (mean k 0.50), and that of the residents was reasonable (mean k 0.27).

In general, specialists perform better than residents. It is also observed ([Figure 1]) that the coefficients are decreasing as the branches of the classification are followed. [Table 3] describes in detail the intraobserver findings.

Fig. 1 Comparison between specialists and residents of the mean kappa coefficient (intraobserver).

Table 3

	kappa (Cohen)
Expert	Type	Group	Subgroup	Qualifier
1	0.972	0.705	0.607	0.607
2	0.972	0.589	0.376	0.338
3	0.894	0.747	0.599	0.600
4	0.851	0.713	0.500	0.459
Resident
1	0.828	0.468	0.298	0.300
2	0.851	0.443	0.259	0.260
3	0.806	0.640	0.421	0.391
4	0.608	0.230	0.153	0.144

Inter-observer agreement

Considering the first round as standard (used in most studies) for interobserver evaluation, we have an agreement in the type of 93.67% for specialists (k 0.87, almost perfect) and 90.17% for residents (k 0.80, substantial). In the group, the agreement was 60.83% for the specialists (k 0.53, moderate) and 55.5%for the residents (k 0.47, moderate). Advancing to subgroup, agreement dropped to 44.5% among specialists (k 0.41, moderate) and 42.7% for residents (k 0.39, reasonable). Finally, in the qualifiers the agreement was 42.67% for the specialists (k 0.40, reasonable) and 41.0% for the residents (k 0.39, reasonable).

[Table 4] details the interobserver results. Both specialists and residents decreased the coefficients as the classification branches out. However, in the subgroup-qualifier transition, the decrease in agreement was not significant. In the first round, residents reached coefficients always below the coefficients of the specialists ([Figure 2]), but in the second round, residents presented a greater agreement with each other than the specialists ([Figure 3]).

Table 4

	Rating 1		Rating 2
Type	% agreement	Kappa	% agreement	Kappa
Experts	93.67	0.87	97.00	0.94
Residents	90.17	0.80	94.50	0.89
Group
Experts	60.83	0.53	58.83	0.51
Residents	55.50	0.47	69.50	0.63
Subgroup
Experts	44.50	0.41	39.67	0.35
Residents	42.67	0.39	57.67	0.55
Qualifier
Experts	42.67	0.40	37.33	0.35
Residents	41.00	0.39	57.17	0.55

Fig. 2 Interobserver agreement in the first evaluation.

Fig. 3 Interobserver agreement in the second evaluation.

Discussion

In the previous AO/OTA classification, some fracture patterns occurred so rarely that there was no need for an exclusive coding for them. The terminology was the focus of confusion, due to the wide variety of terms for similar fractures. There was also difficulty in defining the fractures of group A2. In the new classification, the definitions and codes have been updated and simplified. The neck fractures were reorganized, and the Pauwels qualifier added to better definition of instability, especially in high-energy fractures.[19]

A fracture classification system should have adequate agreement between the same observer at different opportunities (intraobserver) and between different observers at the same time (interobserver).[6] [17] [18] The kappa (k) coefficient is one of the most used methods to evaluate the diagnostic accuracy of a classification system; its calculation has been adjusted for possible coincidences at random.[3]

In the current study, for the specialists, intraobserver agreement was almost perfect for type, substantial for group and moderate for subgroup and qualifier, while residents performed worse in all divisions.

Comparing interobserver agreements in both rounds was an interesting aspect of this research. A curious observation is that the interobserver agreement of the residents increased between the rounds, which may have indicated the capacity of learning the new classification.

Studies on the previous AO/OTA system obtained results similar to those obtained in our study, but with small variations. The study by Pervez et al.[3] obtained an average k among its observers of 0.62 for group, higher than that observed in our study (k 0.53 - moderate) and 0.33 in the subgroups, lower than we found (k 0.41 - moderate). Urrutia et al.[6] obtained moderate agreement, as our results, among its 9 evaluators for the groups, and only reasonable for the subgroups which is lower than in our study (k 0.41 - moderate). Mattos et al.[4] also obtained similar results with AO/OTA and Tronzo. Schwartsmann et al.,[14] in a study also involving orthopedic surgeons and residents, obtained moderate agreement (0.60) for group, similar to the present study, and reasonable (0.34) for subgroups, lower than that observed in the present study (k 0.41). Another interesting study,[15] with 100 fractures of the femoral neck, graded the Garden[7] classification as only reasonable interobserver agreement, which increased to moderate by simplifying the criterion for fracture displacement. This indicates that, in certain fractures, even a fairly simple classification can generate only moderate agreement.

In summary, considering the old classification, in the present study we obtained similar agreement to that of the literature for type and group and better agreement for subgroup, while qualifiers were not available in the old classification. This indicates that the new system was successful in bringing greater agreement in the subgroups, which were more extensively modified.

Evaluating the issue of experience of the examiners in the AO/OTA classification, Crijins et al.[16] did not observe a difference between 65 surgeons divided between more and less experienced according to the variables of practice time (> or < 17 years), work time dedicated to trauma (> or < 80%), and fractures treated per year (> or < 50). In an analogy with our study, we evaluated that the residents matched the specialists in the second evaluation, indicating a fast-learning curve in this system. Fung et al.[21] also noted that more experienced residents, in their final part of training, had a better assessment than the less experienced, indicating the learning of the old classification.

About the new AO/OTA system specifically, it is interesting to note the elimination of subgroup A2.1, which can help distinguish stable patterns (group A1), from the unstable ones (groups A2 and A3). Studies of the previous system[11] that tried to discern the extent to which a trochanteric fracture was stable had somewhat conflicting results. Radaideh et al.,[22] in a study on the use of cephalomedullary stems, defined groups A2 and A3 as unstable, as well as Zhang et al.[23] However, Knobe et al.[24] mention that groups A2 and A3 are generally considered unstable in the literature, but in a direct evaluation, the fracture of the lesser trochanter was the main criterion of instability for 82% of surgeons, among other considered factors (fracture of the greater trochanter, lateral wall fracture and reverse obliquity). Another study[25] considered subgroups A2.1, A2.2, and A3.3 as unstable.

The current classification has the integrity of the lateral wall (width greater than 20.5 mm) as the division pattern between groups A1 and A2. The rationale for this division was initially described by Gottfried[26] and Palm et al.,[27] who defined the lateral wall as an important structure for implant support. Later, Hsu et al.[28] were able to evaluate the thickness of the lateral wall would be necessary for safe synthesis with sliding screw plate, which motivated the current change of classification. Other studies[29] reviewed the subject and brought strategies for lateral wall reconstruction even with the use of intramedullary synthesis. Based on this literature review and the difficulties of classification into subgroups, we find it interesting to divide the groups from A2 (including) as a parameter to consider an unstable fracture that requires accurate technique of reduction and intramedullary synthesis.

Our study evidences the difficulties in classification systems for fractures of the proximal extremity of the femur. Despite these difficulties, this system demonstrated advantages over its predecessor[11] by simplifying a division for unstable exchange fractures at group level (A2 and A3) and may facilitate a possible choice of implant and reduction techniques. In neck fractures (type B), the new subdivision is simpler than the previous subgroup complexes ([Table 1]), and also encompasses the Pauwels qualifier. Additionally, we verified through the literature a higher reliability of the new AO/OTA classification when compared to other very widespread systems (Garden, Evans, Boyd, Tronzo).

In the present study we sought a consistent methodology for evaluating a classification of fractures, having as strengths the size and representativeness of the sample (larger than previous studies, and all the patterns of the new classification were identified by at least one evaluator), in addition to an adequate number of observers for better reliability of the Kappa coefficient.[20] The methodology of reliability of orthopedic classifications was examined by Audigé et al.,[18] and the present study encompasses all the quality criteria described. In addition, no study was found evaluating this new system in Pubmed, Medline and Scielo databasis, which brings new and relevant data on this classification very popular among orthopedic surgeons. Another interesting point was the evolution of interobserver agreement noted among resident physicians between evaluations, indicating the learning of the system. A deficient point in this research was the inclusion of hip specialists compared to only four second-year residents. Perhaps the inclusion of first- and third-year residents could further demonstrate the learning process.

As another possible limitation of the present study, we highlight the relatively high mean age of the sample, indicating a characteristic typical of the population profile of the hospital where it was performed. However, although there is a potential risk of not representing some more specific fractures of high-energy trauma, more common in younger patients, yet all the patterns of the new AO/OTA classification were identified at some point during the study. If we were to go to other younger patients to include in the sample, we would incur in selection bias. Moreover, studies of this nature have limitations inherent to its design, such as potential memory bias, which we consider low due to the large number of cases, the complexity of the classification and the time elapsed between evaluations.

The new AO/OTA system has moderate interobserver and substantial intraobserver reliability for experienced evaluators. Resident physicians were able to achieve the same levels of agreement after a short learning period. Further studies are needed to assess their ability in relation to the indication of treatment (especially on type of synthesis) and prognosis.

Conclusion

The new AO/OTA classification for fractures of the trochanteric region and femoral neck showed appropriate intra and interobserver agreements for type and group, with worsening in subsequent branches, that is, subgroup and qualifier. Nevertheless, in relation to the old AO/OTA classification, there was an improvement in the agreements for subgroups.

Conflito de Interessses

Os autores declaram não haver conflito de interesses.

Financial Support

There was no financial support from public, commercial, or non-profit sources.

^* Research developed in Criar - Joint Reconstruction Center and Research Institute. Hospital Angelina Caron, Campina Grande do Sul, Paraná, Brazil.

• Referências

• 1 Rapp K, Becker C, Lamb SE, Icks A, Klenk J. Hip fractures in institutionalized elderly people: incidence rates and excess mortality. J Bone Miner Res 2008; 23 (11) 1825-1831
• 2 Martinez-Reig M, Ahmad L, Duque G. The orthogeriatrics model of care: systematic review of predictors of institutionalization and mortality in post-hip fracture patients and evidence for interventions. J Am Med Dir Assoc 2012; 13 (09) 770-777
• 3 Pervez H, Parker MJ, Pryor GA, Lutchman L, Chirodian N. Classification of trochanteric fracture of the proximal femur: a study of the reliability of current systems. Injury 2002; 33 (08) 713-715
• 4 Mattos CA, Jesus AAK, Floter MS, Nunes LFB, Sanches BB, Zabeu JLA. Reprodutibilidade das classificações de Tronzo e AO para fraturas transtrocanteriana. Rev Bras Ortop 2015; 50 (05) 495-500
• 5 Oliveira FAS, Basile R, Pereira BCB, Cunha RLL. Avaliação da reprodutibilidade da classificação de Tronzo para fraturas intertrocantéricas do fêmur. Rev Bras Ortop 2014; 49 (06) 581-585
• 6 Urrutia J, Zamora T, Besa P, Zamora M, Schweitzer D, Klaber I. Inter and intra-observer agreement evaluation of the AO and the Tronzo classification systems of fractures of the trochanteric area. Injury 2015; 46 (06) 1054-1058
• 7 Garden RS. Low-angle fixation in fractures of the femoral neck. J Bone Joint Surg Br 1961; 43 (03) 647-663
• 8 Evans EM. The treatment of trochanteric fractures of the femur. J Bone Joint Surg Br 1949; 31B [phone]
• 9 Boyd HB, Griffin LL. Classification and treatment of trochanteric fractures. Arch Surg 1949; 58 (06) 853-866
• 10 Tronzo RG. Surgery of the Hip Joint. Philadelphia: Lea & Febiger; 1973
• 11 Müller ME, Nazarian S, Koch P. Classification AO des fractures. Tome I. Les os longs.. Berlin: Springer-Verlag; 1987
• 12 Jin WJ, Dai LY, Cui YM, Zhou Q, Jiang LS, Lu H. Reliability of classification systems for intertrochanteric fractures of the proximal femur in experienced orthopaedic surgeons. Injury 2005; 36 (07) 858-861
• 13 Schipper IB, Steyerberg EW, Castelein RM, van Vugt AB. Reliability of the AO/ASIF classification for pertrochanteric femoral fractures. Acta Orthop Scand 2001; 72 (01) 36-41
• 14 Schwartsmann CR, Boschin LC, Moschen GM, Gonçalves R, Ramos AS, Gusmão PD. Classificação das fraturas trocantéricas: avaliação da reprodutibilidade da classificação AO. Rev Bras Ortop 2006; 41 (07) 264-267
• 15 Van Embden D, Rhemrev SJ, Genelin F, Meylaerts SA, Roukema GR. The reliability of a simplified Garden classification for intracapsular hip fractures. Orthop Traumatol Surg Res 2012; 98 (04) 405-408
• 16 Crijns TJ, Janssen SJ, Davis JT, Ring D, Sanchez HB. Science of Variation Group. Reliability of the classification of proximal femur fractures: Does clinical experience matter?. Injury 2018; 49 (04) 819-823
• 17 Gusmão PD, Mothes FC, Rubin LA, Gonçalves RZ, Teloken MA, Schwartsmann CR. Avaliação da reprodutibilidade da classificação de Garden para fraturas do colo femoral. Rev Bras Ortop 2002; 37 (09) 381-386
• 18 Audigé L, Bhandari M, Kellam J. How reliable are reliability studies of fracture classifications? A systematic review of their methodologies. Acta Orthop Scand 2004; 75 (02) 184-194
• 19 Hartline BE, Achor TS. Use of the 95-degree Angled Blade Plate to Treat a Proximal Femur Fracture. J Orthop Trauma 2018; 32 (Suppl. 01) S26-S27
• 20 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33 (01) 159-174
• 21 Fung W, Jonsson A, Buhren V, Bhandari M. Classifying intertrochanteric fractures of the proximal femur: does experience matter?. Med Princ Pract 2007; 16 (03) 198-202
• 22 Radaideh AM, Qudah HA, Audat ZA, Jahmani RA, Yousef IR, Saleh AAA. Functional and Radiological Results of Proximal Femoral Nail Antirotation (PFNA) Osteosynthesis in the Treatment of Unstable Pertrochanteric Fractures. J Clin Med 2018; 7 (04) 78
• 23 Zhang WQ, Sun J, Liu CY, Zhao HY, Sun YF. Comparing the Intramedullary Nail and Extramedullary Fixation in Treatment of Unstable Intertrochanteric Fractures. Sci Rep 2018; 8 (01) 2321
• 24 Knobe M, Gradl G, Ladenburger A, Tarkin IS, Pape HC. Unstable intertrochanteric femur fractures: is there a consensus on definition and treatment in Germany?. Clin Orthop Relat Res 2013; 471 (09) 2831-2840
• 25 Cech O, Kostál R, Váchal J. [Unstable pertrochanteric fractures, biomechanic, classification and therapy.]. Acta Chir Orthop Traumatol Cech 2000; 67 (01) 17-27
• 26 Gotfried Y. The lateral trochanteric wall: a key element in the reconstruction of unstable pertrochanteric hip fractures. Clin Orthop Relat Res 2004; (425) 82-86
• 27 Palm H, Jacobsen S, Sonne-Holm S, Gebuhr P. Hip Fracture Study Group. Integrity of the lateral femoral wall in intertrochanteric hip fractures: an important predictor of a reoperation. J Bone Joint Surg Am 2007; 89 (03) 470-475
• 28 Hsu CE, Shih CM, Wang CC, Huang KC. Lateral femoral wall thickness. A reliable predictor of post-operative lateral wall fracture in intertrochanteric fractures. Bone Joint J 2013; 95-B [phone]
• 29 Tawari AA, Kempegowda H, Suk M, Horwitz DS. What makes an intertrochanteric fracture unstable in 2015? Does the lateral wall play a role in the decision matrix?. J Orthop Trauma 2015; 29 (Suppl. 04) S4-S9

Publication History

Received: 28 May 2020

Accepted: 01 December 2020

Article published online:
14 December 2021