Diagnostic Accuracy and Interobserver Agreement of Quasistatic Ultrasound Elastography in the Diagnosis of Thyroid Nodules

Abstract

Purpose: To assess the best technique and the diagnostic accuracy of Quasistatic Ultrasound Elastography (QUE) in thyroid nodules. Interobserver agreement was also evaluated.

Materials and Methods: A preliminary study of 50 patients with 54 thyroid nodules was performed with quantitative software in order to define the best cut-off value of different imaging methods. All patients underwent total thyroidectomy and histopathology findings served as the standard of reference. Thereafter, 154 nodules in 137 consecutive patients were prospectively evaluated by three operators. Findings at fine-needle aspiration cytology and histopathology (N = 60) served as the standard of reference.

Results: The most accurate technique was the axial peri-intranodular measurement method which achieved an area under the ROC curve of 0.961 (95 %CI 0.848 – 1.00) and had an optimal cut-off value of 3.00. QUE in the differentiation of thyroid nodules showed for operator 1: sensitivity 90 % (95 %CI 73.5 – 97.9 %), specificity 92.7 % (95 %CI 86.7 – 96.6 %), LR+ 12.40 (6.54 – 23.50), LR- 0.11 (0.04 – 0.32) and accuracy 91.4 % (95 %CI 85.4 – 97.3 %); for operator 2: sensitivity 86.7 % (95 %CI 69.3 – 96.2 %), specificity 87.1 % (95 %CI 79.9 – 92.4 %), LR+ 6.72 (4.16 – 10.80), LR- 0.15 (0.06 – 0.38) and accuracy 86.9 % (95 %CI 80.0 – 93.7 %); for operator 3: sensitivity 80 % (95 %CI 61.4 – 92.3 %), specificity 83.9 % (95 %CI 76.2 – 89.9 %), LR+ 4.96 (3.20 – 7.70), LR- 0.24 (0.12 – 0.49) and accuracy 81.9 % (95 %CI 74.0 – 89.9 %). Interobserver agreement values between operator 1 and operator 2 (k = 0.79) (p < 0.05, 95 %CI 0.684 – 0.904), between operator 1 and operator 3 (k = 0.73, 95 %CI: 0.607 – 0.854) and between operator 2 and operator 3 (k = 0.71, 95 %CI: 0.584 – 0.835) were significant.

Conclusion: QUE provides accurate quantitative evaluation of thyroid nodules with low interobserver variability.

Zusammenfassung

Ziel: Die Ermittlung der besten Technik und der diagnostischen Genauigkeit der Quasistatischen-Ultraschall-Elastografie (QUE) von Schilddrüsenknoten. Die Interobserver-Übereinstimmung wurde ebenfalls bewertet.

Material und Methoden: Mit einer quantitativen Software wurde eine Vorstudie an 50 Patienten mit 54 Schilddrüsenknoten durchgeführt, um für verschiedene bildgebende Verfahren die optimalen Grenzwerte festzulegen. Alle Patienten hatten eine Thyreoidektomie und die histopathologischen Befunde dienten als Referenzstandard. Im Anschluss wurden 154 Knoten von 137 Folgepatienten prospektiv von drei Bedienern bewertet. Als Standardreferenz dienten die Befunde der Feinnadelaspirationszytologie und der Histopathologie (N = 60).

Ergebnisse: Die genaueste Technik war die axiale peri-intranoduläre Messmethode, mit der eine Fläche unter der ROC-Kurve von 0,961 (95 %CI 0,848 – 1,00) erzielt wurde und die einen optimalen Grenzwert von 3,00 hatte. Die QUE zeigte bei der Differenzierung von Schilddrüsenknoten für den Bediener 1 folgende Werte: Sensitivität 90 % (95 %CI 73,5 – 97,9 %), Spezifität 92,7 % (95 %CI 86,7 – 96,6 %), LR+ 12,40 (6,54 – 23,50), LR- 0,11 (0,04 – 0,32) und Genauigkeit 91,4 % (95 %CI 85,4 – 97,3 %); für Bediener 2: Sensitivität 86,7 % (95 %CI 69,3 – 96,2 %), Spezifität 87,1 % (95 %CI 79,9 – 92,4 %), LR+ 6,72 (4,16 – 10,80), LR- 0,15 (0,06 – 0,38) und Genauigkeit 86,9 % (95 %CI 80,0 – 93,7 %); für Bediener 3: Sensitivität 80 % (95 %CI 61,4 – 92,3 %), Spezifität 83,9 % (95 %CI 76,2 – 89,9 %), LR+ 4,96 (3,20 – 7,70), LR- 0,24 (0,12 – 0,49) und Genauigkeit 81,9 % (95 %CI 74,0 – 89,9 %). Die Interobserver-Übereinstimmung zwischen Bediener 1 und Bediener 2 (k = 0,79) (p < 0,05; 95 %CI 0,684 – 0,904), zwischen Bediener 1 und Bediener 3 (k = 0,73; 95 %CI: 0,607 – 0,854) und zwischen Bediener 2 und Bediener 3 (k = 0,71; 95 %CI: 0,584 – 0,835) ergab jeweils signifikante Werte.

Schlussfolgerung: Die QUE ermöglicht die genaue quantitative Bewertung von Schilddrüsenknoten bei einer niedrigen Interobserver-Variabilität.

Key words

thyroid nodules - ultrasound elastography - color Doppler ultrasound - thyroid

Introduction

Thyroid nodules are a common clinical condition with a prevalence of 2 – 6 % at palpation, 19 – 35 % at ultrasound, and 8 – 65 % at autopsy [1]. Although most thyroid nodules are benign, the incidence of differentiated thyroid cancer is increasing due to improvements in high-resolution ultrasonography (US) and fine-needle aspiration cytology (FNAC) [2]. US has been widely used to differentiate between malignant and benign nodules based on a few distinctive features such as irregular margins, microcalcifications, marked hypoechogenicity, intranodular vascularization greater than perinodular vascularization, interval growth of diameter > 20 % and deeper than wide shape. The sensitivity, specificity, negative and positive predictive values are not consistent among studies and this limitation makes US not reliable in the diagnosis of malignant nodules [3] [4]. FNAC is considered the most accurate preoperative method to assess the nature of thyroid nodules [4], with the highest specificity (60 %–98 %), but with varying sensitivity (54 %–90 %) [5] [6] [7] [8]. Still FNAC findings are not diagnostic in 5 % to 25 % of cases. Furthermore, since it is an invasive procedure, FNAC can be burdened with periprocedural complications [9].

Recently, US elastography has been employed with success in the assessment of thyroid nodules by evaluating the elasticity of tissues in vivo [10]. All the images are matched with an elasticity color scale and then classified according to the Ueno elasticity score [10]. Rago et al. reported a sensitivity of 97 % and a specificity of 100 % and a positive predictive value of 100 %, in detecting malignant thyroid nodules with this method [11]. However, Park et al. [12] failed to demonstrate a reliable interobserver agreement with freehand ultrasound elastography (UE). Bae et al. proposed a different technique using carotid artery pulsation as an in vivo compression source in order to reduce the variability of different compression levels applied by the operators [13]. The out-of-plane motion, which could decrease the quality of elastosonography images during external compression, is minimized by using in vivo compression as the transducer is fixed during data acquisition.

A newly developed quantitative scoring method called the elasticity contrast index (ECI) can avoid the bias related to external compression elastography [14]. The ECI index is based on the Elastoscan method which is a steady-state quasistatic physiological excitation technique with the use of carotid pulsation as a strain inductor to obtain a quantitative stiffness evaluation. Lim et al. reported a good interobserver agreement and intraobserver reproducibility with this technique [15].

In our prospective study we first tested three different Elastoscan examination techniques for the differential diagnosis of thyroid nodules, and subsequently, in a large and new study population, we assessed the performance of the most accurate Elastoscan technique and the interobserver agreement of three operators with different experience.

Materials and methods

The study cohorts included patients referred to our department between October 2011 and February 2012. The inclusion criteria were the presence of any thyroid nodule, and FNAC and surgery planned at the time of ultrasound examination and finally performed within the study period. The exclusion criteria were: cystic nodules; cystic nodules with solid portions of insufficient size to be sampled (< 5 mm); spongiform nodule (microcystic portion > 50 %) [16]; the presence of a coarse calcification inside the nodule; pregnancy; heart failure; severe pulmonary hypertension. The isthmic nodules were also excluded from this study as carotid pulsation is scarcely transmitted in this site. Ten patients were then excluded (four with isthmic nodules, four because satisfactory elastogram was not achieved due to excessive carotid artery pulsation and, two because of the presence of cystic nodules).

Written informed consent was obtained from all patients. The study was performed in accordance with the ethical guidelines of the Helsinki Declaration and approved by the local ethics committee. A first series of 50 consecutive patients, undergoing thyroid surgery, with 54 thyroid nodules were evaluated with the ACCUVIX A30 (Medison Co. Ltd., Seoul, Korea) with dedicated Elastoscan quantitative software (Samsung Medison) by an expert radiologist (VC) who was blinded to clinical, cytological or histological characteristics. No external compression with the transducer was applied since pulsation from the carotid artery was used (in vivo compression) and the ECI was assessed [17].

The thyroid expanded and compressed because of carotid artery pulsation. In the preliminary study, three different techniques of in vivo elastography measurement were evaluated: axial intranodular ECI; axial peri-intranodular ECI; and longitudinal intranodular ECI The operator performed the examination in axial and longitudinal planes and then drew the region of interest (ROI) including only the nodule or the nodule and some tissue around it. The largest diameter of the thyroid nodule was included in the image. The patient was asked to perform a breath-hold for 3 – 4 seconds to allow the acquisition of QUE data without applying any external compression with the transducer. The strain frames were generated using the acquired data. As a consequence, the nodules and the thyroid were colored depending on their stiffness. On the upper right side of the screen a colorimetric scale indicates the correspondence of stiff and soft areas, with red and blue colors, respectively. An ROI as reported above was then delineated by the operator, and the ECI value was computed and displayed on the screen ([Fig. 1], [2], [3]). A high ECI value index indicated that the thyroid nodule was most likely malignant [15]. Once the most accurate technique was defined in the preliminary study, a further cohort of 137 consecutive patients with 154 thyroid nodules was evaluated by three observers (a senior radiologist and two residents with two and three years of experience, respectively) using that technique. The three radiologists employed this thyroid elastography technique in more than 30 nodules before starting the present analyses. They were blinded to clinical, cytological or histological characteristics and they individually performed both elastography data acquisition and nodule delineation for scoring, in order to obtain data for the technique evaluation. All nodules were determined by FNAC and by histopathology after thyroidectomy (N = 60). FNAC was performed with a 23 – 25-gauge needle attached to a 20 ml syringe. Adequacy of aspirates was defined according to the guidelines of the Papanicolaou Society.

Fig. 1 The image shows an iso-hyperechoic, well marginated and oval-shaped nodule localized in the right middle lobe a of the thyroid gland, with a type III vascular pattern b. The longitudinal intranodular ECI was 1.64 c; the axial peri-intranodular measure was 0.87 d; the axial intranodular measure was 1.07 e. Histology confirmed the diagnosis of benign nodular hyperplasia.

Abb. 1 Das Bild zeigt einen iso-echoreichen, gut begrenzten und ovalen Knoten, der sich im mittleren rechten Lappen a der Schilddrüse befindet und ein vaskuläres Muster vom Typ III aufweist b. Der longitudinale intranoduläre ECI betrug 1,64 c, die axiale peri-intranoduläre Messung ergab 0,87 d und die axiale intranoduläre Messung 1,07 e. Die Histologie bestätigte die Diagnose einer gutartigen nodulären Hyperplasie.

Fig. 2 The image shows a hypoechoic nodule with irregular margins localized in the right middle lobe of the thyroid gland a with a type III vascular pattern b. The axial peri-intranodular ECI measures 4.58 c. Histology confirmed the diagnosis of medullary carcinoma.

Abb. 2 Das Bild zeigt einen echoarmen Knoten mit irregulären Rändern, der sich im mittleren rechten Lappen der Schilddrüse befindet und ein vaskuläres Muster vom Typ III aufweist b. Der axiale peri-intranoduläre ECI misst 4,58 c. Die Histologie bestätigte die Diagnose eines medullären Karzinoms.

Fig. 3 Longitudinal trapezoid baseline US shows a hypoechoic, fairly marginated nodule a; Elastoscan showed an axial peri-intranodular ECI index of 5.05 b. Histology confirmed the diagnosis of papillary carcinoma.

Abb. 3 Der longitudinale trapozoide real-time-US zeigt einen echoarmen, unscharf abgrenzbaren Knoten a; der ElastoScan zeigte einen axialen peri-intranodulären ECI-Index von 5,05 b. Die Histologie bestätigte die Diagnose eines papillären Karzinoms.

Statistical analysis

Data were collected prospectively and recorded by each radiologist in a computerized spreadsheet

(Excel, Microsoft). The sensitivity, specificity, positive likelihood ratios (LR+), negative likelihood ratios (LR-) and diagnostic accuracy of Elastoscan were calculated with their 95 % confidence interval (CI). The optimal cut-off value of each Elastoscan method was calculated using the Youden’s test for receiver-operating characteristic (ROC) analysis. Areas under the curve (AUC) were compared using the Bonferroni’s test. A p< 0.05 was considered statistically significant. Interobserver agreement for the ECI measurements was calculated using the Fleiss’ kappa interpreted according to the Landis and Koch’s method [18], with a Kappa value of 0.20 or less indicating poor agreement; 0.21 – 0.40 indicating fair agreement; 0.41 – 0.60 indicating moderate agreement; 0.61 – 0.80 indicating substantial agreement; and 0.81 – 1.00 indicating excellent agreement.

Results

The preliminary study included 54 nodules in 50 patients (46 females, 4 males; mean age: 58 years; range, 38 – 78 years). At histology, 34 were hyperplastic nodules, 17 were papillary carcinomas, two were adenomas and one was a medullary carcinoma. The prevalence of malignancy was 33.3 %.

The median ECI value of malignant nodules was 3.99, whereas it was 1.99 in benign lesions (Mann-Whitney test: p = 0.003).

The most accurate technique was the axial peri-intranodular measurement method which achieved an area under the ROC curve of 0.961 (95 %CI 0.848 – 1.00). Its optimal cut-off value was 3.00. The longitudinal ECI measurement method had an area under the ROC curve of 0.834 (95 %CI 0.709 – 0.930) and its best cut-off value was 3.03. The axial-intranodular measurement method had an area under the ROC curve of 0.889 (95 %CI 0.803 – 1.00) and its best cut-off value was 3.07 ([Tab. 1, ] [Fig. 1], [2]). Such differences were statistically significant at the Bonferroni’s test. An ECI> 3.00 in an axial peri-intranodular measurement was adopted as a criterion for suspected malignancy in the evaluation of the second cohort study.

Table 1 ROC features for operator 1 in the preliminary study.

measures	cut-off	sensitivity (95 % CI)	specificity (95 % CI)	AUC (95 % CI)
axial peri-intranodular	3.00	91 % (65.3 – 98.6 %)	90 % (73.9 – 96.9 %)	96.1 % (84.8 – 100 %)
axial intranodular	3.07	90 % (65.3 – 98.6 %)	93.2 % (81.3 – 99.3 %)	88.9 % (80.3 – 100 %)
longitudinal intranodular	3.03	80 % (58.6 – 96.4 %)	80.5 % (64.0 – 91.8 %)	83.4 % (70.9 – 93.0 %)

A cohort of 137 patients with 154 thyroid nodules (121 females and 16 males; mean age: 53 years; range, 20 – 70 years) were then prospectively evaluated by three operators by assessment of the peri-intranodular ECI. Patients with malignant nodules were younger than patients with benign nodules (35.4 ± 9.8 vs. 41.2 ± 8.7 years, p < 0.001). There was no association between gender and the malignancy of nodules (p = 0.079). Among these nodules, 118 were hyperplasia, 28 were papillary carcinomas, six were adenomas, one was a follicular carcinoma and one was a medullary carcinoma. The prevalence of malignancy was 19.5 %.

The diameter of benign nodules ranged from 5 to 30 mm (mean 15.1 ± 7.5 mm) in the transverse view (the scan of ECI measurement). The size of the malignant nodules ranged from 9 to 31 mm (mean 16.4 ± 7.5 mm) in the transverse view (the scan of ECI measurement).

The statistical analysis yielded for operator 1: sensitivity 90 % (95 %CI 73.5 – 97.9 %), specificity 92.7 % (95 %CI 86.7 – 96.6 %), LR+ 12.40 (6.54 – 23.50), LR- 0.11 (0.04 – 0.32) and accuracy 91.4 % (95 %CI 85.4 – 97.3 %); for operator 2: sensitivity 86.7 % (95 %CI 69.3 – 96.2 %), specificity 87.1 % (95 %CI 79.9 – 92.4 %), LR+ 6.72 (4.16 – 10.80), LR- 0.15 (0.06 – 0.38) and accuracy 86.9 % (95 %CI 80.0 – 93.7 %); for operator 3: sensitivity 80 % (95 %CI 61.4 – 92.3 %), specificity 83.9 % (95 %CI 76.2 – 89.9 %), LR+ 4.96 (3.20 – 7.70), LR- 0.24 (0.12 – 0.49) and accuracy 81.9 % (95 % CI 74.0 – 89.9 %) ([Table 2]).

Table 2 Operator’s diagnostic performance at elastography with peri-intranodular measures and 3.00 ECI value as a cut-off on 154 nodule.

operator	sensitivity (95 % CI)	specificity (95 % CI)	Acc (95 % CI)	LR+ (95 % CI)	LR- (95 % CI)
op1	90 % (73.5 – 97.9 %)	92.7 % (86.7 – 96.6 %)	91.4 % (85.4 – 97.3 %)	12.40 (6.54 – 23.50)	0.11 (0.04 – 0.32)
op2	86.7 % (69.3 – 96.2 %)	87.1 % (79.9 – 92.4 %)	86.9 % (80.0 %–93.7 %)	6.72 (4.16 – 10.80)	0.15 (0.06 – 0.38)
op3	80 % (61.4 – 92.3 %)	83.9 % (76.2 – 89.9 %)	81.9 % (74.0 – 89.9 %)	4.96 (3.20 – 7.70)	0.24 (0.12 – 0.49)

The interobserver agreement between operator 1 and operator 2 was 92.2 %, between operator 1 and operator 3 was 89.6 % and between operator 2 and operator 3 was 88.3 % ([Table 3]).

Table 3 Interobserver agreement in the second part of the study.

comparison	k value (95 % CI)	agreement
op1 – op2	0.79 (0.68 – 0.90)	92.2 %
op1 – op3	0.73 (0.61 – 0.85)	89.6 %
op2 – op3	0.71 (0.58 – 0.84)	88.3 %

Discussion

Imaging of the viscoelastic properties of tissues has gained popularity over the last decade because of its noninvasive nature and accuracy in the differential diagnosis of thyroid nodules. Elastosonography is based upon the principle that thyroid malignancies have stiffer tissues than benign lesions and that, under compression, the softer parts of tissues deform easier than the harder parts [10]. This technique assesses the elasticity of tissues in vivo, exploiting the potential offered by the ultrasonic waves. The thyroid gland is an interesting target for elastography because its external deformation is easy to detect using the ultrasound transducer. A number of techniques are currently available providing operators with qualitative, semiquantitative or quantitative information [19]. A recent meta-analysis by Bojunga et al. [20] summarized the results of eight studies including 639 thyroid nodules assessed by real-time elastography in the diagnosis of thyroid nodules. They reported a pooled sensitivity and specificity of 92 % and 90 %, respectively. However, significant heterogeneity between the different studies was observed in the specificity of this method. These findings may be due to the fact that 6 out of 8 studies employed qualitative elastography with a subjective evaluation of elastograms. Further technological development led to standardization of elastography providing semiquantitative and quantitative methods [10]. Xing et al. [21] compared semiquantitative elastosonography (strain ratio) with conventional qualitative elastosonography in the assessment of thyroid nodules and observed that the strain ratio measurement had a sensitivity of 97.8 % and a specificity of 85.7 %. More recently, the Q-elastography technique (a semiquantitative method) was compared with multiparametric ultrasound in differentiating the nature of thyroid nodules and showed promising results [22] [23]. Despite the excellent sensitivity and specificity reported in the literature, thyroid US elastography is not yet widely used in clinical practice. The main reason is the low interobserver and intraobserver agreement due to the variability in collecting data and scoring the findings. Park et al. [12] evaluated the interobserver agreement in terms of data acquisition and scoring for thyroid US elastography. In their study, three radiologists independently performed elastography data acquisition and scoring of 52 thyroid nodules with an external compression technique. No interobserver agreement was found among these three radiologists and this was attributed to the extent of compression that influenced both the elastography image and the elasticity score.

However, other studies showed good results with this method. Merino et al. [24] reported an excellent interobserver agreement (k = 0.82, 95 %CI 0.74 – 0.89) employing a qualitative system that scored the nodules according to the strain homogeneity. A recent study by Ragazzoni et al. [25] showed good accuracy (84 %) and interobserver agreement (k-test 0.643) using quantitative elastosonography.

The new method employing the in vivo natural compression of the carotid artery, combined with a quantitative evaluation of stiffness, seems to be a reliable technique. Lim et al. [14] performed a study with in vivo compression elastography which showed significant interobserver and intraobserver agreement. The present study confirmed these findings with carotid pulsation ECI measurements Reliance on nodule stiffness quantification could lead to improved performance and reproducibility. Lim et al. [14] employed in their study the ACCUVIX V20 device (Medison Co. Ltd., Seoul, South Korea), whereas herein we performed our study using the next generation of this US device, the ACCUVIX A30. Unlike Lim, who assessed the interobserver and intraobserver agreement of Elastoscan in a population of patients with only malignant lesions, we conducted a preliminary study to determine which of the Elastoscan techniques had the best performance and then we assessed the accuracy and the interobserver agreement of the ECI in a larger population with benign and malignant lesions. We observed a significant interobserver agreement, but an interobserver variability still exists. This could be due to the variability in selecting the scanning image for measurement with Elastoscan and therefore sampling different portions of the same nodule with different stiffness. The limited experience of two operators may further explain these findings.

This study may be affected by a number of limitations which should be considered. The ECI may be slightly different according to the strength of the pulsation from the carotid artery depending on age, atherosclerosis and hypertension. Because of this, we excluded from this study four patients in whom we could not perform a reliable elastographic examination due to excessive carotid artery pulsation. Four isthmic nodules were excluded from our cohort because the distance of isthmic nodules from the carotid artery did not allow a reliable elastographic evaluation. In our study the prevalence of the patients with thyroid cancer was greater than in the general population. However, such patients are most likely to be encountered in referral centers. In conclusion, the present results suggest that QUE with the Elastoscan method is associated with a high accuracy and low inter-observer variability in the assessment of thyroid nodules. Larger studies are needed to confirm these findings

Abbreviations

op:: operator;

CI:: confidence interval;

AUC:: area under the curve;

LR+:: positive likelihood ratio;

LR-:: negative likelihood ratio;

Acc:: accuracy;

k:: Kappa

Acknowledgment

We acknowledge Professor Corrado De Vito for his contribution to statistical analysis (Department of Public Health and Infectious Diseases, "Sapienza" University of Rome).

• References

• 1 Dean DS, Gharib H. Epidemiology of thyroid nodules. Best Pract Res Clin Endocrinol Metab 2008; 22: 901-911
• 2 Frates MC, Benson CB, Charboneau JW et al. Management of thyroid nodules detected at US: Society of Radiologists in ultrasound consensus conference statement. Radiology 2005; 237: 794-800
• 3 Hoang JK, Lee WK, Lee M et al. US features of thyroid malignancy: pearls and pitfalls. Radiographics 2007; 27: 847-860
• 4 Rodrigues HG, Pontes AA, Adan LF. Use of molecular markers in samples obtained from preoperative aspiration of thyroid. Endocr J 2012; 59 (05) 417-424 Epub 2012 Mar 23.
• 5 Tee YY, Lowe AJ, Brand CA et al. Fine-needle aspiration may miss a third of all malignancy in palpable thyroid nodules: a comprehensive literature review. Ann Surg 2007; 246: 714-720
• 6 Peng Y, Wang HH. A meta-analysis of comparing fine-needle aspiration and frozen section for evaluating thyroid nodules. Diagn Cytopathol 2008; 36: 916-920
• 7 Oertel YC, Miyahara-Felipe L, Mendoza MG et al. Value of repeated fine needle aspirations of the thyroid: an analysis of over ten thousand FNAs. Thyroid 2007; 17: 1061-1066
• 8 Yang J, Schnadig V, Logrono R et al. Fine-needle aspiration of thyroid nodules: a study of 4703 patients with histological and clinical correlations. Cancer 2007; 111: 306-315
• 9 Donatini G, Masoni T, Ricci V et al. Acute respiratory distress following fine needle aspiration of thyroid nodule: case report and review of the literature. G Chir 2010; 31: 387-389
• 10 Garra BS. Elastography: current status, future prospects, and making it work for you. Ultrasound Q 2011; 27: 177-186
• 11 Rago T, Santini F, Scutari M et al. Elastography: new developments in ultrasound for predicting malignancy in thyroid nodules. J Clin Endocrinol Metab 2007; 92: 2917-2922
• 12 Park SH, Kim SJ, Kim EK et al. Interobserver agreement in assessing the sonographic and elastographic features of malignant thyroid nodules. Am J Roentgenol Am J Roentgenol 2009; 193: W416-W423
• 13 Bae U, Dighe M, Dubinsky T et al. Ultrasound thyroid elastography using carotid artery pulsation: preliminary study. J Ultrasound Med 2007; 26: 797-805
• 14 Dighe M, Bae U, Richardson ML et al. Differential diagnosis of thyroid nodules with US elastography using carotid artery pulsation. Radiology 2008; 248: 662-669
• 15 Lim DJ, Luo S, Kim MH et al. Interobserver agreement and intraobserver reproducibility in thyroid ultrasound elastography. Am J Roentgenol Am J Roentgenol 2012; 198: 896-901
• 16 Moon WJ, Jung SL et al. Thyroid Study Group, Korean Society of Neuro- and Head and Neck Radiology Benign and malignant thyroid nodules: US differentiation—multicenter retrospective study. Radiology 2008; 247: 762-770
• 17 Luo S, Lim DJ, Kim Y. Objective ultrasound elastography scoring of thyroid nodules using spatiotemporal strain information. Medical Physics 2012; 39: 1182-1189
• 18 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159-174
• 19 Bamber J, Cosgrove D, Dietrich CF et al. EFSUMB guidelines and recommendations on the clinical use of ultrasound elastography. Part 1: Basic principles and technology. Ultraschall in Med 2013; 34: 169-184
• 20 Bojunga J, Herrmann E, Meyer G et al. Real-time elastography for the differentiation of benign and malignant thyroid nodules: a meta-analysis. Thyroid 2010; 20: 1145-1150
• 21 Xing P, Wu L, Zhang C et al. Differentiation of benign from malignant thyroid lesions: calculation of the strain ratio on thyroid sonoelastography. Ultrasound Med 2011; 30: 663-669
• 22 Cantisani V, D'Andrea V, Biancari F et al. Prospective evaluation of multiparametric ultrasound and quantitative elastosonography in the differential diagnosis of benign and malignant thyroid nodules: Preliminary experience. Eur J Radiol 2012; 81: 2678-2683
• 23 Cantisani V, D’Andrea V, Mancuso E et al. “Prospective evaluation in 123 patients of strain ratio as provided by quantitative Elastosonography and multiparametric ultrasound evaluation (ultrasound score) for the characterization of thyroid nodule”. Radiol Med 2013; 118: 1011-1021
• 24 Merino S, Arrazola J, Càrdenas A et al. Utility and interobserver agreementof UE in the detection of malignant thyroid nodules in clinical care. Am J of Neuroradiol 2011; 32: 2142-2148
• 25 Ragazzoni F, Deandrea M, Mormile A et al. High Diagnostic Accuracy and Interobserver Reliability of Real-Time Elastography in the Evaluation of Thyroid Nodules. Ultrasound Med Biol 2012; 38: 1154-1162
• 26 Cosgrove D, Piscaglia F, Bamber J et al. EFSUMB guidelines and recommendations on the clinical use of ultrasound elastography. Part 2: Clinical applications. Ultraschall in Med 2013; 34: 238-253