Validation of Multiparametric Ultrasonography Criteria with Digital Subtraction Angiography in Carotid Artery Disease: A Prospective Multicenter Study^[*]

Abstract

Purpose The German Society of Ultrasound in Medicine (DEGUM) recently revised its multiparametric criteria for duplex ultrasonography (DUS) grading of internal carotid artery (ICA) disease. We determined the diagnostic accuracy of the revised DEGUM criteria for ultrasonography grading of ICA disease in a prospective multicenter study.

Materials and Methods We evaluated consecutive patients who underwent digital subtraction angiography of the extracranial carotid arteries at four tertiary care hospitals. Blinded investigators graded ICA disease according to DEGUM-recommended ultrasonography criteria and calculated NASCET-type percent stenosis from angiography images. Endpoints included overall classification accuracy, prediction of clinically relevant disease categories and between-test agreement in the continuous range of percent stenosis.

Results A total of 121 patients (median age: 69 [IQR, 16] years; 74 % men; median time between DUS and angiography: 1 day [IQR, 2]) provided 163 DUS-angiography carotid artery pairs. The classification accuracy of the DEGUM criteria to predict stenosis within 10 % increments as compared to angiography was 34.9 % (95 % CI, 28.0 – 42.6). The sensitivity of DUS for the detection of moderate (50 – 69 %) and severe (70 – 99 %) stenosis was 35 % and 81 %, with an overall accuracy of 73 % and 74 %, respectively. The specificity was 89 % and 69 %, respectively. Considering the continuous spectrum of the disease (0 – 100 %), the Bland-Altman interval limit of agreement was 51 %.

Conclusion At laboratories experienced with ultrasound grading of the extracranial ICA, the revised DEGUM multiparametric ultrasonography criteria do not eliminate the need for a confirmatory test for the identification of clinically relevant grades of the disease.

Zusammenfassung

Ziel Die multiparametrischen Ultraschallkriterien zur Graduierung von Karotisstenosen der Deutschen Gesellschaft für Ultraschall in der Medizin (DEGUM) wurden revidiert, bislang aber nicht mit dem Goldstandard Angiografie validiert. Ziel dieser prospektiven multizentrischen Validierungsstudie war es, die diagnostische Güte der DEGUM Ultraschallkriterien zu bestimmen.

Material und Methoden An vier Zentren wurden konsekutive Patienten eingeschlossen, bei denen routinemäßig eine digitale Subtraktionsangiografie (DSA) der extrakraniellen Arteria carotis interna (ACI) durchgeführt wurde. Die Graduierung des Stenosegrades der ACI erfolgte verblindet gemäß der multiparametrischen DEGUM Ultraschallkriterien. Angiografisch erfolgte die verblindete Stenosegraduierung entsprechend der NASCET Definition. Endpunkte umfassten die Gesamttrefferquote zur Vorhersage von Stenosen in vorgegebenen DEGUM Stenoseintervallen, Vorhersage von klinisch relevanten Stenosekategorien und diagnostische Übereinstimmung beider Methoden im kontinuierlichen Stenosebereich.

Ergebnisse Von insgesamt 121 rekrutierten Patienten (mittleres Alter, 69 [IQR, 16] Jahre, 74 % Männer, mediane Zeit zwischen Duplexsonografie und DSA, 1 Tag [IQR, 2]) ergaben sich 163 Duplex-DSA Gefäßpaare für die Validierung. Die Gesamttrefferquote der DEGUM-Ultraschallkriterien zur Vorhersage des Stenosegrades innerhalb von 10 % -Schritten im Vergleich zur DSA betrug 34,9 % (95 % CI, 28,0 – 42,6). Die Sensitivität der Ultraschallkriterien für den Nachweis von mittelgradigen (50 – 69 %) und hochgradigen (70 – 99 %) Stenosen betrug 35 % und 81 % mit einer Gesamtgenauigkeit von 73 % und 74 %. Die Spezifität betrug 89 % und 69 %. Der Bland-Altman-Übereinstimmungsbereich zwischen beiden Methoden betrug 51 %.

Schlussfolgerung In der Identifikation von klinisch relevanten Stenosekategorien der ACI beseitigen die multiparametrischen DEGUM Ultraschallkriterien nicht die Notwendigkeit eines vaskulären Bestätigungstestes.

Key words

angiography - ultrasound - carotid artery disease - diagnostic testing

Introduction

Ever since randomized controlled trials provided evidence-based proof for carotid endarterectomy in patients with symptomatic internal carotid artery (ICA) disease, it has been a challenge for clinicians to accurately identify clinically relevant grades of the disease [1] [2]. In fact, the benefit of surgery in secondary stroke prevention is greatest in patients with at least 70 % stenosis and, to a lesser extent, in those with 50 % stenosis. Aside from this, asymptomatic patients with higher degrees or rapid progression of the disease may also benefit from revascularization, if carefully selected [3] [4] [5]. Precise ICA stenosis grading, therefore, is inevitable for the selection of patients amenable to carotid revascularization procedures.

While digital subtraction angiography (DSA) is considered the gold standard for ICA grading, its routine clinical utilization is not practical due to its invasiveness and potential harm [6]. In contrast, duplex ultrasonography (DUS) is readily available, quick to perform and noninvasive, and most vascular centers worldwide utilize this modality as a first-line diagnostic test for carotid artery disease. However, the common use of single grading criteria results in significant diagnostic variability between ultrasonography- and angiography-determined degree of ICA stenosis, and the lack of uniformity in the interpretation of DUS poses the risk for substantial misclassification of carotid artery disease [7] [8] [9].

To overcome these limitations, the German Society of Ultrasound in Medicine (known by its German acronym DEGUM) recently revised its multiparametric DUS approach that implements a multitude of imaging and hemodynamic criteria [10] [11]. As opposed to recent ultrasound approaches, the DEGUM criteria quantify stenosis degree in increments of 10 % in order to enhance the diagnostic accuracy of stenosis grading. Meanwhile, this approach has been adopted by the Neurosonology Research Group of the World Federation of Neurology [12]. The revised DEGUM approach, however, has not yet been validated against the gold standard DSA, thus leaving uncertainty as to whether it ultimately improves the diagnostic accuracy of DUS [13]. Therefore, we aimed to evaluate the diagnostic accuracy of the multiparametric DEGUM ultrasound approach compared with the gold standard DSA for the assessment of extracranial ICA steno-occlusive disease in a prospective, multicenter, cross-sectional study.

Materials and Methods

Study design and multicenter population

This was a prospective study of diagnostic accuracy that adopted the Standards for Reporting of Diagnostic Accuracy (STARD) guidelines [14]. We evaluated consecutive patients who underwent DSA of the extracranial carotid arteries at four tertiary care angiography-equipped hospitals between March 2013 and December 2016. More specifically, both adult (≥ 18 years) inpatients and outpatients were screened for study eligibility if a diagnostic or therapeutic angiography was clinically indicated by the ordering physicians. As per study protocol, DUS was performed prior to DSA but no earlier than 30 days. In patients who underwent diagnostic angiography, DUS could be performed afterwards but no later than 30 days. Patients with known congestive heart failure (ejection fraction < 35 %), severe anemia (hemoglobin < 5 mmol/L) and elevated body temperature (> 38.5 °C) were excluded from this study, as these conditions are known to alter blood flow velocities, thus adversely affecting the index test.

Participating study centers were equipped with ultrasound laboratories that provided regular work-up of patients with extracranial ICA disease and staffed by physicians certified in ultrasonography either by the German Society of Ultrasound in Medicine (DEGUM) or the American Society of Neuroimaging (ASN). A central core lab was implemented for independent adjudication of DUS and angiography studies.

Sample size estimation was based on our primary endpoint, namely classification accuracy of the DEGUM ultrasonography approach for overall prediction of ICA disease using increments of 10 % as compared to angiography. Assuming a classification accuracy of 90 ± 5 % and a two-sided 95 % confidence interval for proportions with normal distribution, we estimated a required sample of 139 carotid arteries with angiographically detected steno-occlusive disease. Secondary endpoints included further measures of diagnostic performance of the DEGUM approach such as prediction of clinically relevant stenosis categories, between-test agreement in the continuous range of the disease and test reliability. The institutional review boards of each participating center approved the study and written informed consent was obtained from all patients or guardians.

Duplex ultrasonography of the carotid artery

All ultrasonography examinations were performed prospectively by investigators blinded to clinical and imaging information, and complied with technical regulations and the revised multiparametric grading criteria as specified by the DEGUM, which have been described in detail elsewhere [10] [11]. Briefly, the revised DEGUM criteria aim to measure the degree of ICA stenosis as described by the North American Symptomatic Carotid Endarterectomy Trial (NASCET) approach on angiography (i. e., narrowing related to the distal ICA diameter) and allow for disease categorization in one of eight narrow strata ([Fig. 1]) [1].

Fig. 1 Multiparametric DEGUM ultrasonography approach (modified from Arning et al.) [10]. Corresponding criteria have been detailed elsewhere [10] [11] [12]. NASCET stands for North American Symptomatic Carotid Endarterectomy Trial; PSV: peak systolic velocity; ACA: anterior cerebral artery; CCA: common carotid artery; EDV: end-diastolic velocity; ICA: internal carotid artery.

For ultrasonography examinations of the extracranial carotid arteries, a high-frequency (≥ 7 MHz) linear array transducer was used in all participating centers. Transcranial Doppler or Duplex (2 – 4 MHz) ultrasound was used for assessment of the intracranial arteries, and transcranial continuous-wave (4 – 8 MHz) Doppler for assessment of the periorbital arteries.

Ultrasonography records were sent from all participating centers to the core lab for central adjudication. Two investigators with advanced training in cerebral and pre-cerebral ultrasonography interpreted all written ultrasound findings in accordance with the multiparametric DEGUM criteria and the degree of ICA disease was reconciled by consensus, independent of other information. If ICA grading yielded a range of stenosis (e. g., 70 – 80 %), the final degree corresponded to its average value (i. e., 75 %; [Fig. 2]).

Fig. 2 Duplex ultrasonography showing severe narrowing of proximal ICA (arrow) with local aliasing and distal confetti sign A. According to the multiparametric DEGUM approach, ultrasonography yielded 70 – 80 % (75 %) stenosis of the proximal ICA A–D. Invasive angiography revealed 71 % stenosis according to NASCET grading E–F. CCA, common carotid artery.

Digital subtraction angiography of the carotid artery

All patients underwent DSA at the respective participating center with local procedural and technical regulations applied. Written declaration of the agreement was given on the locally used consent forms for both diagnostic and therapeutic angiography. In addition, external data consignment and study processing were accepted in writing by all patients on a separate study form confirmed by the local ethics boards.

Bi-planar common carotid artery and extracranial ICA angiography was acquired in all recruited subjects. General post-interventional intracranial ICA two-plane angiography was acquired. Digital imaging data was pseudonymized, saved on a compact disc and sent to the core lab. For this study, all angiographic images were re-evaluated regardless of locally performed measurements. Two expert neuroradiologists determined the NASCET type of stenosis independently of clinical information and the results of the ultrasonography studies. Both the smallest diameter within the ICA stenosis (B) and the normal far-distal ICA diameter with parallel walls beyond post-stenotic dilation (A) were measured and the percentage grade of stenosis was calculated ([1-B/A] × 100) [1]. To achieve optimal diameter measurements, the best plane (anterior-posterior or lateral) was selected to allow optimal perpendicular gauging of the contrasted vessel lumen. Contrast adjustments where applied in case of severe calcifications, high-grade stenosis or insufficient contrast media concentration.

Severe discrepancies between ultrasound and angiography data were registered by an independent third observer and re-evaluated by a neuroradiologist to allow data consistency and eradication of erroneous side selection.

Statistical analysis

Statistical analyses were performed with STATA software (version 12.1, StataCorp, College Station, TX). Continuous variables are presented as mean ± standard deviation (SD) for normally distributed data or as median (interquartile range, IQR) for skewed distributed data. Non-continuous variables are presented as percentages. The significance level was set at 0.05.

Using DSA as the reference standard, the classification accuracy of the overall DEGUM approach was derived from corresponding confusion matrices for multiple classes by dividing the sum of correctly classified categories of the disease by the total number of tests performed. For this purpose and to avoid an overlap in the head-to-head comparisons, DEGUM categories were defined as follows: normal (0 %), 10 % (1 – 14 %), 20 – 40 % (15 – 44 %), 50 % (45 – 54 %), 60 % (55 – 64 %), 70 % (65 – 74 %), 80 % (75 – 84 %), 90 % (85 – 99), occlusion (100 %).

We also calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and overall accuracy of DUS for identifying clinically relevant stenosis strata (i. e., normal, mild [1 – 49 %], moderate [50 – 69 %], severe [70 – 99 %], occlusion [100 %]), as previously described [1]. For this purpose, we computed true-positive, false-positive, true-negative and false-negative values for corresponding clinically relevant categories of the disease. To assess the precision of the accuracy estimates, the modified Wald method was used for computation of corresponding 95 % confidence intervals (95 % CI) [15].

To examine the diagnostic agreement between the DEGUM ultrasonography approach and angiography in the continuous range of percent stenosis (0 – 100 %), a Bland-Altman plot was drawn and the mean difference (i. e., bias) and the 95 % limits of agreement (i. e. mean ± 1.96 SD) were computed [16].

Cohen weighted κ was computed to assess the interrater reliability of the ultrasonography DEGUM approach by comparing stenosis categories independently obtained in a subset of 21 carotid arteries by two expert sonographers, as previously described [17]. We also evaluated the intra- and interrater reliability of percent stenosis measurements on angiography by calculating the two-way mixed intraclass correlation coefficient (ICC) for absolute agreement, where ICC values < 0.40 were considered poor, 0.40 – 0.59 fair, 0.60 – 0.74 good and 0.75 – 1.00 excellent [18]. To this end, a subset of randomly selected carotid arteries with varying degrees of the disease were reevaluated by both neuroradiologist raters independently and blinded to each other’s results after 3 months, and the initial measurements were compared to each other.

Results

Multicenter population and baseline characteristics

A total of 131 consecutive patients were enrolled at four participating centers. Of these patients, 10 were found ineligible for the final analyses for the following reasons: prolonged time interval between DUS and angiography, n = 1; DUS examinations not available, n = 2; invasive angiography not performed, n = 6; angiography not analyzable owing to severely ulcerated carotid plaques, n = 1.

For the validation analysis, we evaluated 163 DUS-angiography carotid artery pairs from 121 patients (median age was 69 [IQR, 16] years; 74 % were men). The patients’ baseline characteristics are detailed in [Table 1].

Table 1 Baseline characteristics of patients with duplex-angiography pairs (n = 121).

variable
age, median (IQR)	69 (16)
men, n (%)	90 (74)
history of risk factors, n (%)
arterial hypertension	90 (74)
hyperlipidemia	64 (53)
diabetes mellitus	30 (25)
tobacco use	34 (28)
coronary artery disease	24 (20)
indication for angiography, n (%)
symptomatic ICA disease	75 (62)
asymptomatic ICA disease	38 (31)
intracranial disease	8 (7)

The median elapsed time between DUS and angiography was 1 day (IQR, 2). No serious complications occurred from performing the index and the reference tests. Invasive angiography showed measurable steno-occlusive disease (1 – 99 % or occlusion) in 139/163 (85.3 %) of the internal carotid arteries. Steno-occlusive lesions were mainly found in the proximal portion of the ICA (96.4 %), mostly less than 1 cm in length (61.1 %) and predominantly of atherosclerotic cause (95.7 %), in-stent restenosis (3.6 %) and fibromuscular dysplasia (0.7 %).

Measures of diagnostic performance

The classification accuracy of DEGUM criteria to predict stenosis using increments of 10 % as compared to angiography was 34.9 % (95 % CI, 28.0 – 42.6). The accuracy did not change (37.4 %; 95 % CI, 29.3 – 46.2) when we evaluated the DEGUM criteria for lesions greater than or equal to 50 % (i. e., 50 %, 60 %, 70 %, 80 %, 90 %, occlusion). When applying wider and clinically relevant ranges of stenosis (i. e., 50 – 69 %, 70 – 99 %, occlusion), the overall classification accuracy of the DEGUM criteria increased to 55.2 % (95 % CI, 47.6 – 62.6). The overall classification accuracy also did not change when we excluded non-atherosclerotic causes of steno-occlusive disease (n = 6; 35.7 %; 95 % CI, 28.6 – 43.4).

Compared with angiography, DUS emerged as false-positive in almost one in two findings in the 50 – 69 % stenosis category, with 6 (20 %) being overrated and 7 (23.3 %) underrated. For the 70 – 99 % stenosis category, 31 studies were found to be false-positive on DUS, with 29 (34.9 %) being overrated and 2 (2.4 %) underrated. Occlusions were falsely classified as severe stenosis in 4 (40 %) out of 10 studies. [Table 2], [3] illustrate cross-classification and corresponding accuracy parameters of the DEGUM criteria for the identification of clinically relevant ranges of carotid steno-occlusive disease. The accuracy parameters for detecting clinically relevant NASCET degrees of extracranial internal carotid artery disease did not change when we repeated our analysis without non-atherosclerotic causes of steno-occlusive disease (n = 6; data not shown).

Table 2 Cross-classification of clinically relevant carotid disease categories by angiography and ultrasonography.

		ultrasonography, n
		normal	1 – 49 %	50 – 69 %	70 – 99 %	100 %	Total
angiography, n	normal	7	17	0	0	0	24
	1 – 49 %	1	8	6	4	0	19
	50 – 69 %	0	6	17	25	0	48
	70 – 99 %	0	1	7	52	4	64
	100 %	0	0	0	2	6	8
	total	8	32	30	83	10	163

Table 3 Accuracy parameters with corresponding 95 % confidence intervals for detecting clinically relevant NASCET degrees of extracranial internal carotid artery stenosis and occlusion by duplex ultrasonography.

degree of stenosis	sensitivity (95 % CI)	specificity (95 % CI)	PPV (95 % CI)	NPV (95 % CI)	accuracy (95 % CI)
< 50 %	42.1 (23.1 – 63.8)	83.3 (76.3 – 88.6)	25.0 (13.0 – 42.3)	91.6 (85.4 – 95.4)	78.5 (71.6 – 84.2)
50 – 99 %	90.2 (83.1 – 94.6)	76.5 (63.1 – 86.1)	89.4 (82.2 – 93.9)	78.0 (64.6 – 87.4)	85.9 (79.7 – 90.5)
50 – 69 %	35.4 (23.4 – 49.6)	88.7 (81.5 – 93.4)	56.7 (39.2 – 72.6)	76.7 (68.8 – 83.1)	73.0 (65.7 – 79.3)
70 – 99 %	81.3 (69.9 – 89.1)	68.7 (58.9 – 77.0)	62.3 (51.9 – 72.3)	85.0 (75.4 – 91.4)	73.6 (66.4 – 79.8)
100 %	75.0 (40.9 – 93.7)	97.4 (93.3 – 99.2)	60.0 (31.2 – 83.3)	98.7 (95.1 – 99.9)	96.3 (92.0 – 98.5)

The Bland-Altman plot revealed a 95 % limit of agreement interval of 50.89 % for the corresponding pairs of measurements, indicating that the degree of stenosis derived from DUS may be 23.61 % below or 27.28 % above the one derived from invasive angiography. The Bland-Altman plot is depicted in [Fig. 3].

Fig. 3 Bland-Altman plot comparing the degree of internal carotid artery stenosis derived from digital subtraction angiography and duplex ultrasonography (DUS) measurements. The mean difference ± 1.96 standard deviation (SD) are shown as dashed and dotted lines, respectively. The size of the circles represents the number of measurements.

Test reliability

The interrater reliability of the multiparametric DEGUM ultrasonography approach was almost perfect (κ: 0.92; 95 % CI, 0.88 – 0.96). The intraclass correlation coefficients for NASCET-type angiography measurements between the two expert neuroradiologists (ICC: 0.94; 95 % CI, 0.82 – 0.98) and repeated measurements among both raters (ICC₁: 0.98; 95 % CI 0.82 – 0.98 and ICC₂: 0.96; 95 % CI 0.82 – 0.98) indicated excellent intra- and interrater reliability in our study.

Discussion

Our multicenter study suggests that the revised multiparametric DEGUM criteria do not enhance the diagnostic accuracy of duplex ultrasonography in the identification of narrow carotid artery disease categories when compared with the reference standard DSA. The interrater reliability of the ultrasonography approach, however, appeared to be excellent when the test was performed by expert sonographers.

There have been numerous attempts to enhance the diagnostic accuracy of ultrasonography grading of carotid artery disease [19] [20] [21] [22]. However, none of the currently recommended ultrasonography criteria seem to be clearly superior. Nonetheless, most recent approaches have defined disease categories exclusively on the basis of uniparametric (i. e., intra-stenotic) measurements. The implementation of a multitude of local and pre- and post-stenotic criteria and corresponding disease classification in small categories appeared to be the major strength of the DEGUM criteria, as it was possible for the first time to pinpoint the degree of stenosis and closely monitor patients with progression of the disease. On the downside, while the use of multiple ultrasonography criteria may increase the specificity of a test, at the same time it may degrade its sensitivity as suggested by our report [13]. The number of true-positives across all DEGUM stenosis strata was low in our study and the variability of stenosis measurements (as shown by the 95 % limit of agreement interval of 50.89 %) was considerable and clinically important. Particularly the latter observation conflicts with the methodological attempt of DEGUM criteria to narrow down disease categories on DUS. However, this variability is expected from wide variations of the peak systolic velocity across all stenosis strata [19].

When we considered broader and clinically relevant NASCET-type disease categories, the predictive values of the multiparametric DEGUM approach increased, but not beyond values demonstrated for other recommended grading criteria. In fact, a recent meta-analysis of 17 prospective diagnostic accuracy studies with 921 patients showed a sensitivity of 36 % and a specificity of 91 % for DUS identification of 50 – 69 % carotid stenosis, and 89 % and 84 %, respectively, for identification of 70 – 99 % carotid stenosis, when compared with invasive angiography [23]. Another meta-analysis arrived at similar accuracy estimates for DUS grading of carotid stenosis [24]. It is still unclear whether the test performance of DUS carotid disease grading is generally limited by its technical ability and whether further adjustments in ultrasonography criteria may eventually result in an increase in diagnostic accuracy in contrast to the results of a recent correlation analysis [7]. On the other side, advanced methods such as plaque imaging and the utilization of contrast agents might aid in the identification of patients who may benefit from revascularization procedures, regardless of the degree of stenosis [25].

Our observation of limited diagnostic agreement between stenosis grading via angiography based on NASCET criteria and DUS using the DEGUM approach is clinically relevant. It supports the previous suggestion that a confirmatory noninvasive neurovascular test should be employed in patients who underwent DUS screening and were deemed potential candidates for revascularization procedures [26]. Conversely, using DUS alone might put patients at risk of being falsely classified as either conservative or interventional/surgical management of the corresponding disease category. For instance, if DUS was the sole imaging modality in our study cohort, eight carotid arteries underrated as mild or moderate stenosis in the 70 – 99 % stenosis category would not have been qualified for carotid revascularization procedure.

Moreover, four arteries were misclassified as occlusion that otherwise would have been suitable for revascularization. While one of these arteries was eventually classified as 91 % stenosis on DSA, the remaining three arteries were classified as near-occlusion (i. e., 99 %), a category for which the revised DEGUM approach does not provide specific criteria. Since current evidence still favors treatment of these patients or at least requires individual consideration of it, specific criteria should be developed and implemented in future randomized trials that include patients with ICA near-occlusion [27].

It should be acknowledged that recent clinical trials showing a benefit of carotid endarterectomy solely categorized the disease by two-plane invasive angiography [1] [2] [3]. Two-plane invasive angiography still remains the gold standard for carotid stenosis grading. However, its continued implementation as such, despite evolving noninvasive imaging modalities (e. g., CT angiography), might be rather historical than empirical [28]. With different shapes of stenosis, diameter reduction on invasive angiography may differ remarkably from cross-sectional area reduction that is reflected by hemodynamic disturbances on DUS. While the implementation of the true reference standard including planimetric measurements of pathologic specimens is barely possible, imperfect gold standard bias should therefore be considered when interpreting results of validation studies where invasive angiography was used instead [29]. At least, recently published studies comparing the multiparametric DEGUM approach with noninvasive imaging modalities such as CT angiography showed some concordance between both tests in the identification of carotid artery disease [30] [31] [32].

Several limitations of our study need to be acknowledged. First, the diagnostic accuracy results for the multiparametric DEGUM approach are only conditionally generalizable to other vascular ultrasonography laboratories, as any laboratory should validate newly adopted diagnostic criteria individually to determine test performance. In addition, about two-thirds of included patients underwent reference testing for assessment of symptomatic carotid artery disease, leaving the possibility that our results are prone to spectrum bias with limited generalizability to asymptomatic patient cohorts. On the other hand, accurate grading of the disease is particularly required in the acute phase of stroke as carotid revascularization procedures are time-sensitive with the largest benefit given in the first two weeks after the index stroke [1] [2]. Thus, our report reflects real-world practice and the exclusion of patients with symptomatic disease may have led to inflated diagnostic accuracy estimates. Second, most of the included patients were referred for reference testing when further diagnostic assessment or therapeutic intervention was deemed necessary following positive findings on noninvasive imaging. The presence of a partial verification bias, therefore, is likely and may have falsely increased sensitivity and decreased specificity in our study [29]. Nonetheless, given the invasive nature of angiography, a different study design referring consecutive patients for reference testing regardless of noninvasive imaging results would not have been ethical. Lastly, our data do not allow any inference whether the DEGUM approach performs better or worse than other commonly used DUS grading criteria as head-to-head comparisons, for instance with the widely-used Society of Radiologists in Ultrasound Consensus Conference criteria, are lacking [19].

The strengths of our study include its prospective multicenter design, the prespecified sample of patients undergoing standardized assessment of carotid artery disease and the first ever comparison of the multiparametric DEGUM ultrasonography approach with the reference standard DSA.

Conclusion

At laboratories experienced with ultrasound grading of the extracranial ICA, the revised DEGUM multiparametric ultrasonography criteria do not eliminate the need for a confirmatory test for the identification of clinically relevant grades of the disease.

No conflict of interest has been declared by the author(s).

^* This study was funded by the Roland-Ernst-Foundation (Project # 6/14)

• References

• 1 Barnett HJ, Taylor DW, Eliasziw M. et al. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators. N Engl J Med 1998; 339: 1415-1425
• 2 European Carotid Surgery Trialists’ Collaborative Group. Randomised trial of endarterectomy for recently symptomatic carotid stenosis: final results of the MRC European Carotid Surgery Trial (ECST). Lancet 1998; 351: 1379-1387
• 3 Halliday A, Harrison M, Hayter E. et al. 10-Year stroke prevention after successful carotid endarterectomy for asymptomatic stenosis (ACST-1): a multicentre randomized trial. Lancet 2010; 376: 1074-1084
• 4 Muluk SC, Muluk VS, Sugimoto H. et al. Progression of asymptomatic carotid stenosis: a natural history study in 1004 patients. J Vasc Surg 1999; 29: 208-214
• 5 Rosenfield K, Matsumura JS, Chaturvedi S. et al. Randomized Trial of Stent versus Surgery for Asymptomatic Carotid Stenosis. N Engl J Med 2016; 374: 1011-1020
• 6 Barlinn K, Alexandrov AV. Vascular imaging in stroke: comparative analysis. Neurotherapeutics 2011; 8: 340-348
• 7 Beach KW, Leotta DF, Zierler RE. Carotid Doppler velocity measurements and anatomic stenosis: correlation is futile. Vasc Endovascular Surg 2012; 46: 466-474
• 8 Mozzini C, Roscia G, Casadei A. et al. Searching the perfect ultrasonic classification in assessing carotid artery stenosis: comparison and remarks upon the existing ultrasound criteria. J Ultrasound 2016; 19: 83-90
• 9 Gray C, Murray SA, Connolly M. et al. The implications of different criteria for grading carotid artery stenosis by duplex ultrasound. J Clin Ultrasound 2015; 43: 563-566
• 10 Arning C, Widder B, von Reutern GM. et al. Revision of DEGUM ultrasound criteria for grading internal carotid artery stenoses and transfer to NASCET measurement. Ultraschall in Med 2010; 31: 251-257
• 11 Klingelhöfer J. Ultrasonography of carotid stenosis. Int J Clin Neurosci Ment Health 2014; 1: S04
• 12 von Reutern GM, Goertler MW, Bornstein NM. et al. Grading carotid stenosis using ultrasonic methods. Stroke 2012; 43: 916-921
• 13 Alexandrov AV, Needleman L. Carotid artery stenosis: making complex assessments of a simple problem or simplifying approach to a complex disease?. Stroke 2012; 43: 627-628
• 14 Bossuyt PM, Reitsma JB, Bruns DE. et al. STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies. BMJ 2015; 351: h5527
• 15 Lewis JR, Sauro J. When 100% really isn’t 100%: improving the accuracy of small-sample estimates of completion rates. J Usability Studies 2006; 1: 136-150
• 16 Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; i: 307-310
• 17 Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968; 70: 213-220
• 18 Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86: 420-428
• 19 Grant EG, Benson CB, Moneta GL. et al. Carotid artery stenosis: gray-scale and Doppler US diagnosis–Society of Radiologists in Ultrasound Consensus Conference. Radiology 2003; 229: 340-346
• 20 Oates CP, Naylor AR, Hartshorne T. et al. Joint recommendations for reporting carotid ultrasound investigations in the United Kingdom. Eur J Vasc Endovasc Surg 2009; 251-261
• 21 Beach KW, Bergelin RO, Leotta DF. et al. Standardized ultrasound evaluation of carotid stenosis for clinical trials: University of Washington Ultrasound Reading Center. Cardiovasc Ultrasound 2010; 8: 39
• 22 Hathout GM, Fink JR, El-Saden SM. et al. Sonographic NASCET index: a new doppler parameter for assessment of internal carotid artery stenosis. AJNR Am J Neuroradiol 2005; 26: 68-75
• 23 Wardlaw JM, Chappell FM, Best JJ. et al. Non-invasive imaging compared with intra-arterial angiography in the diagnosis of symptomatic carotid stenosis: a metaanalysis. Lancet 2006; 367: 1503-1512
• 24 Chappell FM, Wardlaw JM, Young GR. et al. Carotid artery stenosis: accuracy of noninvasive tests—individual patient data meta-analysis. Radiology 2009; 251: 493-502
• 25 Rafailidis V, Charitanti A, Tegos T. et al. Contrast-enhanced ultrasound of the carotid system: a review of the current literature. J Ultrasound 2017; 20: 97-109
• 26 Patel SG, Collie DA, Wardlaw JM. Outcome, observer reliability, and patient preferences if CTA, MRA, or Doppler ultrasound were used, individually or together, instead of digital subtraction angiography before carotid endarterectomy. J Neurol Neurosurg Psychiatry 2002; 73: 21-28
• 27 Mylonas SN, Antonopoulos CN, Moulakakis KG. et al. Management of Patients with Internal Carotid Artery Near-total Occlusion: An Updated Meta-analysis. Ann Vasc Surg 2015; 29: 1664-1672
• 28 Fox AJ. How to measure carotid stenosis. Radiology 1993; 186: 316-318
• 29 Kohn MA, Carpenter CR, Newman TB. Understanding the direction of bias in studies of diagnostic test accuracy. Acad Emerg Med 2013; 20: 1194-1206
• 30 Matz O, Nikoubashman O, Rajkumar P. et al. Grading of proximal internal carotid artery (ICA) stenosis by Doppler/duplex ultrasound (DUS) and computed tomographic angiography (CTA): correlation and interrater reliability in real-life practice. Acta Neurol Belg 2017; 117: 183-188
• 31 Müller M, Agten CA, Österreich M. et al. Assessing internal carotid artery stenosis with a semiautomated computed tomography angiography tool and duplex ultrasound. J Vasc Surg 2015; 61: 1449-1456
• 32 Barlinn K, Floegel T, Kitzler HH. et al. Multi-parametric ultrasound criteria for internal carotid artery disease-comparison with CT angiography. Neuroradiology 2016; 58: 845-851