Ultraschall Med 2018; 39(02): 198-205
DOI: 10.1055/s-0043-106737
Original Article
© Georg Thieme Verlag KG Stuttgart · New York

Validation of the Automated Electronic Microemboli Detection System in Patients Undergoing Carotid Endarterectomy

Bewertung des automatisierten elektronischen Mikroemboli-Detektionssystems bei Patienten mit Carotis-Endarteriektomie
Tesse Leunissen
1   Vascular Surgery, University Medical Center Utrecht, Netherlands
,
Daniel van Vriesland
2   Clinical Neurophysiology, University Medical Center Utrecht, Netherlands
,
Hester den Ruijter
3   Experimental Cardiology, University Medical Center Utrecht, Netherlands
4   Neurology/Clinical Neurophysiology, Academic Hospital Maastricht, Netherlands
,
Frans Moll
1   Vascular Surgery, University Medical Center Utrecht, Netherlands
,
Werner Mess
4   Neurology/Clinical Neurophysiology, Academic Hospital Maastricht, Netherlands
,
Gert Jan de Borst
1   Vascular Surgery, University Medical Center Utrecht, Netherlands
› Author Affiliations
Further Information

Publication History

13 June 2016

05 January 2017

Publication Date:
06 April 2017 (online)

Abstract

Purpose To assess the diagnostic value of automatic embolus detection software (AEDS) in transcranial Doppler (TCD) monitoring for the detection of solid microemboli in patients at risk for perioperative stroke during carotid endarterectomy (CEA).

Materials and Methods In 50 patients undergoing CEA, perioperative TCD registration was recorded. All recorded events, identified and saved by the AEDS, were analyzed off-line doubly by two human experts (HEs) within a time frame of > 4 months. The inter- and intraobserver variability was assessed. The overall agreement with the HEs, the sensitivity, specificity, negative and positive predictive values (NPV and PPV) of the AEDS were computed for different cut-offs (patient displaying perioperative 5, 10, 20, 25, or 50 microemboli).

Results 77 233 events were analyzed. The inter- and intraobserver variability was good (min κ = 0.72, max κ = 0.79). AEDS and the HEs identified 760 and 470 solid emboli, respectively. The agreement between AEDS and the HEs for solid emboli detection was poor (κ = 0.24, SE = 0.016). The specificity and NPV were high (99.2 % and 99.6 %) but the sensitivity and PPV were low (30.6 % and 19.8 %). Applying a threshold of > 20 microemboli resulted in the best sensitivity (100.0 %), specificity (84.4 %), PPV (42.7 %), NPV (100.0 %) and area under the curve (0.898). However, 58.3 % of the patients were false positive as classified by AEDS.

Conclusion In this validation cohort, AEDS has insufficient agreement with HEs in the identification of solid emboli. AEDS and HEs disagree with respect to the identification of specific patients at risk. Therefore, AEDS cannot be used as a standalone system to identify patients at risk for perioperative stroke during CEA.

Zusammenfassung

Ziel Untersuchung des diagnostischen Werts der automatischen Emboli-Detektionssoftware (AEDS) in der transkraniellen Dopplersonografie (TCD) für den Nachweis von soliden Mikroemboli bei Patienten mit Risiko für perioperative Schlaganfälle bei Carotis-Endarteriektomie (CEA).

Material und Methoden Bei 50 Patienten mit CEA wurde eine perioperative TCA-Registrierung dokumentiert. Alle mittels AEDS aufgenommenen und gesicherten Ereignisse wurden innerhalb von > 4 Monaten doppelt off-line durch zwei Experten (HEs) ausgewertet und die Inter- und Intraobserver-Variabilität wurde bestimmt. Die Gesamtübereinstimmung der beiden HEs und die Sensitivität, Spezifität, negative und positive Vorhersagewerte (NPV und PPV) der AEDS wurden für verschiedene Cut-offs (Patienten mit 5, 10, 20, 25 oder 50 Mikroemboli) berechnet.

Ergebnisse 77 233 Ereignisse wurden ausgewertet. Die Inter- und Intraobserver-Variabilität war gut (min κ = 0,72; max κ = 0,79). AEDS identifizierte 760 solide Emboli, die HEs fanden 470. Die Übereinstimmung von AEDS und HEs beim Nachweis solider Emboli war schlecht (κ = 0,24; SE = 0,016). Spezifität und NPV waren hoch (99,2 % and 99,6 %), Sensitivität und PPV hingegen niedrig (30,6 % und 19,8 %). Bei einem Grenzwert von > 20 Mikroemboli wurden die höchsten Raten für Sensitivität (100,0 %), Spezifität (84,4 %), PPV (42,7 %), NPV (100,0 %) und Area-under-the-curve (0,898) erzielt. Allerdings wurden 58,3 % der Patienten durch AEDS als falsch-positiv klassifiziert.

Schlussfolgerung In dieser Kohorte zeigte AEDS eine unzureichende Übereinstimmung mit den HEs bei der Identifizierung von Risikopatienten. Deshalb kann AEDS nicht als autonomes Verfahren angewandt werden, um Patienten mit Risiko für einen perioperativen Schlaganfalls bei CEA zu identifizieren.

 
  • References

  • 1 Executive Committee for the Asymptomatic Carotid Atherosclerosis Study. Endarterectomy for asymptomatic carotid artery stenosis. JAMA 1995; 273: 1421-1428
  • 2 Halliday A. Mansfield A. Marro J. et al. Prevention of disabling and fatal strokes by successful carotid endarterectomy in patients without recent neurological symptoms: randomised controlled trial. Lancet 2004; 363: 1491-1502
  • 3 de Borst GJ. Moll FL. van de Pavoordt HD. et al. Stroke from carotid endarterectomy: when and how to reduce perioperative stroke rate?. Eur J Vasc Endovasc Surg 2001; 21: 484-489
  • 4 Radak D. Popovic AD. Radicevic S. et al. Immediate reoperation for perioperative stroke after 2250 carotid endarterectomies: differences between intraoperative and early postoperative stroke. J Vasc Surg 1999; 30: 245-251
  • 5 Bonati LH. Dobson J. Featherstone RL. et al. Long-term outcomes after stenting versus endarterectomy for treatment of symptomatic carotid stenosis: the International Carotid Stenting Study (ICSS) randomised trial. Lancet 2015; 385: 529-538
  • 6 Huibers A. Calvet D. Kennedy F. et al. Mechanism of Procedural Stroke Following Carotid Endarterectomy or Carotid Artery Stenting Within the International Carotid Stenting Study (ICSS) Randomised Trial. Eur J Vasc Endovasc Surg 2015; 50: 281-288
  • 7 de Groot PG. Urbanus RT. Roest M. Platelet interaction with the vessel wall. Handb Exp Pharmacol 2012; 210: 87-110
  • 8 Van Lammeren GW. Van De Mortel RH. Visscher M. et al. Spontaneous preoperative microembolic signals detected with transcranial Doppler are associated with vulnerable carotid plaque characteristics. J Cardiovasc Surg (Torino) 2014; 55: 375-380
  • 9 King A. Markus HS. Doppler embolic signals in cerebrovascular disease and prediction of stroke risk: a systematic review and meta-analysis. Stroke 2009; 40: 3711-3717
  • 10 Horn J. Naylor AR. Laman DM. et al. Identification of patients at risk for ischaemic cerebral complications after carotid endarterectomy with TCD monitoring. Eur J Vasc Endovasc Surg 2005; 30: 270-274
  • 11 Pennekamp CW. Moll FL. de Borst GJ. The potential benefits and the role of cerebral monitoring in carotid endarterectomy. Curr Opin Anaesthesiol 2011; 24: 693-697
  • 12 Russell D. Brucher R. Online automatic discrimination between solid and gaseous cerebral microemboli with the first multifrequency transcranial Doppler. Stroke 2002; 33: 1975-1980
  • 13 Ringelstein EB. Droste DW. Babikian VL. et al. Consensus on microembolus detection by TCD. International Consensus Group on Microembolus Detection. Stroke 1998; 29: 725-729
  • 14 Van Zuilen EV. Mess WH. Jansen C. et al. Automatic embolus detection compared with human experts. A Doppler ultrasound study. Stroke 1996; 27: 1840-1843
  • 15 Devuyst G. Darbellay GA. Vesin JM. et al. Automatic classification of HITS into artifacts or solid or gaseous emboli by a wavelet representation combined with dual-gate TCD. Stroke 2001; 32: 2803-2809
  • 16 Keunen RW. Hoogenboezem R. Wijnands R. et al. Introduction of an embolus detection system based on analysis of the transcranial Doppler audio-signal. J Med Eng Technol 2008; 32: 296-304
  • 17 Naylor AR. Rothwell PM. Bell PR. Overview of the principal results and secondary analyses from the European and North American randomised trials of endarterectomy for symptomatic carotid stenosis. Eur J Vasc Endovasc Surg 2003; 26: 115-129
  • 18 Mayberg MR. Wilson SE. Yatsu F. et al. Carotid endarterectomy and prevention of cerebral ischemia in symptomatic carotid stenosis. Veterans Affairs Cooperative Studies Program 309 Trialist Group. JAMA 1991; 266: 3289-3294
  • 19 European Carotid Surgery Trialists’ Collaborative Group. Randomised trial of endarterectomy for recently symptomatic carotid stenosis: final results of the MRC European Carotid Surgery Trial (ECST). Lancet 1998; 351: 1379-1387
  • 20 Jansen C. Vriens EM. Eikelboom BC. et al. Carotid endarterectomy with transcranial Doppler and electroencephalographic monitoring. A prospective study in 130 operations. Stroke 1993; 24: 665-669
  • 21 van der Schaaf IC. Horn J. Moll FL. et al. Transcranial Doppler monitoring after carotid endarterectomy. Ann Vasc Surg 2005; 19: 19-24
  • 22 Joseph FleissBL. Myunghee CP. Statistical Methods For Rates And Proportions. John Wiley And Sons Ltd; 2003: 800 p
  • 23 Markus HS. MacKinnon A. Asymptomatic embolization detected by Doppler ultrasound predicts stroke risk in symptomatic carotid artery stenosis. Stroke 2005; 36: 971-975
  • 24 Mackinnon AD. Aaslid R. Markus HS. Ambulatory transcranial Doppler cerebral embolic signal detection in symptomatic and asymptomatic carotid stenosis. Stroke 2005; 36: 1726-1730
  • 25 Tsivgoulis G. Kerasnoudis A. Krogias C. et al. Clopidogrel load for emboli reduction in patients with symptomatic carotid stenosis undergoing urgent carotid endarterectomy. Stroke 2012; 43: 1957-1960
  • 26 Payne DA. Jones CI. Hayes PD. et al. Beneficial effects of clopidogrel combined with aspirin in reducing cerebral emboli in patients undergoing carotid endarterectomy. Circulation 2004; 109: 1476-1481
  • 27 Levi CR. Roberts AK. Fell G. et al. Transcranial Doppler microembolus detection in the identification of patients at high risk of perioperative stroke. Eur J Vasc Endovasc Surg 1997; 14: 170-176
  • 28 Cullinane M. Reid G. Dittrich R. et al. Evaluation of new online automated embolic signal detection algorithm, including comparison with panel of international experts. Stroke 2000; 31: 1335-1341
  • 29 Markus HS. Punter M. Can transcranial Doppler discriminate between solid and gaseous microemboli? Assessment of a dual-frequency transducer system. Stroke 2005; 36: 1731-1734
  • 30 Schoenburg M. Baer J. Schwarz N. et al. EmboDop: insufficient automatic microemboli identification. Stroke 2006; 37: 342-343
  • 31 Evans DH. Embolus differentiation using multifrequency transcranial Doppler. Stroke 2006; 37: 1641
  • 32 Molloy J. Markus HS. Multigated Doppler ultrasound in the detection of emboli in a flow model and embolic signals in patients. Stroke 1996; 27: 1548-1552
  • 33 Georgiadis D. Kaps M. Siebler M. et al. Variability of Doppler microembolic signal counts in patients with prosthetic cardiac valves. Stroke 1995; 26: 439-443
  • 34 Kemeny V. Droste DW. Hermes S. et al. Automatic embolus detection by a neural network. Stroke 1999; 30: 807-810