Download PDF
J Neurol Surg B Skull Base 2022; 83(05): 485-495
DOI: 10.1055/s-0041-1740621
Original Article

Interpretable Machine Learning–Based Prediction of Intraoperative Cerebrospinal Fluid Leakage in Endoscopic Transsphenoidal Pituitary Surgery: A Pilot Study

Pier Paolo Mattogno‡   *
1   Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy
,
Valerio M. Caccavella  ‡   *
1   Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy
,
Martina Giordano
1   Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy
,
Quintino G. D'Alessandris
1   Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy
,
Sabrina Chiloiro
2   Department of Endocrinology, Fondazione Policlinico Universitario A. Gemelli Istituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy
,
Leonardo Tariciotti
3   Unit of Neurosurgery, Fondazione Istituto di Ricovero e Cura a Carattere Scientifico Cà Granda Ospedale Maggiore Policlinico, Milan, Italy
4   University of Milan, Milan, Italy
,
Alessandro Olivi
1   Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy
,
Liverana Lauretti
1   Department of Neurosurgery, Fondazione Policlinico Universitario A. Gemell iIstituto di Ricovero e Cura a Carattere Scientifico Università Cattolica del Sacro Cuore, Rome, Italy
› Author Affiliations
› Further Information
    Permissions and Reprints

Abstract

Purpose Transsphenoidal surgery (TSS) for pituitary adenomas can be complicated by the occurrence of intraoperative cerebrospinal fluid (CSF) leakage (IOL). IOL significantly affects the course of surgery predisposing to the development of postoperative CSF leakage, a major source of morbidity and mortality in the postoperative period. The authors trained and internally validated the Random Forest (RF) prediction model to preoperatively identify patients at high risk for IOL. A locally interpretable model-agnostic explanations (LIME) algorithm is employed to elucidate the main drivers behind each machine learning (ML) model prediction.

Methods The data of 210 patients who underwent TSS were collected; first, risk factors for IOL were identified via conventional statistical methods (multivariable logistic regression). Then, the authors trained, optimized, and audited a RF prediction model.

Results IOL reported in 45 patients (21.5%). The recursive feature selection algorithm identified the following variables as the most significant determinants of IOL: Knosp's grade, sellar Hardy's grade, suprasellar Hardy's grade, tumor diameter (on X, Y, and Z axes), intercarotid distance, and secreting status (nonfunctioning and growth hormone [GH] secreting). Leveraging the predictive values of these variables, the RF prediction model achieved an area under the curve (AUC) of 0.83 (95% confidence interval [CI]: 0.78; 0.86), significantly outperforming the multivariable logistic regression model (AUC = 0.63).

Conclusion A RF model that reliably identifies patients at risk for IOL was successfully trained and internally validated. ML-based prediction models can predict events that were previously judged nearly unpredictable; their deployment in clinical practice may result in improved patient care and reduced postoperative morbidity and healthcare costs.

Keywords

random forest - cerebrospinal fluid leak - machine learning - pituitary adenoma - transsphenoidal surgery - pituitary surgery

Ethical Approval

Ethical approval was waived by the local Ethics Committee in view of the retrospective nature of the study and all the procedures being performed were part of the routine care.


Consent to Participate

Informed consent was obtained from all individual participants included in the study.


Availability of Data and Material

The dataset that supports the findings of this study are available from the corresponding author, M.G., on request.


Code Availability

The source code employed to develop the herein presented machine learning model is available at the following GitHub repository: https://github.com/valerio-mc/ML-fistola-pituitary .


* These authors contributed equally to the work.


Supplementary Material



Publication History

Received: 12 March 2021

Accepted: 12 November 2021

Article published online:
16 January 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Chen CJ, Ironside N, Pomeraniec IJ. et al. Microsurgical versus endoscopic transsphenoidal resection for acromegaly: a systematic review of outcomes and complications. Acta Neurochir (Wien) 2017; 159 (11) 2193-2207
    CrossrefPubMedGoogle Scholar
  • 2 Dhandapani S, Singh H, Negm HM, Cohen S, Anand VK, Schwartz TH. Cavernous sinus invasion in pituitary adenomas: systematic review and pooled data meta-analysis of radiologic criteria and comparison of endoscopic and microscopic surgery. World Neurosurg 2016; 96: 36-46
    CrossrefPubMedGoogle Scholar
  • 3 Fatemi N, Dusick JR, Mattozo C. et al. Pituitary hormonal loss and recovery after transsphenoidal adenoma removal. Neurosurgery 2008; 63 (04) 709-718 , discussion 718–719
    CrossrefPubMedGoogle Scholar
  • 4 Przybylowski CJ, Dallapiazza RF, Williams BJ. et al. Primary versus revision transsphenoidal resection for nonfunctioning pituitary macroadenomas: matched cohort study. J Neurosurg 2017; 126 (03) 889-896
    CrossrefPubMedGoogle Scholar
  • 5 Strickland BA, Lucas J, Harris B. et al. Identification and repair of intraoperative cerebrospinal fluid leaks in endonasal transsphenoidal pituitary surgery: surgical experience in a series of 1002 patients. J Neurosurg 2018; 129 (02) 425-429
    CrossrefPubMedGoogle Scholar
  • 6 Zhang C, Ding X, Lu Y, Hu L, Hu G. Cerebrospinal fluid rhinorrhoea following transsphenoidal surgery for pituitary adenoma: experience in a Chinese centre. Acta Otorhinolaryngol Ital 2017; 37 (04) 303-307
    CrossrefPubMedGoogle Scholar
  • 7 Karnezis TT, Baker AB, Soler ZM. et al. Factors impacting cerebrospinal fluid leak rates in endoscopic sellar surgery. Int Forum Allergy Rhinol 2016; 6 (11) 1117-1125
    CrossrefPubMedGoogle Scholar
  • 8 Mehta GU, Oldfield EH. Prevention of intraoperative cerebrospinal fluid leaks by lumbar cerebrospinal fluid drainage during surgery for pituitary macroadenomas. J Neurosurg 2012; 116 (06) 1299-1303
    CrossrefPubMedGoogle Scholar
  • 9 Conger A, Zhao F, Wang X. et al. Evolution of the graded repair of CSF leaks and skull base defects in endonasal endoscopic tumor surgery: trends in repair failure and meningitis rates in 509 patients. J Neurosurg 2018; 130 (03) 861-875
    CrossrefPubMedGoogle Scholar
  • 10 Zhou Q, Yang Z, Wang X. et al. Risk factors and management of intraoperative cerebrospinal fluid leaks in endoscopic treatment of pituitary adenoma: analysis of 492 patients. World Neurosurg 2017; 101: 390-395
    CrossrefPubMedGoogle Scholar
  • 11 Jakimovski D, Bonci G, Attia M. et al. Incidence and significance of intraoperative cerebrospinal fluid leak in endoscopic pituitary surgery using intrathecal fluorescein. World Neurosurg 2014; 82 (3,4): e513-e523
    CrossrefPubMedGoogle Scholar
  • 12 Senders JT, Staples PC, Karhade AV. et al. Machine learning and neurosurgical outcome prediction: a systematic review. World Neurosurg 2018; 109: 476-486.e1
    CrossrefPubMedGoogle Scholar
  • 13 Staartjes VE, Serra C, Muscas G. et al. Utility of deep neural networks in predicting gross-total resection after transsphenoidal surgery for pituitary adenoma: a pilot study. Neurosurg Focus 2018; 45 (05) E12
    CrossrefPubMedGoogle Scholar
  • 14 Staartjes VE, Zattra CM, Akeret K. et al. Neural network-based identification of patients at high risk for intraoperative cerebrospinal fluid leaks in endoscopic pituitary surgery. J Neurosurg 2019; (e-pub ahead of print) DOI: 10.3171/2019.4.JNS19477.
    CrossrefPubMedGoogle Scholar
  • 15 Ribeiro MT, Singh S, Guestrin C. Why Should I Trust You?”: Explaining the predictions of any classifier. Accessed December 2, 2021 at: https://aclanthology.org/N16-3020.pdf
    PubMedGoogle Scholar
  • 16 Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015; 162 (01) 55-63 DOI: 10.7326/M14-0697.
    CrossrefPubMedGoogle Scholar
  • 17 Breiman L. Random forests. Mach Learn 2001; 45: 5-32
    CrossrefPubMedGoogle Scholar
  • 18 Liaw A, Wiener M. Classification and regression by random forest. R News 2002; 2: 18-22
    PubMedGoogle Scholar
  • 19 Cutler A, Cutler DR, Stevens JR. Random forests. In: Zhang C, Ma Y. eds. Ensemble Machine Learning. Boston, MA: Springer; 2012: 157-175
    Google Scholar
  • 20 Kursa MB, Rudnicki WR. Feature selection with the Boruta package. J Stat Softw 2010; 36 (11) 1-13
    CrossrefPubMedGoogle Scholar
  • 21 Degenhardt F, Seifert S, Szymczak S. Evaluation of variable selection methods for random forests and omics data sets. Brief Bioinform 2019; 20 (02) 492-503
    CrossrefPubMedGoogle Scholar
  • 22 Sakr S, Elshawi R, Ahmed AM. et al. Comparison of machine learning techniques to predict all-cause mortality using fitness data: the Henry ford exercIse testing (FIT) project. BMC Med Inform Decis Mak 2017; 17 (01) 174
    CrossrefPubMedGoogle Scholar
  • 23 Chawla NV, Bowyer K, Hall LO. et al. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 2002; 16 (01) 321-357
    CrossrefPubMedGoogle Scholar
  • 24 DiCiccio TJ, Efron B. Bootstrap confidence intervals. Institute of Mathematical Statistics 1996; 11 (03) 189-228
    PubMedGoogle Scholar
  • 25 Park JH, Choi JH, Kim YI, Kim SW, Hong YK. Modified graded repair of cerebrospinal fluid leaks in endoscopic endonasal transsphenoidal surgery. J Korean Neurosurg Soc 2015; 58 (01) 36-42
    CrossrefPubMedGoogle Scholar
  • 26 Hadad G, Bassagasteguy L, Carrau RL. et al. A novel reconstructive technique after endoscopic expanded endonasal approaches: vascular pedicle nasoseptal flap. Laryngoscope 2006; 116 (10) 1882-1886
    CrossrefPubMedGoogle Scholar
  • 27 Cavallo LM, Solari D, Somma T, Cappabianca P. The 3F (fat, flap, and flash) technique for skull base reconstruction after endoscopic endonasal suprasellar approach. World Neurosurg 2019; 126: 439-446
    CrossrefPubMedGoogle Scholar
  • 28 Ostrom QT, Gittleman H, Xu J. et al. CBTRUS statistical report: Primary brain and other central nervous system tumors diagnosed in the United States in 2009–2013. Neuro Oncol 2017; 18 (Suppl. 05) v1-v75
    CrossrefPubMedGoogle Scholar
  • 29 Komotar RJ, Starke RM, Raper DM, Anand VK, Schwartz TH. Endoscopic endonasal compared with microscopic transsphenoidal and open transcranial resection of giant pituitary adenomas. Pituitary 2012; 15 (02) 150-159
    CrossrefPubMedGoogle Scholar
  • 30 Patel PN, Stafford AM, Patrinely JR. et al. Risk Factors for Intraoperative and postoperative cerebrospinal fluid leaks in endoscopic transsphenoidal sellar surgery. Otolaryngol Head Neck Surg 2018; 158 (05) 952-960
    CrossrefPubMedGoogle Scholar
  • 31 Xue H, Wang X, Yang Z, Bi Z, Liu P. Risk factors and outcomes of cerebrospinal fluid leak related to endoscopic pituitary adenoma surgery. Br J Neurosurg 2020; 34 (04) 447-452
    CrossrefPubMedGoogle Scholar
  • 32 Cavallo LM, Messina A, Cappabianca P. et al. Endoscopic endonasal surgery of the midline skull base: anatomical study and clinical considerations. Neurosurg Focus 2005; 19 (01) E2
    PubMedGoogle Scholar
  • 33 Fleseriu M, Karavitaki N. Non-functioning pituitary adenomas, not all the same and certainly not boring!. Pituitary 2018; 21: 109-110
    CrossrefPubMedGoogle Scholar
  • 34 Snow RB, Johnson CE, Morgello S, Lavyne MH, Patterson Jr RH. Is magnetic resonance imaging useful in guiding the operative approach to large pituitary tumors?. Neurosurgery 1990; 26 (05) 801-803
    CrossrefPubMedGoogle Scholar
  • 35 Wei L, Lin SA, Fan K, Xiao D, Hong J, Wang S. Relationship between pituitary adenoma texture and collagen content revealed by comparative study of MRI and pathology analysis. Int J Clin Exp Med 2015; 8 (08) 12898-12905
    PubMedGoogle Scholar
  • 36 Azimi P, Mohammadi HR, Benzel EC, Shahzadi S, Azhari S, Montazeri A. Artificial neural networks in neurosurgery. J Neurol Neurosurg Psychiatry 2015; 86 (03) 251-256
    CrossrefPubMedGoogle Scholar
  • 37 LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015; 521 (7553): 436-444
    CrossrefPubMedGoogle Scholar
  • 38 Fernandez-Delgado M, Cernadas E, Barro S. et al. Do we need hundreds of classifiers to solve real world classification problems?. J Mach Learn Res 2014; 15: 3133-3181
    PubMedGoogle Scholar
  • 39 Nawar S, Mouazen AM. Comparison between random forests, artificial neural networks and gradient boosted machines methods of on-line Vis-NIR spectroscopy measurements of soil total nitrogen and total carbon. Sensors (Basel) 2017; 17 (10) 2428 DOI: 10.3390/s17102428.
    CrossrefPubMedGoogle Scholar
  • 40 Senders JT, Staples P, Mehrtash A. et al. An online calculator for the prediction of survival in glioblastoma patients using classical statistics and machine learning. Neurosurgery 2020; 86 (02) E184-E192
    CrossrefPubMedGoogle Scholar
  • 41 Ahmad MW, Mourshed M, Rezgui Y. Trees vs Neurons: Comparison between random forest and ANN for high-resolution prediction of building energy consumption. Energy Build 2017; 147: 77-89
    CrossrefPubMedGoogle Scholar
  • 42 Reyes M, Meier R, Pereira S. et al. On the interpretability of artificial intelligence in radiology: challenges and opportunities. Radiol Artif Intell 2020; 2 (03) e190043
    CrossrefPubMedGoogle Scholar