Download PDF
J Neurol Surg B Skull Base 2022; 83(06): 635-645
DOI: 10.1055/a-1885-1447
Review Article

Machine Learning Models for Predicting Postoperative Outcomes following Skull Base Meningioma Surgery

Adrian E. Jimenez
1   Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
,
Jose L. Porras
1   Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
,
Tej D. Azad
1   Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
,
Pavan P. Shah
1   Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
,
Christopher M. Jackson
1   Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
,
Gary Gallia
1   Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
,
Chetan Bettegowda
1   Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
,
Jon Weingart
1   Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
,
Debraj Mukherjee 
1   Department of Neurosurgery, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
› Author Affiliations Funding The authors received no financial support for the research, authorship, and/or publication of this article. The authors acknowledge assistance for clinical data coordination and retrieval from the Core for Clinical Research Data Acquisition, supported in part by the Johns Hopkins Institute for Clinical and Translational Research (UL1TR001079).
› Further Information
    Permissions and Reprints

Abstract

Objective While predictive analytic techniques have been used to analyze meningioma postoperative outcomes, to our knowledge, there have been no studies that have investigated the utility of machine learning (ML) models in prognosticating outcomes among skull base meningioma patients. The present study aimed to develop models for predicting postoperative outcomes among skull base meningioma patients, specifically prolonged hospital length of stay (LOS), nonroutine discharge disposition, and high hospital charges. We also validated the predictive performance of our models on out-of-sample testing data.

Methods Patients who underwent skull base meningioma surgery between 2016 and 2019 at an academic institution were included in our study. Prolonged hospital LOS and high hospital charges were defined as >4 days and >$47,887, respectively. Elastic net logistic regression algorithms were trained to predict postoperative outcomes using 70% of available data, and their predictive performance was evaluated on the remaining 30%.

Results A total of 265 patients were included in our final analysis. Our cohort was majority female (77.7%) and Caucasian (63.4%). Elastic net logistic regression algorithms predicting prolonged LOS, nonroutine discharge, and high hospital charges achieved areas under the receiver operating characteristic curve of 0.798, 0.752, and 0.592, respectively. Further, all models were adequately calibrated as determined by the Spiegelhalter Z-test (p >0.05).

Conclusion Our study developed models predicting prolonged hospital LOS, nonroutine discharge disposition, and high hospital charges among skull base meningioma patients. Our models highlight the utility of ML as a tool to aid skull base surgeons in providing high-value health care and optimizing clinical workflows.

Keywords

meningioma - neurovascular structures - central nervous system - overall survival - outcomes

Reporting Guidelines

The authors found no applicable reporting guidelines that would apply to this article. By following the EQUATOR reporting guidelines decision tree, (http://www.equatornetwork.org/wp-content/uploads/2013/11/20160226-RG-decision-tree-for-Wizard-CC-BY-26- February-2016.pdf), we found that none of the most popular checklists are appropriate for our study design.




Publication History

Received: 31 December 2021

Accepted: 20 June 2022

Accepted Manuscript online:
25 June 2022

Article published online:
25 August 2022

© 2022. Thieme. All rights reserved.

Georg Thieme Verlag KG
Rüdigerstraße 14, 70469 Stuttgart, Germany

 

  • References

  • 1 Ostrom QT, Gittleman H, Fulop J. et al. CBTRUS statistical Report: primary brain and central nervous system tumors diagnosed in the United States in 2008–2012. Neuro-oncol 2015; 17 (suppl 4): iv1-iv62
    CrossrefPubMedGoogle Scholar
  • 2 Wang N, Osswald M. Meningiomas: overview and new directions in therapy. Semin Neurol 2018; 38 (01) 112-120
    Article in Thieme ConnectPubMedGoogle Scholar
  • 3 Meling TR, Da Broi M, Scheie D, Helseth E. Meningiomas: skull base versus non-skull base. Neurosurg Rev 2019; 42 (01) 163-173
    CrossrefPubMedGoogle Scholar
  • 4 Voß KM, Spille DC, Sauerland C. et al. The Simpson grading in meningioma surgery: does the tumor location influence the prognostic value?. J Neurooncol 2017; 133 (03) 641-651
    CrossrefPubMedGoogle Scholar
  • 5 Chen CM, Huang APH, Kuo LT, Tu YK. Contemporary surgical outcome for skull base meningiomas. Neurosurg Rev 2011; 34 (03) 281-296 , discussion 296
    CrossrefPubMedGoogle Scholar
  • 6 DeMonte F, McDermott M, Al-Mefty O. . Al-Mefty's Meningiomas. 2nd ed. Thieme; 2011
  • 7 Saklad M. Grading of patients for surgical procedures. Anesthesiology 1941; 2 (03) 281-284
    CrossrefPubMedGoogle Scholar
  • 8 Subramaniam S, Aalberg JJ, Soriano RP, Divino CM. New 5-factor modified frailty index using American College of Surgeons NSQIP Data. J Am Coll Surg 2018; 226 (02) 173-181 .e8
    CrossrefPubMedGoogle Scholar
  • 9 Cahill PJ, Pahys JM, Asghar J. et al. The effect of surgeon experience on outcomes of surgery for adolescent idiopathic scoliosis. J Bone Joint Surg Am 2014; 96 (16) 1333-1339
    CrossrefPubMedGoogle Scholar
  • 10 Lau D, Deviren V, Ames CP. The impact of surgeon experience on perioperative complications and operative measures following thoracolumbar 3-column osteotomy for adult spinal deformity: overcoming the learning curve. J Neurosurg Spine 2019; 32 (02) 207-220
    CrossrefPubMedGoogle Scholar
  • 11 Dasenbrock HH, Liu KX, Devine CA. et al. Length of hospital stay after craniotomy for tumor: a National Surgical Quality Improvement Program analysis. Neurosurg Focus 2015; 39 (06) E12
    CrossrefPubMedGoogle Scholar
  • 12 Lakomkin N, Hadjipanayis CG. Resident participation is not associated with postoperative adverse events, reoperation, or prolonged length of stay following craniotomy for brain tumor resection. J Neurooncol 2017; 135 (03) 613-619
    CrossrefPubMedGoogle Scholar
  • 13 Muhlestein WE, Akagi DS, Chotai S, Chambless LB. The impact of presurgical comorbidities on discharge disposition and length of hospitalization following craniotomy for brain tumor. Surg Neurol Int 2017; 8: 220
    CrossrefPubMedGoogle Scholar
  • 14 Kalakoti P, Missios S, Menger R, Kukreja S, Konar S, Nanda A. Association of risk factors with unfavorable outcomes after resection of adult benign intradural spine tumors and the effect of hospital volume on outcomes: an analysis of 18, 297 patients across 774 US hospitals using the National Inpatient Sample (2002-2011). Neurosurg Focus 2015; 39 (02) E4
    CrossrefPubMedGoogle Scholar
  • 15 Muhlestein WE, Akagi DS, Chotai S, Chambless LB. The impact of race on discharge disposition and length of hospitalization after craniotomy for brain tumor. World Neurosurg 2017; 104: 24-38
    CrossrefPubMedGoogle Scholar
  • 16 Khalafallah AM, Jimenez AE, Patel P, Huq S, Azmeh O, Mukherjee D. A novel online calculator predicting short-term postoperative outcomes in patients with metastatic brain tumors. J Neurooncol 2020; 149 (03) 429-436
    CrossrefPubMedGoogle Scholar
  • 17 Brandel MG, Rennert RC, Wali AR. et al. Impact of preoperative endovascular embolization on immediate meningioma resection outcomes. Neurosurg Focus 2018; 44 (04) E6
    CrossrefPubMedGoogle Scholar
  • 18 Kuhn M. Building predictive models in R using the caret package. J Stat Softw 2008; 28 (05) 1-26
    CrossrefPubMedGoogle Scholar
  • 19 Zou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc Series B Stat Methodol 2005; 67 (05) 768
    CrossrefPubMedGoogle Scholar
  • 20 Bergstra J, Bengio Y. Random search for hyper-parameter optimization. J Mach Learn Res 2012; 13: 281-305
    PubMedGoogle Scholar
  • 21 Swets JA. Measuring the accuracy of diagnostic systems. Science 1988; 240 (4857): 1285-1293
    CrossrefPubMedGoogle Scholar
  • 22 Rufibach K. Use of Brier score to assess binary predictions. J Clin Epidemiol 2010; 63 (08) 938-939 , author reply 939
    CrossrefPubMedGoogle Scholar
  • 23 Clopper CJ, Pearson ES. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 1934; 26 (04) 404-413
    CrossrefPubMedGoogle Scholar
  • 24 Muhlestein WE, Akagi DS, McManus AR, Chambless LB. Machine learning ensemble models predict total charges and drivers of cost for transsphenoidal surgery for pituitary tumor. J Neurosurg 2018; 131 (02) 507-516
    CrossrefPubMedGoogle Scholar
  • 25 Navarro SM, Wang EY, Haeberle HS. et al. Machine learning and primary total knee arthroplasty: patient forecasting for a patient-specific payment model. J Arthroplasty 2018; 33 (12) 3617-3623
    CrossrefPubMedGoogle Scholar
  • 26 Jimenez AE, Khalafallah AM, Lam S. et al. Predicting high-value care outcomes after surgery for skull base meningiomas. World Neurosurg 2021; 149: e427-e436
    CrossrefPubMedGoogle Scholar
  • 27 Little AS, Chapple K. Predictors of resource utilization in transsphenoidal surgery for Cushing disease. J Neurosurg 2013; 119 (02) 504-511
    CrossrefPubMedGoogle Scholar
  • 28 Zacharia BE, Deibert C, Gupta G. et al. Incidence, cost, and mortality associated with hospital-acquired conditions after resection of cranial neoplasms. Neurosurgery 2014; 74 (06) 638-647
    CrossrefPubMedGoogle Scholar
  • 29 McKee SP, Yang A, Gray M. et al. Intracranial meningioma surgery: value-based care determinants in New York State, 1995-2015. World Neurosurg 2018; 118: e731-e744
    CrossrefPubMedGoogle Scholar
  • 30 Chapman EK, Doctor T, Gal JS. et al. The impact of non-elective admission on cost of care and length of stay in anterior cervical discectomy and fusion: a propensity-matched analysis. Spine 2021; 46 (22) 1535-1541
    CrossrefPubMedGoogle Scholar
  • 31 Ahn J, Iqbal A, Manning BT. et al. Minimally invasive lumbar decompression-the surgical learning curve. Spine J 2016; 16 (08) 909-916
    CrossrefPubMedGoogle Scholar
  • 32 Huq S, Khalafallah AM, Patel P. et al. Predictive model and online calculator for discharge disposition in brain tumor patients. World Neurosurg 2021; 146: e786-e798
    CrossrefPubMedGoogle Scholar
  • 33 Lakomkin N, Hadjipanayis CG. Non-routine discharge disposition is associated with post-discharge complications and 30-day readmissions following craniotomy for brain tumor resection. J Neurooncol 2018; 136 (03) 595-604
    CrossrefPubMedGoogle Scholar
  • 34 Sastry RA, Pertsch NJ, Tang O, Shao B, Toms SA, Weil RJ. Frailty and outcomes after craniotomy for brain tumor. J Clin Neurosci 2020; 81: 95-100
    CrossrefPubMedGoogle Scholar
  • 35 Muhlestein WE, Akagi DS, Kallos JA. et al. Using a guided machine learning ensemble model to predict discharge disposition following meningioma resection. J Neurol Surg B Skull Base 2018; 79 (02) 123-130
    Article in Thieme ConnectPubMedGoogle Scholar
  • 36 Kidwai SM, Yang A, Gray ML. et al. Hospital charge variability across New York State: sociodemographic factors in pituitary surgery. J Neurol Surg B Skull Base 2019; 80 (06) 612-619
    Article in Thieme ConnectPubMedGoogle Scholar
  • 37 Hamill CS, Villwock JA, Sykes KJ, Chamoun RB, Beahm DD. Socioeconomic factors affecting discharge status of patients with uncomplicated transsphenoidal adenohypophysectomy. J Neurol Surg B Skull Base 2018; 79 (05) 501-507
    Article in Thieme ConnectPubMedGoogle Scholar
  • 38 Abou-Al-Shaar H, Azab MA, Karsy M, Guan J, Couldwell WT, Jensen RL. Assessment of costs in open microsurgery and stereotactic radiosurgery for intracranial meningiomas. World Neurosurg 2018; 119: e357-e365
    CrossrefPubMedGoogle Scholar
  • 39 Alvin MD, Lubelski D, Alam R. et al. Spine surgeon treatment variability: the impact on costs. Global Spine J 2018; 8 (05) 498-506
    CrossrefPubMedGoogle Scholar
  • 40 Doumouras AG, Saleh F, Gmora S, Anvari M, Hong D. The value of surgical experience: excess costs associated with the Roux-en-Y gastric bypass learning curve. Surg Endosc 2019; 33 (06) 1944-1951
    CrossrefPubMedGoogle Scholar
  • 41 Breiman L. Statistical modeling: the two cultures. Stat Sci 2001; 16 (03) 199-215
    CrossrefPubMedGoogle Scholar
  • 42 Hannun AY, Rajpurkar P, Haghpanahi M. et al. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat Med 2019; 25 (01) 65-69
    CrossrefPubMedGoogle Scholar
  • 43 Senior AW, Evans R, Jumper J. et al. Improved protein structure prediction using potentials from deep learning. Nature 2020; 577 (7792): 706-710
    CrossrefPubMedGoogle Scholar
  • 44 Lockhart R, Taylor J, Tibshirani RJ, Tibshirani R. A significance test for the lasso. Ann Stat 2014; 42 (02) 413-468
    PubMedGoogle Scholar
  • 45 Lee JD, Sun DL, Sun Y, Taylor JE. Exact post-selection inference, with application to the lasso. Ann Stat 2016; 44 (03) 907-927
    PubMedGoogle Scholar
  • 46 Christodoulou E, Ma J, Collins GS, Steyerberg EW, Verbakel JY, Van Calster B. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol 2019; 110: 12-22
    CrossrefPubMedGoogle Scholar
  • 47 Marcus AP, Marcus HJ, Camp SJ, Nandi D, Kitchen N, Thorne L. Improved prediction of surgical resectability in patients with glioblastoma using an artificial neural network. Sci Rep 2020; 10 (01) 5143
    CrossrefPubMedGoogle Scholar
  • 48 Bae S, Massie AB, Caffo BS, Jackson KR, Segev DL. Machine learning to predict transplant outcomes: helpful or hype? A national cohort study. Transpl Int 2020; 33 (11) 1472-1480
    CrossrefPubMedGoogle Scholar
  • 49 Oosterhoff JHF, Gravesteijn BY, Karhade AV. et al; Machine Learning Consortium. Feasibility of machine learning and logistic regression algorithms to predict outcome in orthopaedic trauma surgery. J Bone Joint Surg Am 2022; 104 (06) 544-551
    CrossrefPubMedGoogle Scholar