Evaluating the Zero-Shot Segmentation Performance of Meta’s “Segment Anything Model” on Meningioma MRI Imaging

Background: Preoperative MR imaging is the standard for meningioma diagnosis; however, manual segmentation of meningioma from normal tissue for volumetric analysis or research is an arduous and time-consuming process and requires neuroradiology expertise. Several artificial intelligence (AI) solutions have been proposed to automate this process and enhance the clinical workflow. Key limitations of these studies include small cohorts derived from single institutions and unforeseen biases in training datasets leading to model inefficiencies. Additionally, these models are often computationally expensive to train and retrain. These challenges limit the external validity and practical implementation of developed tools.

The rise of generative AI has allowed for significant advances in segmentation modeling. Meta recently released the “Segment Anything Model” (SAM), touted as able to build accurate image segments rapidly without any domain-specific training.

Methods: Preoperative MR images were extracted from the BraTS database, a well-annotated, multi-institutional repository of meningiomas that was initially published as part of an online segmentation challenge in 2023. The SAM model was deployed on T1-contrasted MRI scans without prompting or pretraining. Produced model segments for each patient were compared to the corresponding enhancing or non-enhancing ground truth tumor segments at the MRI slice corresponding to the maximal tumor diameter ([Fig. 1]). Dice scores were used to quantify performance (0 being completely incorrect segmentation and 1 being a perfect segmentation). Each tumor was also classified via manual review into location within the cranium (skull base vs. non-skull base). Relevant metadata accompanying each MRI was also extracted. Exclusions were made for missing data across variables of interest.

A post-hoc analysis using standard statistical techniques was conducted to assess model performance across clinically relevant subgroups, defined by various patient characteristics (age, sex) as well as tumor characteristics (WHO grade, location, tumor volume, percentage of non-enhancing tumor).

Results: A total of 646 MRIs were included in the analysis. The overall cohort skewed female (68%) with most tumors being WHO grade 1 (75%) and located in the convexity (59%; [Table 1]). The average patient was 59 years old (SD: 15 years).

The average Dice score across all tumors was 0.84. There was no significant difference in performance across age subgroups (p = 0.0509), sex (p = 0.4453), or WHO grade (p = 0.1472). The model did, however, perform significantly worse on smaller tumor volumes (0.82 vs. 0.91, p < 0.0001), skull base lesions (0.82 vs. 0.86, p = 0.0262), and tumors that had a large percentage of non-enhancing signal (0.82 vs. 0.90, p = 0.0083).

Overall model performance was comparable to the best performing model in the BraTS 2023 challenge (0.87).

Conclusion: Without any training data or model optimization, SAM demonstrated comparable overall meningioma segmentation performance to other custom-developed, data-expensive AI models. Subgroup analysis demonstrated relative model weaknesses in segmentation of skull base tumors, relatively smaller lesions, and lesions with significant heterogeneity, indicating key areas for future refinement.

Publication History

Article published online:
07 February 2025