Development of a multi-modal AI model integrating EUS imaging and clinical and endoscpic data for enhanced diagnosis of gastric mesenchymal tumors

Aims Artificial intelligence (AI)-assisted diagnostic tools using endoscopic ultrasonography (EUS) have demonstrated outstanding performance in identifying gastric mesenchymal tumors. However, the diagnostic performance of these tools decreases when assessing smaller lesions. Recent studies have suggested that incorporating clinical and endoscopic characteristics alongside EUS images may enhance diagnostic performance. We aim to evaluate the performance of an AI diagnostic model that integrates EUS images with clinical and endoscopic characteristics.

Methods The data from 751 patients diagnosed histopathologically with gastric mesenchymal tumors (GIST, leiomyoma, and schwannoma) at three medical centers in Korea between March 2009 and August 2022 were retrospectively analyzed. Of these, 576 were diagnosed with GIST, 96 with leiomyoma, and 79 with schwannoma. Our model integrates EfficientNet-based image feature extraction with XGBoost predictions on clinical and endoscopic data, creating a Keras-based multimodal deep learning model.

Results The model demonstrated excellent diagnostic performance. Specifically, it achieved a sensitivity of 0.973 for GIST, 0.827 for leiomyoma, and 0.725 for schwannoma, reflecting its ability to correctly identify true positive cases in each category. In terms of specificity, the model reached values of 0.952 for GIST, 0.887 for leiomyoma, and 0.795 for schwannoma, highlighting its accuracy in correctly excluding non-target cases. Additionally, the area under the curve (AUC) values were 0.982 for GIST, 0.985 for leiomyoma, and 0.978 for schwannoma, indicating high overall diagnostic accuracy across these gastric mesenchymal tumors. These findings suggest that the model, which integrates clinical and endoscopic characteristics with EUS image data, demonstrates excellent diagnostic performance in differentiating various types of gastric mesenchymal tumors.

Conclusions Our AI model integrating EUS image and clinical and endoscopic data offers a promising tool for diagnosis of gastric mesenchymal tumor. This multi-modal approach holds potential for broader application in clinical settings.