Development and Validation of a Multimodal Deep Learning Model for Early Esophageal Squamous Neoplasia Detection and Invasion Depth Prediction

Introduction: Early detection of esophageal squamous cell carcinoma (ESCC) is critical for optimizing patient outcomes. Magnifying endoscopy (ME) and endoscopic ultrasonography (EUS) serve as established diagnostic modalities. MUMA-EDx (Multimodal Ultrasound & Magnifying-endoscopic Algorithm for Early ESCC Diagnostics) integrates deep learning-based ME and EUS imaging to improve early-stage ESCC identification and invasion depth assessment. Methods: Model development and internal validation utilized the retrospective dataset, while the prospective cohort served for external validation. MUMA-EDx developed two TResNet_m-based classifiers (ME/EUS) followed by feature-level fusion. Model performance was evaluated using area under the receiver operating characteristic curve (AUC-ROC), accuracy, sensitivity, specificity, positive predictive value, and negative predictive value. Results: MUMA-EDx was developed and validated using a retrospective dataset comprising 460 patients (20,889 images) and subsequently tested prospectively on an independent cohort of 131 patients (9,124 images). The feature-level multimodal approach significantly outperformed single-modality models. For tumor discrimination, the model achieved an AUC of 0.94 (95% CI: 0.92-0.96) in retrospective validation and a perfect patient-level AUC of 1.00 (95% CI: 1.00-1.00) in prospective testing. For the more complex task of multiclass invasion depth classification, it achieved a retrospective AUC of 0.95 (95% CI: 0.88-0.99), which remained strong at 0.80 (95% CI: 0.67-0.87) in the prospective cohort. In a comparative study on invasion depth classification, MUMA-EDx's performance exceeded that of novice endoscopists and was comparable to expert-level diagnostics. Conclusion: MUMA-EDx demonstrably delivers exceptional early ESCC detection and robust invasion depth classification, achieving performance comparable to expert endoscopists and poised to significantly enhance diagnostic precision and patient outcomes.