An intelligent system for real-time automated anatomical recognition system during endoscopic ultrasound (EUS)

Aims Learning EUS is a very time-consuming process requiring an experienced teacher because of the EUS images are often very abstract. We aim to build an intelligent software for real-time recognition of anatomical stations during EUS. With the help of this newly developed software, we aim to assist the role of the supervisor in teaching beginners EUS, reduce the workload and numbers required from supervisors, dampen the steep learning curve of EUS, promote the use of EUS even in countries with low caseload.

Methods YOLOv8, the state-of-the-art, anchor-free deep learning model. We implemented the YOLOv8-seg model with Pytorch 1.10.1 using the NVIDIA RTX 3060Ti Graphics Processing Unit. The current dataset contains 860 images, and we randomly divided the entire dataset into two parts. The first part contains 706 images for training, and the remaining 154 images are used as the validation set. The method we use here is image segmentation, which can be formulated as the problem of classifying pixels with semantic labels. During the labeling process, pixel-level labeling with a set of object categories is needed. During training, the hyperparameters were set as epoch 100, batch size 16, initial learning rate 0.01, momentum 0.937, and weight attenuation coefficient 0.0005. Images were resized to 736 x 736 pixels before feeding them into the model. Mosaic data augmentation method was used to enrich the features and improve robustness.

Results The model is based on different object class assignment of the dataset. The model performs segmentation on 1: Kidney, 2: Pancreas, 3: Spleen, 4: Aorta, 5: Coeliac artery, 6: Splenic artery, 7: Splenic vein. The F1 confidence curve of the annotated structures were analysed. The F1 score was only 0.80 for all structures. The F1-score of each individual data are as follows: 1: Kidney (0.97), 2: Pancreas (0.98), 3: Spleen (0.99), 4: Aorta (0.95), 5: Coeliac artery (0.64), 6: Splenic artery (0.54), 7: Splenic vein (0.69). In the subgroup analysis, it showed that for solid organs, the accuracy of the model is much higher than that of the vessels.

Conclusions This AI software has achieved a F1 score of 0.80 for the accuracy of image recognition. The accuracy for solid organs is up to 95%. The next step is to apply it clinically in real-time to see its real clinical effectiveness.

Publikationsverlauf

Artikel online veröffentlicht:
27. März 2025