LOW COST REAL-TIME COLONOSCOPY PROCESSING WITH DEEP LEARNING

Aims Developing a low cost microsystem that can automatically identify in real-time different entities during a colonoscopy. It is based on a pre-trained deep learning neural network retrained on 700 annotated frames. The trained network works on a Jetson Xavier NX microsystem from NVIDIA.

Methods Starting with a database of colonoscopies, 300,000 frames were extracted. After several selections we ended up with 70 images for each of the 10 classes: sessile and pedunculated polyps, lipoma, diverticulum, bleeding, vascular tissue, water jet, tool head, forceps and snare. These frames were annotated by qualified colonoscopists. A pretrained Mobilenet neural network was retrained on our microsystem to detect entities pertaining to the selected classes. The resulted neural network was exported in the .onnx format.

Results On the same microsystem, several video files were processed in real-time and the results were very good for some classes (e.g. bleeding, tool heads) and at least satisfactory for others (e.g. small sessile polyps). One important result is that a Jetson Xavier NX microsystem has the computer power capacity to process real-time colonoscopies, i.e. to detect and mark important entities in frames, and also to save these results as a new video file.

Conclusions We developed a microsystem capable of processing in real-time video colonoscopy files. A database of 700 frames were used to retrain a deep learning neural network. Ten classes of entities were detected with good accuracy. A low cost Jetson Xavier NX microsystem from NVIDIA was used and it proved to have enough computer power for this complex task.

Publication History

Article published online:
14 April 2022