A Combination of YOLO and OSNet Re-ID Neuronal Networks for Tracking Abnormalities in Upper Gastrointestinal Endoscopy Videos

Endoscopic image processing is vital in medicine, but manual analysis is time-consuming and labor-intensive. Tracking abnormalities in endoscopic videos is a key focus of artificial intelligence (AI), yet it remains a complex task that depends heavily on medical experts. Utilizing AI can provide benefits like real-time detection during procedures and improved diagnostic support afterward, along with aiding medical research and education. However, challenges such as tissue deformation, unstable lighting, and camera movement complicate the process, requiring sophisticated image processing and machine learning techniques for accurate tracking. Given the challenges, this research explores the use of recent advances in neural networks including the YOLO (You Only Look Once), StrongSORT tracking algorithm, and OSNet (Omni-Scale Feature Learning Network) Re-identification models for tracking abnormalities in upper gastrointestinal (GI) endoscopy videos. The results show that combining these models improves detection efficiency and accuracy, enabling real-time operation. Moreover, the ratio of accurately identified detections to the average number of validated detections (IDF1 70.5%), along with Precision (90.9%) and Recall (56.1%), are relatively satisfactory results, indicating that the integration of these technologies has great potential in improving the process of diagnosis and monitoring of pathological conditions through endoscopic videos. Notably, the reduction of ID switches to an insignificant amount further highlights the robustness of the system, ensuring continuous and accurate tracking over time. This significant improvement offers substantial potential for advancing endoscopic diagnosis and healthcare quality.

In International Symposium on Information and Communication Technology