(UroToday.com) Dr. Tzevat Tefik from Istanbul University presented an innovative study exploring the use of machine learning models for real-time kidney stone detection during robotic flexible ureterorenoscopy (fURS). While robotic fURS offers enhanced maneuverability for intrarenal procedures, real-time identification and segmentation of stones remain challenging under endoscopic visualization. Dr. Tefik and his team aimed to develop and evaluate a machine learning-based approach to improve stone detection accuracy and procedural efficiency.
The researchers utilized a dataset derived from 9 robotic fURS videos capturing intrarenal stone treatment. A total of 2,520 frames were extracted and manually annotated for the presence of kidney stones. Curated through Roboflow and exported in COCO segmentation JSON format, the dataset was stratified into training (70%, n=1,760), validation (20%, n=510), and testing (10%, n=250) cohorts (Figure 1).
Figure 1
To train the detection system, the team leveraged a Mask R-CNN framework built within the Detectron2 platform, known for its strong capabilities in medical image analysis and object segmentation. The model was initialized using pre-trained weights from the COCO dataset and then specifically fine-tuned on their manually annotated fURS video frames. Training involved adjusting feature strides between 256 and 1024 pixels, with performance improvements monitored in real-time through TensorBoard during the 299 completed iterations. For rapid image processing, inference was performed using a GPU-accelerated OpenCV pipeline.
Figure 2
In terms of performance, the model achieved an impressive area under the receiver operating characteristic curve (AUC-ROC) of 92.22%. Each frame was processed in less than 450 milliseconds, allowing for real-time application. The Mask R-CNN-based model revealed excellent object classification and segmentation abilities (Figure 2). These findings underscore the potential for AI-assisted tools to enhance visualization and efficiency in minimally invasive urologic surgery.
Following the presentation, moderator Dr. Roger Sur of the University of California, San Diego, inquired about the next steps for the machine learning initiative. Dr. Tefik informed the audience that the team plans to further evaluate the algorithm’s performance under more challenging conditions, including during stone dusting and in settings of blurry endoscopic vision. Ultimately, the team’s goal is to assist in the automation of robotic-assisted surgery and further enhance intraoperative efficiency.
Dr. Tefik and colleagues concluded their presentation by highlighting the promising role of artificial intelligence in robotic fURS procedures:
- A custom-trained Mask R-CNN model exhibited high accuracy for real-time stone detection and segmentation.
- The model's real-time processing capability (<450 ms per frame) supports its potential integration into intraoperative workflows.
- Implementation of AI-driven annotation tools may enhance precision, procedural efficiency, and surgical outcomes during minimally invasive urologic interventions.
- Future directions may include expanding the dataset to include a broader range of stone morphologies and lighting conditions, further optimizing model architecture, and conducting prospective validation studies during live surgeries.
Presented by: Tzevat Tefik, MD, FEBU, Istanbul University, Istanbul Faculty of Medicine
Written by: Mariah Hernandez, Research Specialist, Department of Urology, University of California, Irvine, @mariahch00 on Twitter during the American Urological Association's 2025 Annual Meeting, between April 26 – 29, 2025 in Las Vegas, NV.