Introduction Chemical composition analysis is important in prevention counseling for kidney stone disease. Advances in laser technology have made dusting techniques more prevalent, but this offers no consistent way to collect enough material to send for chemical analysis, leading many to forgo this test. We developed a novel machine learning (ML) model to effectively assess stone composition based on intraoperative endoscopic video data. Methods Two endourologists performed ureteroscopy for kidney stones ≥ 10mm. Representative videos were recorded intraoperatively. Individual frames were extracted from the videos and the stone was outlined by human tracing. An ML model, UroSAM, was built and trained to automatically identify kidney stones in the images and predict the majority stone composition: calcium oxalate monohydrate (COM), dihydrate (COD), calcium phosphate (CAP), or uric acid (UA). UroSAM was built on top of the publicly available Segment Anything Model (SAM) and incorporated a U-Net convolutional neural network (CNN). Discussion A total of 78 ureteroscopy videos were collected; 50 were used for the model after exclusions (32 COM, 8 COD, 8 CAP, 2 UA). The ML model segmented the images with 94.77% precision. Dice coefficient (0.9135) and Intersection over Union (0.8496) confirmed good segmentation performance of the ML model. A video-wise evaluation demonstrated 60% correct classification of stone composition. Subgroup analysis showed correct classification in 84.4% of COM videos. A post-hoc adaptive threshold technique was used to mitigate biasing of the model towards COM due to data imbalance - this improved the overall correct classification to 62% while improving the classification of COD, CAP, and UA videos. Conclusions This study demonstrates the successful development of UroSAM, an ML model that precisely identifies kidney stones from natural endoscopic video data. More high quality video data will improve the performance of the model in classifying the majority stone composition.
Journal of endourology. 2024 May 16 [Epub ahead of print]
Jixuan Leng, Junfei Liu, Galen Cheng, Haohan Wang, Scott Orzech Quarrier, Jiebo Luo, Rajat Jain
University of Rochester, Computer Science, Rochester, New York, United States; ., University of Rochester, Computer Science, Rochester, New York, United States; ., University of Rochester Medical Center, Urology, Rochester, New York, United States; ., University of Illinois at Urbana-Champaign, Computer Science, Urbana, Illinois, United States; ., University of Rochester, Urology, 601 Elmwood Ave, Rochester, New York, United States, 14642., University of Rochester, Computer Science, Rochester, New York, United States; ., University of Rochester Medical Center, Urology, Rochester, New York, United States; .