In this study, the staff and residents from the Dalhousie Urology program completed 5 TURBT simulated procedures using the device. Data was collected by pre- and post-questionnaires, built in metrics, and the objective structured assessment of technical skills (OSATS) which was conducted by two experts, independent Urologists. Validity of the technology was determined on a 5-point Likert scale with ≤ 2.9 meaning unacceptable, 3.0-3.9 as somewhat acceptable, and 4.0 or above as acceptable. The validity determination was carried out by expert’s opinions (expert refers to having performed greater than 50 TURBTs).
At the completion of the study, 9 experts and 9 novices took part in the training simulation. There were no significant differences between the mean operative scores between expert and novice in regard to resection, bleeding control, safety, economy, and visualization. There was, however, a significant increase from total score comparing the first attempt and the fifth attempt for both the novices and experts. In regard to the postoperative questionnaire, 56% of novices “agreed” that the simulator improved their confidence in the procedure. When all 18 individuals were asked whether the simulator should be in the official curriculum, 50% “strongly agreed”, but interestingly, 5% “disagreed.” The best qualities of the simulator, as determined by the validity test, were the graphical appearance, camera, and 3D movement. The worst qualities were the bladder distention, resection depth, and fluids. The complete list of Likert Scale validity qualities are shown below.
As Dr. Moore concluded his speech, though the face and content validity was somewhat acceptable, he determined that the construct validity was unacceptable due to the inability to differentiate skill levels and the need for an improved in-simulator scoring system. However, due to the feedback and improvement in score by the trainees and staff, there may be a potential role as an introduction to TURBT for novice urology trainees.
Presented by: Jonathan Moore, MD
Written by Zachary Valley, Department of Urology, University of California-Irvine, at the 73rd Canadian Urological Association Annual Meeting - June 23 - 26, 2018 - Halifax, Nova Scotia