Objective Assessment and Standard Setting for Basic Flexible Ureterorenoscopy Skills among Urology Trainees using Simulation-based Methods.

To objectively assess the performance of graduating urology residents performing flexible ureterorenoscopy (fURS) using a simulation-based model and to set an entrustibility standard or benchmark for use across the educational spectrum.

Chief urology residents and attending endourologists performed a standardized fURS task (ureterorenoscopy & repositioning of stones) using a Boston Scientific© Lithovue ureteroscope on a Cook Medical© URS model. All performances were video-recorded and blindly scored by both endourology experts and crowd-workers (C-SATS) using the Ureteroscopic Global Rating Scale, plus an overall entrustability score. Validity evidence supporting the scores was collected and categorized. The Borderline Group method was used to set absolute performance standards for the expert and crowdsourced ratings.

A total of 44 participants (40 chief residents, 4 faculty) completed testing. Eighty-three percent of participants had performed over 50 fURS cases at the time of the study. Only 47.7% (mean score 12.6/20) and 61.4% (mean score 12.4/20) of participants were deemed 'entrustable' by experts and crowd-workers, respectively. The Borderline Group method produced entrustability benchmarks of 11.8/20 for experts and 11.4/20 for crowd-worker ratings, resulting in pass rates of 56.9% and 61.4%.

Using absolute standard setting methods, benchmark scores were set to identify trainees that could safely carry out flexible ureterorenoscopy in the simulated setting. Only 60% of residents in our cohort were rated as entrustable. These findings support the use of benchmarks to earlier identify trainees requiring remediation.

Journal of endourology. 2020 Feb 14 [Epub ahead of print]

Mitchell G Goldenberg, Michael Ordon, John D'A Honey, Sero Andonian, Jason Y Lee

St. Michael's Hospital, Urology, 9th Floor, St. Michael's Health Centre, 61 Queen Street East, Toronto, Ontario, Canada, M5C 2T2; ., University of Toronto, 7938, Surgery, 61 Queen Street East, Suite 9-103, Toronto, Ontario, Canada, M5C 2T2; ., St Michael's Hospital, Division of Urology, Toronto, Ontario, Canada; ., McGill University Health Centre, Quebec, Urology, Quebec, Quebec, Canada; ., University of Toronto, Urology, 585 University Ave, TGH 11 PMB Rm186A, Toronto, Ontario, Canada, M2G 2N2.

Go Beyond the Abstract and Read a Commentary by the Authors