AUA 2019: Crowdsourcing: Renal Hilar Dissection

Chicago, IL ( This plenary session moderatred by Khurshid Ghani focused on the evaluation of 3 1-minute clips (unedited clips) of renal hilar dissection during robotic partial nephrectomy – while all were complex cases, no further data was provided. The set-up of the session was as follows:

  1. 1-minute unedited clip of an unknown robotic surgeon (who submitted this clip to the MUSIC collaborative) of renal hilar dissection
  2. The crowd voted on the score of 4 main domains of the GEARS evaluation focusing on the following characteristics: Depth Perception, Bimanual dexterity, Efficiency and Force Sensitivity
  3. Each was grade on a 1-5 likert scale (1 worst, 5 best)
  4. An expert surgeon would then review their own score of the video (and provide a brief discussion)
Two of the 3 expert surgeons panelists are experienced robotic surgeons (Benjamin Lee and James Porter), but Tim O’Brien is an experienced vascular surgeon (open surgery).

This was a highly interactive session, but I provide below some of the main points made by the speakers.

    1. Dr. Ghani highlighted that in the MUSIC collaborative, video review has been instituted for improvement purposes – videos are sent in and reviewed by peers in a blinded fashion; feedback from this can yield peer-to-peer coaching, changes to technique – and hopefully improved outcomes1
    2. Dr. Lee highlighted work on crowdsourcing in this same field – crowdsourcing is the blinded evaluation by a general population rather than an expert known panel. This can yield quicker feedback and, often, larger volume feedback. He, along with others in a multi-institutional prospective study, compared crowdsourcing vs. content expert feedback
    3. Crowdsourcing resulted in ~39 GEARS reviews per video – in 12 hours!
    4. 5 expert reviewers took 13 days to review those same videos
    5. The reviews were pretty concordant
    6. Dr. Lee next highlighted prior work (in which the authors looked at the EEG findings of trainees and staff at different levels completing the same robotic tasks – and found that: expert users are more focused on specific tasks (less jumping around between tasks) and have less stress with the same activities. They have an interest in testing whether this can change over time with increased training – or if it can be used to measure where an individual lies on the learning curve.
    7. Dr. Porter, the expert speaker for the 2nd video, highlighted some important points re: intra-operative management in robotic RETROPERITONEAL partial nephrectomy, as this was a RP partial nephrectomy video.
    8. Fat management – one key aspect is to excise and mobilize the paranephric (outside Gerota’s) fat off and displace it inferiorly. This allows clean access to Gerota’s fascia and perinephric fat.
    9. The 4th arm in a RP is critical to provide cephalad retraction and place the kidney and hilum on stretch. This enables two handed surgery. 
    10. Dr. Porter actually noted that video review has been incorporated into his institution’s robotic certification pathway. In surgeons who do less than 20 cases per year, they must submit 2 videos for review – and must score > 15/25 (5 questions, each 5 points – median score 3 on each) in order to maintain certification.
Dr. O’Brien provided important insight as an open surgeon looking at a robotic video – importantly, his review correlated well with the crowdsourced score. This is an interesting concept and has a lot of potential for: education, peer-review, certification, and mentoring.

Presented by: Khurshid Ghani, MD, University of Michigan, Benjamin Lee, Md, University of Arizona College of Medicine, James Porter, MD, University of Washington Medical Center, Tim O’Brien, MD, Guy's Hospital London

Written by: Thenappan Chandrasekar, MD (Clinical Instructor, Thomas Jefferson University) (twitter: @tchandra_uromd, @TjuUrology)  at American Urological Association's 2019 Annual Meeting (AUA 2019), May 3 – 6, 2019 in Chicago, Illinois 

1. Prebay ZJ, Peabody JO, Miller DC, et al.Video review for measuring and improving skill in urological surgery. Nat Rev Urol. 2019 Apr;16(4):261-267. doi: 10.1038/s41585-018-0138-2.