2. UroToday Home
  3. Conference Highlights
  4. CUA 2018
  5. CUA 2018 Leadership, Communication and Practice Management

CUA 2018

CUA 2018: Validation of Real-time, Intraoperative, Surgical Competence Assessments Linked to Clinically Relevant Patient Outcomes: A Model of Competency Assessment in Urology

Halifax, Nova Scotia (UroToday.com) Despite significant progress in the objective evaluation of technical skill in the simulation skills lab, intraoperative assessment of surgical proficiency remains largely subjective and invalidated. For the graduating urology resident, there are objective assessments of knowledge, provided by licensing exams from the American Board of Urology or the Royal College of Physicians & Surgeons of Canada, with subjective assessment of technical skill based on opinions of the surgical faculty. For the experienced surgeon, knowledge may be based on continuing medical education credits, with limited, if any, assessment of technical skill. The linkage between objective evaluations of intraoperative surgical competence and real patient outcomes have rarely (if ever) been systematically reported. At the CUA 2018 podium session, Michael Elfassy, a second-year medical student at the University of Toronto, presented their groups work determining if intraoperative evaluations of technical skill using Real-time, Intraoperative, Surgical Competence (RISC) assessments predicts clinical and operative outcomes in real patients.

For this study, the authors used TURBT as the procedure of choice for RISC evaluation. TURBT was selected as it is:

  • A commonly performed urological procedure
  • Easily recorded
  • Performed by all levels of trainees and staff
  • Is associated with regular scheduled follow-up and surveillance
  • Associated with a high event rate (recurrence)
Bladder tumor recurrence is known to be associated with tumor stage, grade, size, multifocality, CIS, patient factors, and intravesical immunotherapy, but whether it is associated with surgical skill (ie. quality of surgery) is unknown. Surgeons were surveyed as to their level of training and experience with TURBT (n=124 junior residents; n=29 senior residents; n=11 fellows; n=23 faculty). Live TURBTs (n=187) were prospectively recorded and evaluated in a blinded fashion by 2-4 urologic surgeons as to the overall technical quality of the TURBT using RISC. The RISC assessment was developed following a blinded review by the expert surgeons of unedited surgical videos of TURBT cases both with and without bladder tumor recurrence following surgery, with the goal to identify fundamental technical skill domains influencing a tumor recurrence/recurrence-free state. Twenty competency domains were identified and used to create the RISC assessment. The technical skill domains comprising the RISC assessment were structured as a composite of previously validated global rating scales and final product scores. Patients were followed for 18 months for evidence of tumor recurrence. RISC scores were correlated with case-matched clinical and operative patient outcomes.  

For internal validation, bladder tumor recurrence, as expected, correlated with tumor stage (r=0.24, p=0.03) and tumor grade (r=0.22, p=0.04). RISC scores discriminated between experienced and novice surgeons and correlated significantly with the number of previous surgical cases performed (r=0.2; p=0.04). Also, RISC assessments of surgical skill during TURBT correlated significantly with rates of cystoscopic bladder tumor recurrence. Both global ratings of surgical performance and final surgical product ratings were significantly higher (suggestive of superior technical skill) in cases without evidence of bladder tumor recurrence.
UroToday CUA 2018 Validation of Real time Intraoperative Surgical Competence RISC
Elfassy concluded with several take-home messages:

  • RISC assessments of surgical skills demonstrated both construct and predictive validity for bladder tumor recurrence following TURBT, suggesting that technical skill can be objectively evaluated in the operating room, on real patients, and that it does influence patient outcomes.
  • A similar methodology can be applied to develop RISC assessments for a variety of surgical procedures and disease states.
Presented by: Michael Elfassy, the University of Toronto of Toronto School of Medicine, Toronto, ON, Canada
Co-Authors: Ethan D. Grober, Mitchell Goldenberg, Mohammed Mahdi, Armando Lorenzo, Matthew Roberts, Trustin Domes, Michael A.S. Jewett. Department of Surgery, Division of Urology, Women's College & Mount Sinai Hospital, University of Toronto, Toronto, ON, Canada

Written By: Zachary Klaassen, MD, Urologic Oncology Fellow, University of Toronto, Princess Margaret Cancer Centre Twitter: @zklaassen_md at the 73rd Canadian Urological Association Annual Meeting - June 23 - 26, 2018 - Halifax, Nova Scotia