External validation of risk calculators (RCs) is necessary to determine their clinical applicability beyond the setting in which these were developed.
To assess the performance of the Rotterdam Prostate Cancer RC (RPCRC) and the Prostate Biopsy Collaborative Group RC (PBCG-RC).
We used data from the prospective, population-based STHLM3 screening study, performed in 2012-2015. Participants with prostate-specific antigen ≥3 ng/ml who underwent systematic prostate biopsies were included.
Probabilities for clinically significant prostate cancer (csPCa), defined as International Society of Urological Pathology grade ≥2, were calculated for each participant. External validity was assessed by calibration, discrimination, and clinical usefulness for both original and recalibrated models.
Out of 5841 men, 1054 (18%) had csPCa. Distribution of risk predictions differed between RCs; median risks for csPCa using the RPCRC and PBCG-RC were 3.3% (interquartile range [IQR] 2.1-7.1%) and 20% (IQR 15-28%), respectively. The correlation between RC risk estimates on individual level was moderate (Spearman's r = 0.55). Using the RPCRC's recommended risk threshold of ≥4% for finding csPCa, 36% of participants would get concordant biopsy recommendations. At 10% risk cut-off, RCs agreed in 23% of cases. Both RCs showed good discrimination, with areas under the curves for the RPCRC of 0.74 (95% confidence interval [CI] 0.72-0.76) and the PBCG-RC of 0.70 (95% CI 0.68-0.72). Calibration was adequate using the PBCG-RC (calibration slope: 1.13 [95% CI 1.03-1.23]), but the RPCRC underestimated the risk of csPCa (calibration slope: 0.73 [0.68-0.79]). The PBCG-RC showed a net benefit in a decision curve analysis, whereas the RPCRC showed no net benefit at clinically relevant risk threshold levels. Recalibration improved clinical benefit, and differences between RCs decreased.
Assessment of calibration is essential to ensure the clinical value of risk prediction tools. The PBCG-RC provided clinical benefit in its current version online. On the contrary, the RPCRC cannot be recommended in this setting.
Predicting the probability of finding prostate cancer on biopsy differed between two assessed risk calculators. After recalibration, the agreement of the models improved, and both were shown to be clinically useful.
European urology open science. 2022 May 19*** epublish ***
Jan Chandra Engel, Thorgerdur Palsdottir, Donna Ankerst, Sebastiaan Remmers, Ashkan Mortezavi, Venkatesh Chellappa, Lars Egevad, Henrik Grönberg, Martin Eklund, Tobias Nordström
Department of Clinical Sciences at Danderyd Hospital, Karolinska Institutet, Stockholm, Sweden., Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden., Department of Mathematics and Life Science Systems, Technical University of Munich, Munich, Germany., Department of Urology, Erasmus MC Cancer Institute, Erasmus University Medical Center, Rotterdam, The Netherlands., Department of Oncology-Pathology, Karolinska Institutet, Stockholm, Sweden.