MARCAP Meta-Analysis of Surrogate Endpoints for Overall Survival, Journal Club - Rashid Sayyid & Zachary Klaassen
May 31, 2023
Rashid Sayyid and Zach Klaassen discuss a publication by the MARCAP Group titled “Meta-Analysis of Candidate Surrogate End Points in Advanced Prostate Cancer.” The study aims to validate intermediate clinical endpoints in advanced prostate cancer, which are faster to report, reducing resources required for clinical trials and potentially allowing quicker patient access to effective therapies. The analysis includes a systematic review of randomized therapeutic trials in advanced prostate cancer, exploring endpoints such as time to biochemical failure and progression-free survival. However, the study finds that none of these intermediate clinical endpoints meet the surrogacy criteria. Results are consistent across different patient groups and treatment types. Despite the potential benefits of surrogate endpoints in clinical trials, no valid options currently exist for advanced prostate cancer. This analysis underscores the need for further investigation into time-specific or alternative composite endpoints to serve as valid surrogate endpoints in advanced prostate cancer clinical trials.
Biographies:
Rashid Sayyid, MD, MSc, Urologic Oncology Fellow, Division of Urology, University of Toronto, Toronto, Ontario
Zachary Klaassen, MD, MSc, Urologic Oncologist, Assistant Professor Surgery/Urology at the Medical College of Georgia at Augusta University, Georgia Cancer Center
Biographies:
Rashid Sayyid, MD, MSc, Urologic Oncology Fellow, Division of Urology, University of Toronto, Toronto, Ontario
Zachary Klaassen, MD, MSc, Urologic Oncologist, Assistant Professor Surgery/Urology at the Medical College of Georgia at Augusta University, Georgia Cancer Center
Read the Full Video Transcript
Rashid Sayyid: Hello everyone. This is Rashid Sayyid, I'm a urologic oncology fellow at the University of Toronto, along with Zach Klaassen, assistant professor and program director at Augusta University. We'll be detailing the latest publication from the MARCAP Group looking at a meta-analysis of candidate surrogate endpoints in advanced prostate cancer. This article was recently published in the open-access NEJM Evidence.
Why do we need to validate intermediate clinical endpoints as surrogates of overall survival? Typically, we get results faster with intermediate clinical endpoints, and as such, if we validate them, we can decrease the required resources for conduct of clinical trials, and this can allow for more rapid patient access to effective therapies. The ICECaP Working Group, which is the Intermediate Clinical Endpoints in Cancer of the Prostate, established previously metastasis-free survival as a surrogate endpoint for overall survival, and this has since been validated for localized and biochemical recurrent prostate cancer. However, responding surrogates for patients with advanced prostate cancer, meaning those with metastatic and/or castrate-resistant disease, are currently lacking.
The ICECaP, in contrast to other working groups, relies on a two-stage meta-analytic approach for evaluation of the surrogacy of intermediate clinical endpoints. This is in contrast to the one-step Prentice criteria approach, whereby surrogate endpoints identified via this specific approach have subsequently failed validation with the two-stage approach. And so given the increased volume of clinical trials of drug therapy in advanced prostate cancer, there clearly is a need for validated intermediary endpoints in this disease space in lieu of consistently available overall survival data.
The authors conducted a systematic review of all randomized therapeutic trials in the advanced prostate cancer space, and they conducted a systematic Medline search using PubMed from January 1970 to until October 2022. They defined advanced prostate cancer as those patients with lymph node-positive disease, metastatic castrate-sensitive prostate cancer, non-metastatic castrate-resistant, or metastatic castrate-resistant disease. They only included those studies that reported on overall survival outcomes and at least one intermediate clinical endpoint with a sample size of at least 70 patients.
In total, 143 studies were identified and the variables were extracted by three authors, and the variables were treatment information, what the primary endpoint was, whether a study was single versus multi-center, what was the median follow-up, the sample size, the median age, and the median PSA level. Importantly, they extracted the hazard ratios for the overall survival outcome and the intermediate clinical endpoints. If these were not available, yet data from a Kaplan-Meier curve, for example, were available, then these were inferred using the WebPlotDigitizer app.
The intermediate clinical endpoints that were evaluated included the time to biochemical failure, the time to clinical failure, biochemical failure-free survival, and progression-free survival. The correlations were performed only if there were five or more trials available for assessment of the treatment correlations. The authors did conduct subgroup analysis by the castration status, meaning whether patients were castrate sensitive or castrate resistant. They also looked at the treatment classes that the patients received, and this was grouped in one of three. Chemotherapy was one ADT, meaning orchiectomy or first generation LHRH, agonist/antagonist and/or an anti-androgen, and then they looked at androgen receptor signaling inhibitors such as Abi, Enza, Apa, and Daro.
The authors did conduct a sensitivity analysis by conducting a leave-one-out cross validation being performed, and this is to assess the effect of individual data points. They also assessed for publication bias using funnel plots. Furthermore, they performed a residual plot sensitivity analysis to assess for changes over time the candidate surrogacy of the endpoints. And based on these results, they performed three further sensitivity analyses looking at year of publication, divided into one of three, pre 2010, 2011-2016, or 2017-2022. They also conducted it based on the Prostate Cancer Working Group definition used, and they also did that by the prior receipt of chemotherapy in the mCRPC setting.
The candidacy for surrogates was evaluated at the trial level using the second condition of the two-stage meta-analytic approach, and this evaluated the correlation between treatment effect estimates on the ICE and the overall survival. Correlation was quantified using Pearson's R-squared, and this was weighted by the inverse variance of the log of the intermediate clinical endpoint hazard ratio, with surrogacy defined on the basis of an R-squared of at least 0.7. Importantly, they also looked at the surrogate threshold defects, which were calculated using the 95% confidence interval for the regression line.
Of number one, the treatment effect on overall survival versus the treatment effect on the intermediate clinical endpoint, and the intersection of the upper bound of the 95% confidence interval of the ICE with the horizontal line, meaning a hazard ratio of 1 or no effect of the treatment on overall survival. This intersection was defined as the surrogate threshold effect corresponding to no overall survival benefit.
At this point, I'll turn it over to Zach to discuss the results and the discussion.
Zach Klaassen: Thanks so much, Rashid. This is the PRISMA diagram for this MARCAP analysis. As you can see at the top, there was 4,420 records that were identified through database searching. Ultimately, 3,910 were excluded, 523 were assessed at the full-text level, and ultimately, 143 trials were analyzing this large analysis.
This is the table one, looking at patient and prostate cancer characteristics for each subgroup analyzed. Again, I've highlighted in the oval circle here that this is a large analysis of 143 trials, more than 75,000 patients. You can see the breakdown here by mCRPC, mCSPC, chemo, ADT, second-generation ARSI, radiographic plus other, and radiographic only, and good numbers of trials in each of these specific disease states. When we look at the median follow up, this roughly ranged from about 1.8 to 3.9 years. As expected, the median age for these trials was roughly 70 years of age. In terms of multi-center studies, at the very bottom here we can see that the majority of all of these trials were multi-center, as expected.
The next several slides will look at the correlation of treatment effect of several of the intermediate clinical endpoints with overall survival. This is the first one looking at biochemical failure. We can see here R-squared of only 0.42, and based on Rashid's definition as explained in the methods, this was less than 0.70 and thus not a good surrogate for overall survival. When we look at the effect of biochemical failure-free survival with overall survival, again, we've seen R-squared slightly higher at 0.57, but again, less than the threshold of 0.70 for surrogacy. Clinical failure was the worst of these intermediate clinical endpoints, R-squared of only 0.31, so no correlation of treatment effect between clinical failure with overall survival. Finally, in this slide here we see progression-free survival with an R-squared of only 0.50 surrogate threshold for efficacy, 0.44, again, less than the threshold of 0.70.
What theauthors then looked at was the correlation of treatment effective progression-free survival with overall survival for trials that allowed the use of radiographic findings in the definition of rPFS. And so the next three slides will look at several breakdowns of this definition. This is for all the advanced prostate cancer trials. Again, disappointingly an R-squared of only 0.50, again, less than the threshold of 0.70. This looked at just the metastatic castrate-sensitive prostate cancer trials, and we do see an improvement in the R-squared at 0.634 mCSPC. So this is the highest R-squared we've seen thus far, although slightly less than the threshold of 0.70. And again, disappointingly for mCRPC trials, no correlation here. R-squared of 0.46 surrogate for threshold of effect of 0.42.
When we look at the surrogacy analysis of subgroups, this is broken down again similar to the table one, looking at mCRPC, mCSPC, chemo, ADT, et cetera, we see several important points. On the left-hand side is the intermediate clinical endpoint listed here, biochemical failure, biochemical failure-free survival, clinical failure, and progression-free survival. And there's several important points to note out here.
First is that for mCSPC, biochemical failure did have an R-squared of 0.83, which is higher than the threshold of 0.70, but this is based on only six trials with a low surrogate threshold effect at 0.30. Again, when we look at the ADT-only trials for clinical failure, this is, again, based on a small number of trials, only five trials, and based on the current paradigm for treatment, ADT alone is probably less clinically relevant. I will also point out that these had low R-squared values for the second-generation ARSIs, 0.21 for biochemical failure and 0.25 for progression-free survival.
By way of discussion, these intermediate clinical endpoints for advanced prostate cancer, namely biochemical failure, biochemical failure-free survival, clinical failure, and progression-free survival, did not meet the second criteria of the two-stage meta-analytical approach for surrogacy. As we discussed, these results were consistent across castration sensitive status treatment, subgroup and PFS definition. As Rashid mentioned in the introduction, identifying surrogate endpoints may shorten the lengthy follow-up time needed in clinical trials, as well as decreased cost associated with running these clinical trials. However, no rigorously evaluated validated surrogate endpoints exist for advanced prostate cancer.
We did see that in this analysis, all endpoints tested seem to be numerically improved for metastatic castrate-sensitive disease versus metastatic castrate-resistant disease, however, this should be interpreted with caution given the generally wide confidence intervals for mCSPC.
In conclusion, this MARCAP analysis was unable to identify any intermediate clinical endpoints that could consistently serve as a surrogate endpoint in advanced prostate cancer. Further efforts are needed to determine if time-specific endpoints or alternative PFS composite endpoints, including molecular imaging, specific tumor size, or PSA cutoffs, could serve as valid and robust surrogate endpoints. Until then, cautions should be realized in the use of biochemical failure, biochemical failure-free survival, progression-free survival, and radiographic progression-free survival as surrogate endpoints for overall survival in advanced prostate cancer clinical trial design.
We thank you very much for your attention. We hope we enjoyed the UroToday discussion of the recently published MARCAP analysis.
*NEJM Evidence is a new monthly digital journal from NEJM Group and is separate from the New England Journal of Medicine (NEJM). NEJM Evidence focuses on areas like clinical trial design, multiyear follow-ups of original research breakthroughs, and first-in-human early-phase trials. They ultimately hope to help clinicians make decisions based on evidence rather than opinion.
Rashid Sayyid: Hello everyone. This is Rashid Sayyid, I'm a urologic oncology fellow at the University of Toronto, along with Zach Klaassen, assistant professor and program director at Augusta University. We'll be detailing the latest publication from the MARCAP Group looking at a meta-analysis of candidate surrogate endpoints in advanced prostate cancer. This article was recently published in the open-access NEJM Evidence.
Why do we need to validate intermediate clinical endpoints as surrogates of overall survival? Typically, we get results faster with intermediate clinical endpoints, and as such, if we validate them, we can decrease the required resources for conduct of clinical trials, and this can allow for more rapid patient access to effective therapies. The ICECaP Working Group, which is the Intermediate Clinical Endpoints in Cancer of the Prostate, established previously metastasis-free survival as a surrogate endpoint for overall survival, and this has since been validated for localized and biochemical recurrent prostate cancer. However, responding surrogates for patients with advanced prostate cancer, meaning those with metastatic and/or castrate-resistant disease, are currently lacking.
The ICECaP, in contrast to other working groups, relies on a two-stage meta-analytic approach for evaluation of the surrogacy of intermediate clinical endpoints. This is in contrast to the one-step Prentice criteria approach, whereby surrogate endpoints identified via this specific approach have subsequently failed validation with the two-stage approach. And so given the increased volume of clinical trials of drug therapy in advanced prostate cancer, there clearly is a need for validated intermediary endpoints in this disease space in lieu of consistently available overall survival data.
The authors conducted a systematic review of all randomized therapeutic trials in the advanced prostate cancer space, and they conducted a systematic Medline search using PubMed from January 1970 to until October 2022. They defined advanced prostate cancer as those patients with lymph node-positive disease, metastatic castrate-sensitive prostate cancer, non-metastatic castrate-resistant, or metastatic castrate-resistant disease. They only included those studies that reported on overall survival outcomes and at least one intermediate clinical endpoint with a sample size of at least 70 patients.
In total, 143 studies were identified and the variables were extracted by three authors, and the variables were treatment information, what the primary endpoint was, whether a study was single versus multi-center, what was the median follow-up, the sample size, the median age, and the median PSA level. Importantly, they extracted the hazard ratios for the overall survival outcome and the intermediate clinical endpoints. If these were not available, yet data from a Kaplan-Meier curve, for example, were available, then these were inferred using the WebPlotDigitizer app.
The intermediate clinical endpoints that were evaluated included the time to biochemical failure, the time to clinical failure, biochemical failure-free survival, and progression-free survival. The correlations were performed only if there were five or more trials available for assessment of the treatment correlations. The authors did conduct subgroup analysis by the castration status, meaning whether patients were castrate sensitive or castrate resistant. They also looked at the treatment classes that the patients received, and this was grouped in one of three. Chemotherapy was one ADT, meaning orchiectomy or first generation LHRH, agonist/antagonist and/or an anti-androgen, and then they looked at androgen receptor signaling inhibitors such as Abi, Enza, Apa, and Daro.
The authors did conduct a sensitivity analysis by conducting a leave-one-out cross validation being performed, and this is to assess the effect of individual data points. They also assessed for publication bias using funnel plots. Furthermore, they performed a residual plot sensitivity analysis to assess for changes over time the candidate surrogacy of the endpoints. And based on these results, they performed three further sensitivity analyses looking at year of publication, divided into one of three, pre 2010, 2011-2016, or 2017-2022. They also conducted it based on the Prostate Cancer Working Group definition used, and they also did that by the prior receipt of chemotherapy in the mCRPC setting.
The candidacy for surrogates was evaluated at the trial level using the second condition of the two-stage meta-analytic approach, and this evaluated the correlation between treatment effect estimates on the ICE and the overall survival. Correlation was quantified using Pearson's R-squared, and this was weighted by the inverse variance of the log of the intermediate clinical endpoint hazard ratio, with surrogacy defined on the basis of an R-squared of at least 0.7. Importantly, they also looked at the surrogate threshold defects, which were calculated using the 95% confidence interval for the regression line.
Of number one, the treatment effect on overall survival versus the treatment effect on the intermediate clinical endpoint, and the intersection of the upper bound of the 95% confidence interval of the ICE with the horizontal line, meaning a hazard ratio of 1 or no effect of the treatment on overall survival. This intersection was defined as the surrogate threshold effect corresponding to no overall survival benefit.
At this point, I'll turn it over to Zach to discuss the results and the discussion.
Zach Klaassen: Thanks so much, Rashid. This is the PRISMA diagram for this MARCAP analysis. As you can see at the top, there was 4,420 records that were identified through database searching. Ultimately, 3,910 were excluded, 523 were assessed at the full-text level, and ultimately, 143 trials were analyzing this large analysis.
This is the table one, looking at patient and prostate cancer characteristics for each subgroup analyzed. Again, I've highlighted in the oval circle here that this is a large analysis of 143 trials, more than 75,000 patients. You can see the breakdown here by mCRPC, mCSPC, chemo, ADT, second-generation ARSI, radiographic plus other, and radiographic only, and good numbers of trials in each of these specific disease states. When we look at the median follow up, this roughly ranged from about 1.8 to 3.9 years. As expected, the median age for these trials was roughly 70 years of age. In terms of multi-center studies, at the very bottom here we can see that the majority of all of these trials were multi-center, as expected.
The next several slides will look at the correlation of treatment effect of several of the intermediate clinical endpoints with overall survival. This is the first one looking at biochemical failure. We can see here R-squared of only 0.42, and based on Rashid's definition as explained in the methods, this was less than 0.70 and thus not a good surrogate for overall survival. When we look at the effect of biochemical failure-free survival with overall survival, again, we've seen R-squared slightly higher at 0.57, but again, less than the threshold of 0.70 for surrogacy. Clinical failure was the worst of these intermediate clinical endpoints, R-squared of only 0.31, so no correlation of treatment effect between clinical failure with overall survival. Finally, in this slide here we see progression-free survival with an R-squared of only 0.50 surrogate threshold for efficacy, 0.44, again, less than the threshold of 0.70.
What theauthors then looked at was the correlation of treatment effective progression-free survival with overall survival for trials that allowed the use of radiographic findings in the definition of rPFS. And so the next three slides will look at several breakdowns of this definition. This is for all the advanced prostate cancer trials. Again, disappointingly an R-squared of only 0.50, again, less than the threshold of 0.70. This looked at just the metastatic castrate-sensitive prostate cancer trials, and we do see an improvement in the R-squared at 0.634 mCSPC. So this is the highest R-squared we've seen thus far, although slightly less than the threshold of 0.70. And again, disappointingly for mCRPC trials, no correlation here. R-squared of 0.46 surrogate for threshold of effect of 0.42.
When we look at the surrogacy analysis of subgroups, this is broken down again similar to the table one, looking at mCRPC, mCSPC, chemo, ADT, et cetera, we see several important points. On the left-hand side is the intermediate clinical endpoint listed here, biochemical failure, biochemical failure-free survival, clinical failure, and progression-free survival. And there's several important points to note out here.
First is that for mCSPC, biochemical failure did have an R-squared of 0.83, which is higher than the threshold of 0.70, but this is based on only six trials with a low surrogate threshold effect at 0.30. Again, when we look at the ADT-only trials for clinical failure, this is, again, based on a small number of trials, only five trials, and based on the current paradigm for treatment, ADT alone is probably less clinically relevant. I will also point out that these had low R-squared values for the second-generation ARSIs, 0.21 for biochemical failure and 0.25 for progression-free survival.
By way of discussion, these intermediate clinical endpoints for advanced prostate cancer, namely biochemical failure, biochemical failure-free survival, clinical failure, and progression-free survival, did not meet the second criteria of the two-stage meta-analytical approach for surrogacy. As we discussed, these results were consistent across castration sensitive status treatment, subgroup and PFS definition. As Rashid mentioned in the introduction, identifying surrogate endpoints may shorten the lengthy follow-up time needed in clinical trials, as well as decreased cost associated with running these clinical trials. However, no rigorously evaluated validated surrogate endpoints exist for advanced prostate cancer.
We did see that in this analysis, all endpoints tested seem to be numerically improved for metastatic castrate-sensitive disease versus metastatic castrate-resistant disease, however, this should be interpreted with caution given the generally wide confidence intervals for mCSPC.
In conclusion, this MARCAP analysis was unable to identify any intermediate clinical endpoints that could consistently serve as a surrogate endpoint in advanced prostate cancer. Further efforts are needed to determine if time-specific endpoints or alternative PFS composite endpoints, including molecular imaging, specific tumor size, or PSA cutoffs, could serve as valid and robust surrogate endpoints. Until then, cautions should be realized in the use of biochemical failure, biochemical failure-free survival, progression-free survival, and radiographic progression-free survival as surrogate endpoints for overall survival in advanced prostate cancer clinical trial design.
We thank you very much for your attention. We hope we enjoyed the UroToday discussion of the recently published MARCAP analysis.
*NEJM Evidence is a new monthly digital journal from NEJM Group and is separate from the New England Journal of Medicine (NEJM). NEJM Evidence focuses on areas like clinical trial design, multiyear follow-ups of original research breakthroughs, and first-in-human early-phase trials. They ultimately hope to help clinicians make decisions based on evidence rather than opinion.