AUA 2017: John K. Lattimer Lecture – Net benefit and clinical decision-making in urology

Boston, MA (UroToday.com) Dr. Vickers gave the annual John K. Lattimer lecture discussing decision-making in urology. Dr. Vickers got his start in prostate cancer biomarkers where he noted a huge number of publications with many statistical analyses within each reference. Curiously, none of these works were able to provide an easily understandable assessment of how good a new marker is for prostate cancer. Traditional measures such as sensitivity, specificity, positive predictive value, negative predictive value, and area under the curve do not offer clear answers to the question of what makes a good biomarker. In fact, even famous statisticians are unsure and often disagree regarding how to best use these traditional statistical measures to determine the value of a test. Specifically, these statistical measures do not answer the real clinical question at hand; namely, does the biomarker help to decide whether or not a prostate biopsy should be performed.

Decision analysis has emerged as a statistical tool to answer real world questions. It requires a definition of the probability of an event occurring and the value of a given result. These two are multiplied together to obtain expected utility. In truth, these concepts are not new (well known in economics and business analyses); however, their application to medicine and urology is novel. Proportions are not difficult to obtain as they are readily available in the literature. Value judgements are more challenging. For example, how does one put a value on the harm of an unnecessary biopsy or the harm of missing a clinically significant prostate cancer? Dr. Vickers went about this by asking his urologic oncology colleagues at Memorial Sloan Kettering Cancer Center the following question: how many biopsies would you do to find a single high grade cancer? The answer settled on approximately 10. Therefore, the “exchange rate” in this example would be 9 unnecessary biopsies to find 1 high-grade cancer. With that known, we can apply these concepts to the following set of circumstances to determine utility. Let’s say that when all men with elevated PSA are biopsied, 250 high grade cancers are found at the expense of 750 unnecessary biopsies. Now, let’s say that when biopsy is done only in the presence of a positive biomarker test, 225 high risk cancers are found at the expense of 400 unnecessary biopsies. Applying the 9:1 unnecessary biopsy to high-risk cancer discovery threshold, the utility of biopsy with elevated PSA is 166.7 (250 – 750/9). Similarly, the utility of biopsy only in the presence of a positive biomarker is 180.6 (225 – 400/9). Comparing the two, decision analysis would lead one to choose the utility of biopsy only in the presence of a positive biomarker under these circumstances.

There are some nuances within decision analysis. For example, the “exchange rate” can change on a personalized basis. For older men, a higher threshold may be used (accept only 4 unnecessary biopsies for each high-risk cancer detected. In contrast, a lower threshold may be used for younger, healthy men (accept 20 unnecessary biopsies for each high-risk cancer detected). The decision curve, then, plots threshold probability on the x axis and net benefit on the y axis. For a given threshold probability, utility can be compared across different scenarios to determine the best course of action.

Dr. Vickers concluded that many of the statistics currently reported in medical research have no value for clinical decision making. Fortunately, urology has led the way in simple-to-apply decision analysis with the goal of promoting net benefit analysis to make better decisions for our patients.

Presented By: Andrew Vickers, PhD, Memorial Sloan Kettering Cancer Center

Written By: Benjamin T. Ristau, MD, Fox Chase Cancer Center, Philadelphia, PA

at the 2017 AUA Annual Meeting - May 12 - 16, 2017 – Boston, Massachusetts, USA

Login