Low Methodological Quality of Systematic Reviews Published in the Urological Literature (2016-2018) - Beyond the Abstract

Systematic reviews are considered the gold standard in evidence-based medicine. However, like primary research, they are subject to a range of biases. We sought to evaluate the quality of systematic reviews published in the urological literature from 2016-2018 using the recently updated Assessment of Multiple Systematic Reviews (AMSTAR)-2 tool.



PubMed was systematically searched for systematic reviews related to questions of prevention and therapy published from January 2016 to December 2018 in five major urology journals: BJU International, European Urology, The Journal of Urology, Urology, and World Journal of Urology. Two reviewers followed an a priori protocol to independently screen references in Rayyan and abstract data using a piloted form based on the 16 domains of AMSTAR-2. Preplanned statistical hypothesis testing was performed by the journal of publication in SPSS version 24.0.

Of the 260 relevant references in our initial search, 144 ultimately met inclusion criteria. The largest contributors by journal of publication were European Urology (53; 36.8%) followed by Urology (36; 25.0%), and BJU International (24; 16.6%). The most common clinical topics were oncology (64; 44.4%) and voiding dysfunction (32; 22.2%) followed by stones/endourology (14; 9.7%). Just over one-third (52; 36.2%) of reviews had a registered protocol. Nearly all studies (139; 96.5%) searched at least two databases. Less than one-third (46; 31.9%) also searched trial registries and one-fifth (30; 20.8%) consulted experts for additional trials. Few studies (14; 10.4%) provided a list of potentially relevant but excluded studies. Only six (4.2%) studies met all AMSTAR-2 critical domains as a prerequisite for high-quality reviews.

These findings suggest a large number of systematic reviews of suboptimal quality are published in the urological literature. Reviews with critical methodological flaws may not provide an accurate summary of the literature on their question of interest. There is a need for educating authors, peer reviewers, and editors alike on the established metrics for high-quality systematic reviews to ensure future improvement.

Written by: Maylynn Ding, Leah Soderberg, Jae Hung Jung, Philipp Dahm

McMaster University, School of Medicine, Hamilton, ON, Canada., Minneapolis VA Medical Center, Urology Section, Minneapolis, MN; University of Minnesota, Department of Urology, Minneapolis, MN., Yonsei University, Wonju College of Medicine, Department of Urology, Wonju, South Korea., Minneapolis VA Medical Center, Urology Section, Minneapolis, MN; University of Minnesota, Department of Urology, Minneapolis, MN. Electronic address: .

Read the Abstract
email news signup