Artificial intelligence (AI) has shown promise for diagnosing prostate cancer in biopsies. However, results have been limited to individual studies, lacking validation in multinational settings. Competitions have been shown to be accelerators for medical imaging innovations, but their impact is hindered by lack of reproducibility and independent validation. With this in mind, we organized the PANDA challenge-the largest histopathology competition to date, joined by 1,290 developers-to catalyze development of reproducible AI algorithms for Gleason grading using 10,616 digitized prostate biopsies. We validated that a diverse set of submitted algorithms reached pathologist-level performance on independent cross-continental cohorts, fully blinded to the algorithm developers. On United States and European external validation sets, the algorithms achieved agreements of 0.862 (quadratically weighted κ, 95% confidence interval (CI), 0.840-0.884) and 0.868 (95% CI, 0.835-0.900) with expert uropathologists. Successful generalization across different patient populations, laboratories and reference standards, achieved by a variety of algorithmic approaches, warrants evaluating AI-based Gleason grading in prospective clinical trials.
Nature medicine. 2022 Jan 13 [Epub]
Wouter Bulten, Kimmo Kartasalo, Po-Hsuan Cameron Chen, Peter Ström, Hans Pinckaers, Kunal Nagpal, Yuannan Cai, David F Steiner, Hester van Boven, Robert Vink, Christina Hulsbergen-van de Kaa, Jeroen van der Laak, Mahul B Amin, Andrew J Evans, Theodorus van der Kwast, Robert Allan, Peter A Humphrey, Henrik Grönberg, Hemamali Samaratunga, Brett Delahunt, Toyonori Tsuzuki, Tomi Häkkinen, Lars Egevad, Maggie Demkin, Sohier Dane, Fraser Tan, Masi Valkonen, Greg S Corrado, Lily Peng, Craig H Mermel, Pekka Ruusuvuori, Geert Litjens, Martin Eklund, PANDA challenge consortium
Department of Pathology, Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, The Netherlands. ., Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden. ., Google Health, Palo Alto, CA, USA. ., Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden., Department of Pathology, Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, The Netherlands., Google Health, Palo Alto, CA, USA., Department of Pathology, Antoni van Leeuwenhoek Hospital, The Netherlands Cancer Institute, Amsterdam, The Netherlands., Laboratory of Pathology East Netherlands, Hengelo, The Netherlands., Department of Pathology and Laboratory Medicine, University of Tennessee Health Science Center, Memphis, TN, USA., Laboratory Medicine, Mackenzie Health, Toronto, Ontario, Canada., Department of Pathology, Laboratory Medicine and Pathology, University Health Network and University of Toronto, Toronto, Ontario, Canada., Pathology and Laboratory Medicine Service, North Florida/South Georgia Veterans Health System, Department of Pathology, Immunology and Laboratory Medicine, University of Florida, Gainesville, FL, USA., Department of Pathology, Yale School of Medicine, New Haven, CT, USA., Aquesta Uropathology and University of Queensland, Brisbane, QLD, Australia., Department of Pathology and Molecular Medicine, Wellington School of Medicine and Health Sciences, University of Otago, Wellington, New Zealand., Department of Surgical Pathology, School of Medicine, Aichi Medical University, Nagakute, Japan., Faculty of Medicine and Health Technology, Tampere University, Tampere, Finland., Department of Oncology and Pathology, Karolinska Institutet, Stockholm, Sweden., Kaggle Inc, Mountain View, CA, USA., Institute of Biomedicine, Cancer Research Unit and FICAN West Cancer Centre, University of Turku and Turku University Hospital, Turku, Finland.