This study aims to evaluate the reliability, quality and readability of ChatGPT-4o responses regarding pediatric urolithiasis. Forty frequently asked questions about pediatric urinary stones were posed to ChatGPT-4o twice, one week apart. The reliability of ChatGPT-4o's responses was assessed using the five-point DISCERN tool (mDISCERN). The overall quality of the responses was evaluated using the Global Quality Scale (GQS). To assess the readability of ChatGPT-4o's responses, multiple metrics were employed, including the Flesch Reading Ease (FRE) score, the Flesch-Kincaid Grade Level (FKGL), the Gunning Fog Index (GFI), the Coleman-Liau Index (CLI), and the Simple Measure of Gobbledygook (SMOG). The median mDISCERN score was 5 (range: 4-5), and the median GQS score was 5 (range: 3-5), indicating high reliability and quality. However, readability metrics suggested a high level of difficulty: FRE (27.98 ± 13.65), FKGL (11.46 ± 1.88), SMOG (14.96 ± 1.64), GFI (17.27 ± 2.37), and CLI (15.60 ± 1.95). Only 2.5% of responses were understandable to individuals with reading skills at a 10-12-year-old level, 37.5% were suitable for college-level readers, and 60% required professional-level comprehension. A moderate correlation was observed between mDISCERN and GQS scores (r = 0.42, p = 0.007), but neither correlated significantly with readability metrics. ChatGPT-4o may provide reliable and high-quality information about pediatric urinary stones; however, the advanced reading level of its responses presents a significant barrier to accessibility for patients and caregivers. Therefore, despite its potential utility, the readability challenge must be addressed to ensure equitable patient education.
Urolithiasis. 2025 Oct 23*** epublish ***
Esra Ensari, Asya Eylem Boztas
Ministry of Health Antalya City Hospital, Department of Pediatric Nephrology, Antalya, Turkey. ., Health Sciences University, Dr. Behcet Uz Pediatric Diseases and Surgery Training and Research Hospital, Department of Pediatric Surgery, Izmir, Turkey.