Separating the Wheat from the Chaff: An Evaluation of Online Health Information for Treatment of Peyronie's Disease – Beyond the Abstract
The search term "Peyronie's disease" was queried on three major search engines (Google, Bing, Yahoo) and the first 50 search results on each search engine were assessed. All websites were categorized as institutional/reference, commercial, charitable, personal/patient support, or alternative medicine, and cross-referenced with HON. The first 30 sentences on each website discussing treatment options were analyzed for readability using a three-pronged set of validated algorithms: The Gunning-Fog score, SMOG index (Simplified Measure of Gobbledygook), and Dale-Chall readability formula. Quality was analyzed using the validated Oxford DISCERN instrument, a 16-item tool that assesses the quality of written information about treatment choices for a health problem. Finally, a fellowship-trained urologist graded accuracy.
This study found that most of the online health information about the treatment of Peyronie's disease is found on websites without HONcode certification and is on average written at or above the 11th grade level, exceeding the current reading guidelines of 6th-8th grade. The mean total DISCERN score for all website categories was 50.44 (SD 11.94), the upper range of "fair" quality. Both institutional/reference and HONcode-certified websites were of "good" quality (53.44, SD 11.64 and 60.86, SD 8.74, respectively), while those without HONcode certification were of significantly lower “fair” quality (47.84, SD 11.24) as compared to HONcode-certified websites (t19=4.2, P<.001). Alternative medicine websites had the lowest mean total DISCERN score (34.63, SD 13.27) and fell within the “poor” quality range. Across all website categories, DISCERN quality elements that scored lowest were those that assessed if “the sources of information are clear” and if the source describes “the risks of treatment.” The mean accuracy score of all online Peyronie's treatment information was 2.76 (SD 1.23), corresponding to only 25%-50% accurate information. Institutional/reference websites were 50%-75% accurate (3.13, SD 1.20), while personal/patient support (1.50, SD 0.84) and alternative medicine websites (1.25, SD 0.50) were <25% accurate. Websites with HONcode certification did not have significantly more accurate information than those websites without HONcode certification (t19=.82, P=.42).
One of the most concerning findings in this study was that the mean readability of all websites exceeds the reading ability of most US adults by several grade levels. Despite the mediocre quality and accuracy scores across all websites, the information from institutional/reference websites was found to be of better quality and accuracy, and the information from HONcode-certified websites was found to be of better quality. The HONcode toolbar appears to be an appropriate device for patients to assess website quality, but caution should be used when applying it purely as a determinant of medical accuracy, as this is not the toolbar’s intended purpose.4 Based on DISCERN analysis, website quality could potentially be improved by making source citations clearer and better describing the risks of treatment. Urologists should continue to work to enhance health information online for Peyronie’s disease and its treatment while assisting patients in navigating digital resources to find accurate, high-quality websites that are written at the recommended 6th-8th grade reading level. This study could be further developed by assessing non-English websites and devising a system of analyzing non-written content such as videos and figures.
Written by: Peter Andrew Holoch, MD, Larner College of Medicine, the University of Vermont
1) Doak CC, Doak LG, Friedell GH, Meade CD. Improving comprehension for cancer patients with low literacy skills: strategies for clinicians. CA Cancer J Clin. 1998;48(3):151-62.
2) B.D. Weis. Health Literacy and Patient Safety: Help Patients Understand: Manual for Clinicians (2nd ed), American Medical Association, Chicago, IL. 2007.
3) National Institutes of Health. How to Write Easy-to-Read Health Materials. MedlinePlus. Published June 28, 2017. Accessed November 26, 2017.
4) The commitment to reliable health and medical information on the internet. Health on the Net Foundation. Published May 2, 2017. Accessed November 5, 2017.
Read the Abstract