(UroToday.com) The 108th Scientific Assembly and Annual Meeting of the Radiological Society of North America (RSNA) held in Chicago, IL was host to a plenary session discussing machine learning in radiation oncology clinical trials and clinical practice. Dr. Michael Gensheimer discussed medical record-based machine learning for clinical decision support in radiation oncology. Dr. Gensheimer began by noting that medical record-based machine learning has the ability to nudge providers, while empowering non-physician staff, reducing errors, improving the level of “low performers”, and allowing for better treatment decisions in low-data situations.
Medical record-based machine learning has been shown to strongly predict outcomes. In 2020, Hong et al. published the results of their study in the Journal of Clinical Oncology evaluating the predictive ability of machine learning to identify high-risk patients undergoing radiotherapy or chemoradiation and subsequently direct twice-weekly clinical evaluations to reduce acute care visits during treatment. The authors demonstrated that twice-weekly evaluation reduced rates of acute care during treatment from 22.3% to 12.3% (p=0.02).
Furthermore, the model discrimination was good in high- and low-risk patients undergoing standard once-weekly evaluation (AUC: 0.851).1
In a similar analysis, Gensheimer et al. compared the accuracy of life expectancy predictions by the treating physician, a machine learning model, and a traditional model in patients with metastatic cancer. A machine learning model was trained using 14,600 metastatic cancer patients’ data to predict each patient’s distribution of survival time. The authors demonstrated that the machine learning model outperformed physicians and traditional models for predicting one-year survival, with an area under the curve of 0.77 (95% CI: 0.73 to 0.81) versus 0.72 and 0.68, respectively.2
Importantly, implementation outcomes of these machine-based learning interventions/models appear to be promising. Gensheimer et al. used a computer model to select patients with high-risk metastatic cancer and shorter predicted survival. They next evaluated whether advance care planning (ACP) in these model-selected patients would lead to improved ACP documentation frequency and end-of-life quality measures. Using this model-directed intervention in patients with genitourinary and/or thoracic malignancies, the authors demonstrated that ACP documentation improved from 3% to 35% by the end of the intervention and providers’ prognosis documentation rate also increased from 2 to 27%.3
Quality assurances and data reliability for medical record-based machine learning is critical. Another report by Gensheimer et al. compared predicted survival outcomes for patients based on electronic medical records alone versus electronic medical records and cancer registries. The authors demonstrated that the median overall survival from the date of cancer diagnosis using EMR data was greatly overestimated at 58.7 months (95% CI, 54.2 to 63.2) compared to 20.8 months (95% CI, 19.6 to 22.3) using EMR + Cancer Registry data.4
Medical record-based machine learning may also allow for improved data quality via structured data entry. Studies have demonstrated that the correct primary diagnosis for encounters is entered for 57% of visits only. Thirteen percent of errors are caused by a physician diagnostic error, 8% due to missing encounter forms, and 22% due to incorrectly entered data. Furthermore, documentation errors are common with at least four to five errors per clinical encounter with both erroneous information entered and critical patient-reported information missed as well.
Medical record-based machine learning may also allow for natural language processing that allows for accurate detection of clinically relevant outcomes. In a report by Karimi et al., the authors developed a natural language processing model for recurrence that detected distant recurrence with an AUC of 0.98 (95% CI: 0.96 to 0.99) and 0.95 (95% CI: 0.88 to 0.98) in breast and hepatocellular cancer cohorts, respectively.
Dr. Gensheimer concluded his presentation by noting that a future community approach is critical to maximizing the available dataset size in order to optimize future data-driven machine-based learning models.
Presented by: Michael Gensheimer, MD, Clinical Associate Professor, Department of Radiation Oncology, Stanford, Palo Alto, CA
Written by: Rashid Sayyid, MD, MSc – Society of Urologic Oncology (SUO) Clinical Fellow at The University of Toronto, @rksayyid on Twitter during the 108th Radiological Society of North America (RSNA) Scientific Assembly and Annual Meeting, Nov. 27 - Dec. 1, 2022, Chicago, IL
- Hong JC, et al. System for High-Intensity Evaluation During Radiation Therapy (SHIELD-RT): A Prospective Randomized Study of Machine Learning–Directed Clinical Evaluations During Radiation and Chemoradiation. J Clin Oncol. 2020;38(31):3652-3661
- Genshemier MF, et al. Automated model versus treating physician for predicting survival time of patients with metastatic cancer. J Am Med Inform Assoc. 2021;28(6):1108-1116.
- Gensheimer MF, et al. Use of Machine Learning and Lay Care Coaches to Increase Advance Care Planning Conversations for Patients With Metastatic Cancer. JCO Oncol Pract. 2022;OP2200128.
- Gensheimer MF, et al. Accuracy of Electronic Medical Record Follow-Up Data for Estimating the Survival Time of Patients With Cancer. JCO Clin Cancer Inform. 2022;6:e2200019.