Developing Integer‐based Risk Scores for Predicting the Risk of Incident Cognitive Impairment in Cognitively Normal Older Adults
Kellen K. Petersen, Bhargav Teja Nallapu, Ellen Grober, Richard B. Lipton, John C. Morris, Jason J. Hassenstab, Brian A. Gordon, Christos Davatzikos, Ali Ezzati- Psychiatry and Mental health
- Cellular and Molecular Neuroscience
- Geriatrics and Gerontology
- Neurology (clinical)
- Developmental Neuroscience
- Health Policy
- Epidemiology
Abstract
Background
We aimed to developed integer‐based risk scores for predicting incident cognitive impairment up to 10 years in initially cognitively normal older adults.
Methods
Participants were 479 older adults (aged 60 to 90) from longitudinal studies of aging and Alzheimer’s disease from the Knight ADRC at Washington University in St. Louis. The primary outcome was incident cognitive impairment, defined as conversion from Clinical Dementia Rating (CDR®) 0 to >0. The sample was divided into training (60%) and test (40%) sets. Feature‐sets were formed from combinations of four categories of variables: demographics (D), genetics (G), cognitive measures (C), and biomarkers (B) (Details in Figure 1 legend). Utilizing a modified version of the AutoScore‐Survival algorithm, variables were ranked based on variable importance using reduction in predictive accuracy due to replacement with a random permutation value in combination with Random Survival Forests. Variables were iteratively included in constructing risk score models based on coefficients of Cox regression models.
Results
Five risk score models were developed using five feature‐sets. All models retained age, APOE4 status, free recall from the Free and Cued Selective Reminding Test, hippocampal atrophy, and Tau positivity when included as potential variables in the feature‐sets. The risk score derived from the DGCB model had a Harrell’s concordance index (C‐index) of 0.754 and integrated Area Under the ROC Curve (iAUC) of 0.792. The best performing model without biomarkers used the DGC feature‐set and had a C‐index of 0.694 and iAUC of 0.699 while the risk score using the DG feature‐set only had a C‐index of 0.632 and iAUC of 0.647. Participants who had risk scores in the top quartile of the DGCB model risk score had an increased rate of conversion to CDR > 0.
Conclusions
We developed five risk score models for predicting incident cognitive impairment. Risk scores consisted of different sets of measures which differed in ease‐of‐access and costs. Using such simple risk scores that do not include biomakers as clinical decision support tools is practical in the clinical setting. Including biomarkers in risk scores provides slight improvement to classification accuracy, emphasizing their importance in research settings.