Evaluation of Machine-Learning Methods to Estimate Conditional Average Treatment Effects From Randomized Clinical Trial Data With Time-To-Event Outcomes
Elvire Roblin, Paul-Henry Cournède, Stefan MichielsObjective
In randomized clinical trials, prediction models can be used to explore the relationships between patients’ variables and treatment effect magnitude. Our aim was to evaluate flexible machine learning models that incorporate interactions and nonlinear effects from high-dimensional data to estimate individualized treatment recommendations in trials with time-to-event outcomes.
Methods
We compared neural networks (CoxCC and CoxTime) and random survival forests, including Interaction Forests (IF) and Causal Survival Forest (CSF) against a Cox proportional hazards model with an adaptive LASSO (ALASSO) penalty as a benchmark. For individualized treatment recommendations, we adapted metrics originally designed for binary outcomes to accommodate time-to-event data with censoring, using both survival-probability and restricted mean survival time estimates of treatment benefit. Metrics included the C-for-Benefit, E50-for-Benefit, Integrated Calibration Index for benefit, and root-mean-squared error for treatment benefit. We also evaluated decision-making utility using the Area Under the Targeting Operator Characteristic and Population Benefit (PB). An extensive simulation study used two data generation processes incorporating nonlinearity and interactions. The models were applied to three cancer clinical trial datasets.
Results
In the first data generation process, neural networks outperformed ALASSO in calibration while IF showed superior C-for-benefit performance. In the second one, machine learning methods outperformed the benchmark linear ALASSO method across discrimination, calibration, and RMSE. In the cancer trial datasets, the machine learning methods often performed better than ALASSO: neural networks consistently achieved the best calibration, IF achieved the highest C-for-Benefit, and CSF showed strong calibration and the highest PB in the trastuzumab dataset.
Conclusion
Machine learning-based survival models can efficiently estimate conditional average treatment effects from randomized trials when strong nonlinear and interaction effects are expected. These flexible approaches offer valuable alternatives to traditional regression methods for developing treatment recommendation rules from clinical trials with time-to-event outcomes and medium to high-dimensional data set sizes.