Sample Size Reduction by Applying ML Based Causal Inference Methods

ABSTRACT

We conducted a comprehensive comparative analysis of causal machine learning (ML) methods to assess their utility in improving the efficiency of clinical trial designs with or without historical data. Specifically, we compared standard ANCOVA analysis used in a Randomized Controlled Trial (RCT) with several causal ML methods, including PROCOVA, TMLE, DML, and GRF. PROCOVA is gaining popularity in RCT design and requires a historical data for prognostic scores, but other methods can be applied with or without such data. Our primary focus was on strict RCT setting without borrowing historical control data, though we also explored the impact of borrowing data. The historical data used consisted of placebo data from two Phase 3 Ophthalmology studies with a continuous primary endpoint. We employed a generative AI approach, specifically Generative Adversarial Networks (GANs), to simulate RCT data from the historical data under various scenarios, varying treatment effects with and without treatment effect heterogeneity, RCT sizes, and bias. Results showed that causal ML methods can increase power even without borrowing historical data. For example, TMLE increased effective sample size by 21% in one scenario. In scenarios with borrowing of controls, PROCOVA increased power while controlling type 1 error, showing robustness to model misspecification.