A statistical evaluation of decision-making methods and the efficiency of Bayesian multi-arm multi-stage trials

Background/Aims : Multi-arm multi-stage trials benefit patients by providing a flexible and versatile clinical trial design compared with standard randomized controlled trials. Multi-arm multi-stage trials can evaluate multiple interventions for a single disease, which avoids the need to run multiple trials. Multi-arm multi-stage trials incorporate decision-making at interim analysis, which enables the trial to stop early for futility or efficacy of a treatment. As a result, multi-arm multi-stage trials reach conclusions faster, requiring less time and resources. Despite their growing popularity, limited research has been done to examine how decision-making methods used in Bayesian multi-arm multi-stage trials impact the efficiency of the trial.

Methods : This study examines how decision-making strategies influence the efficiency of Bayesian multi-arm multi-stage trials, including approaches for setting thresholds to declare superiority or futility and evaluating multiple treatments simultaneously. We apply the Nelder–Mead optimization algorithm to determine the decision thresholds that maximize statistical power while maintaining family-wise type I error rate below 5%. We conduct a simulation study to compare the conventional method to evaluate multiple treatments in a Bayesian multi-arm multi-stage trial to three alternatives. At each interim analysis, posterior probabilities are derived from a Normal–Gamma conjugate model, and trial decisions are made by comparing decision criteria derived from the posterior probabilities to the optimized decision thresholds. Simulation scenarios vary by treatment effect size and number of treatment arms to assess the robustness of each decision-making strategy.

Results : All treatment comparison methods achieve similar power across simulation scenarios. However, the optimal decision thresholds vary substantially among methods. These thresholds are also lower than those currently used in Bayesian multi-arm multi-stage trials, which are often conservative and lead to reduced power. Thus, adjusting decision thresholds to the optimized values can improve trial efficiency.

Conclusion : This study provides an exploration of alternative decision-making methods in Bayesian multi-arm multi-stage trials. Initial findings show that optimizing decision-making thresholds can improve the power of the trial without inflating the family-wise type I error rate, thus improving the efficiency of the trial. Further research should include the implementation of complex trial designs and non-normal outcomes to confirm that the results apply to adaptive platform trials.