Probabilistic interpretation of clinical trial results to improve interpretation and communication of findings
G Siontis, K Chalkou, T Perneger, G Salanti, A Gayet-Ageron, O EfthimiouAbstract
Background
Clinical trials are usually analyzed using frequentist methods and interpreted through p-values, which are often poorly understood. In contrast, Bayesian analyses yield probabilistic statements that offer a more intuitive interpretation of treatment effects and uncertainty.
Purpose
To demonstrate the utility of probabilistic statements, we conducted Bayesian analyses in two settings: (i) published randomized controlled trials conventionally considered "negative" (p>0.05); (ii) a meta-analysis of percutaneous coronary interventions in elderly patients with NSTEMI.
Methods
We considered 169 primary outcomes in 130 negative trials published in high impact general medical journals during a single year (2021). For each outcome, we extracted the treatment effect and corresponding standard error. We re-analysed data using Bayesian methods to obtain posterior probabilities that the experimental treatment is better than control, and that the effect is above specific thresholds for clinical significance. For the case example of meta-analysis, we used Bayesian random-effects models with minimally informative priors to summarize the available evidence and produce probabilistic statements regarding the treatment effect of an invasive compared to a conservative strategy.
Results
Among the 169 outcomes of the negative trials, 113 (70%) reported the treatment effect using ratios, while 56 (33%) reported it as differences. Although all ratios were formally statistical "non-significant", for 23/113 (20%) of the outcomes there was >90% probability that the treatment is superior to control. For 10/113 (9%), this probability was >95%. In 3/56 (5%) of the differences, the probability that the experimental treatment is better than control was ≥90%. In the meta-analysis of invasive strategy in elderly with NSTEMI, for the composite of all-cause mortality or myocardial infarction, we found weak evidence for superiority of the invasive strategy (HR 0.89, 95%CrI 0.68–1.14; 86% probability of benefit). While there was strong evidence that invasive treatment reduces myocardial infarction risk (HR 0.70, 95%CrI 0.56–0.86; ~100% probability of benefit), there was weak evidence of increased all-cause and cardiovascular mortality compared with conservative management (posterior probabilities of 91% and 67%, respectively).
Conclusions
We illustrated how Bayesian methods can be used to derive probabilistic statements for treatment effects. Such statements can provide a nuanced understanding of scientific findings, improve interpretation, and greatly enhance communication between researchers, clinicians, patients, and guideline developers. Relying solely on statistical significance, without considering clinical importance, can mislead decision-making and potentially deprive patients of beneficial therapies. To facilitate the uptake of the method, we developed a web-app that can be used to produce probabilistic statements.