p ‐Values Are Not Evidence: Navigating the Gap Across Modes of Inference

ABSTRACT

The use of p ‐values and statistical significance testing as the primary mode of inference in science has been long debated among statisticians and more recently among ecologists. Ecological literature has discussed embracing other modes of inference, such as Bayesian statistics and the use of direct likelihood inference. We reviewed ecology, evolution, and environmental science literature covering the past three decades to investigate trends in the use of different modes of inference over time and to assess the degree of discrepancy between reported p ‐values and calculated likelihood ratios. p ‐Values and likelihood ratios were found to scale in a more direct way for single degree of freedom tests, but in a more complex manner for models with more than one parameter. These results demonstrate the degree to which the likelihood ratio (a measure of evidence) differs from a p ‐value, a measure of uncertainty that is widely presented as a measure of evidence among publications within the fields of ecology and evolution. Additionally, our investigation revealed that modes of inference within ecological studies changed little over the 20‐year time period, indicating a failure to adopt alternative modes of inference. We also found downward trends in the reporting of p ‐value thresholds (e.g., “ p < 0.05”) in two journals, with no change in three others. The prevalence of the divergence in the measure of evidence and the p ‐value measure of uncertainty remains to be confirmed in other journals, time points, and subfields. An overreliance on p ‐values alone can alter the conclusions drawn from research, impacting management of species and ecosystems. We recommend that authors report both a measure of evidence and uncertainty, and justify their mode of inference in an attempt to produce statistical results that are comprehensive, replicable, and reliably interpretable.