Multiverse Analysis

Abstract

Improving replication rates in science requires that researchers disclose their degrees of freedom by making explicit the arbitrary yet equally defensible choices in design, data processing, and statistical analysis. This approach, known as multiverse analysis, is gaining traction in cognitive neuroscience but remains underused in cognitive enhancement and brain plasticity research involving neuromodulation techniques. This chapter aims to raise awareness of systematic robustness checks within the framework of multiverse analysis and to review relevant older and recently proposed methodological concepts. We then describe a sample experiment in which transcranial alternating current stimulation was applied to upregulate alpha oscillations in the human electroencephalogram and thereby modulate cognitive processes. Using this example, the text highlights the arbitrariness of various methodological choices. The chapter also proposes a collection of alternative specifications and provides an open dataset for applying multiverse analysis to the experiment discussed.