A Call for Greater Transparency in Piloting

A pilot study is a preliminary investigation conducted before a full-scale research study. Although piloting is a common and valuable part of the research process, pilot studies are not systematically reported (or even acknowledged) in the psychology literature. In this article, we argue that establishing norms for regularly acknowledging preliminary research phases can improve many aspects of psychological research. Specifically, reporting pilot-study information can help researchers provide a more complete record of the research process, reveal selection biases in research designs that may affect the generalizability of findings, evaluate others’ research more accurately, learn from other researchers’ successes and failures, uphold ethical responsibilities, and improve the methodological practice of piloting. We conclude with three concrete suggestions for near-term changes to help pilot reporting become standard practice: Scientific communities and institutions should develop simple pilot-reporting templates, journals should provide clear pilot-reporting guidance for authors, and authors should acknowledge when pilot studies were run and share relevant materials and data. Current reporting standards favor conclusive and polished research narratives, but such narratives are the end product of a complex scientific process that should also be shared. A truly transparent science should spotlight processes, such as piloting, that underpin the conclusions scientific articles draw.