How methodological cues shape trust in psychological research and legal judgements: a vignette study

Abstract

Over the past decade, psychology has embraced transparency and reproducibility reforms to address the replication crisis, raising the question of whether these signals help legal decision makers identify reliable psychological evidence. This study examines how these reforms influence perceived study reliability and downstream decision-making. We hypothesized that higher methodological quality (larger sample sizes, preregistration, and replications or meta-analyses) would increase perceived reliability and influence guilt judgements. We conducted an online vignette experiment, including individuals with doctorates and laypersons, who evaluated fictitious legal cases accompanied by research summaries varying in sample size (low versus high), preregistration status (preregistered versus not) and study design (primary study versus replication versus meta-analysis). Larger sample sizes consistently enhanced perceived reliability across both groups, whereas preregistration did not. Doctorate participants rated replications and meta-analyses as more reliable than primary studies; laypersons did not differentiate among designs. Importantly, none of the methodological quality cues influenced guilt judgements, despite shaping reliability judgements. This suggests that recognizing methodological quality alone may not shape downstream decisions. Future research should replicate the evidence in more ecologically valid legal designs (e.g. expert testimony) and with a broader range of methodological indicators to better understand how methodological quality influences perceived reliability and applied reasoning.