Counting beans in chicken soup: On the limits of experimentalism in literacy educational research

Abstract

Recent systematic reviews by Green and colleagues, including one published in this journal, report an absence of qualifying (quasi‐)experimental evidence for the effectiveness of multiliteracies pedagogy and insufficient rigorous (quasi‐)experimental support for genre‐based pedagogies informed by systemic functional linguistics in improving K–10 literacy outcomes. Specifically, their paper raises the question of whether evaluative paradigms grounded predominantly in experimental ‘bean‐counting’ are epistemologically aligned with the complex, situated and design‐oriented nature of contemporary literacy pedagogies. This commentary argues for a shift from evidentiary monocultures toward plural evidence ecologies.

Rationale for this paper: This commentary responds to Green and colleagues’ systematic reviews, which highlight the limited qualifying (quasi‐)experimental evidence for the effectiveness of multiliteracies and genre‐based pedagogies.

Why the new findings matter: It challenges the use of reductive experimental criteria as the only basis for evaluating complex literacy pedagogies.

Implications for educational researchers and policymakers: It is important to adopt a range of evidence, including qualitative inquiry, discourse analysis, and design‐based research, to understand how and why pedagogies contribute to literacy learning.