Reinterpreting I 2 Thresholds: Toward Context‐Specific Heterogeneity Assessment in Evidence Synthesis
Arturo J. Martí‐Carvajal, David L. StreinerABSTRACT
Background
The Cochrane Handbook's I 2 categorization system (0%–25% “low”, 25%–75% “moderate”, ≥ 50% “high” heterogeneity) defines the standard approach to interpreting heterogeneity in meta‐analysis and informs thousands of systematic reviews each year. Despite its widespread use, its logical coherence and its role as an analytical decision tool have received limited formal examination.
Objective
To examine whether the Cochrane Handbook's I 2 categorization system includes overlapping category definitions that create ambiguous classification, and to propose context‐specific frameworks that preserve I 2 as a decision tool for heterogeneity exploration.
Methods
We conducted a structured genealogical analysis of key methodological sources related to the Cochrane Handbook's I 2 categorization. We examined whether individual I 2 values can satisfy more than one category. We analysed the categorization using principles from formal logic, philosophy of science, and statistical theory. We traced the development of I 2 interpretation from its original formulation to current Cochrane guidance. We developed context‐specific frameworks based on patient‐important outcome categories.
Results
The I 2 categorization system includes overlapping definitions in which identical values satisfy more than one category (e.g., I 2 = 50% corresponds to both “moderate” [25%–75%] and “high” [≥ 50%]). This structure assigns single values to multiple categories and departs from principles that require consistent and mutually exclusive classification. The primary literature provides limited explicit theoretical or empirical justification for the selected thresholds. The current approach uses I 2 as an interpretive endpoint rather than as a decision tool for heterogeneity exploration. These features reduce interpretive clarity and obscure the role of clinical context in heterogeneity assessment.
Conclusions
The current I 2 categorization system introduces ambiguity and leads to inconsistent analytical decisions across outcome contexts. Evidence synthesis requires context‐specific frameworks in which interpretation reflects outcome type and expected variability. We propose the PIOHA framework to align heterogeneity assessment with clinical relevance while preserving I 2 as an analytical decision tool.
Clinical Relevance
Systematic reviews inform clinical guidelines and patient care. Context‐specific heterogeneity assessment supports analytical decisions that reflect the clinical importance and expected variability of patient‐important outcomes.