Response‐Scale Reliability for Count Measurements: Distributional Generalizability Theory for Hurdle Models, With an Application to Twenty20 Cricket

ABSTRACT

Generalizability theory quantifies measurement reliability through variance components, but its classical intraclass correlation coefficient assumes Gaussian measurements. For count outcomes with excess zeros—common in behavioural and performance measurement—the reliability computed on the model's link scale systematically overstates the reliability of the response that practitioners actually interpret, leading decision studies to recommend fewer observations than dependable measurement requires. This paper develops a generalizability framework for hurdle count models (zero‐truncated Poisson and negative binomial), deriving a decomposition of observation‐scale variance that separates engagement‐process and intensity‐process contributions to both signal and noise. The response‐scale intraclass correlation is recovered from this decomposition via a posterior‐simulation estimator. The method is illustrated on Big Bash League data, measuring how reliably batters can be ranked by the number of sixes they hit per innings, and the estimator is validated both in simulation—at the population level and in finite samples—and empirically, through a split‐half test–retest that recovers the predicted reliability. Separating the engagement and intensity processes also exposes what the link‐scale coefficient hides: because it is taken from the intensity submodel alone, it can misstate not only the level of reliability but which process carries the reliable signal. The methods are implemented in the open‐source dgt package.