A Multivariate Stopping Rule for Survey Data Collection: Empirical Evaluation from a Panel Study

Surveys are experiencing declining response rates. With more and more effort expended to combat these declining response rates, the cost of large-scale surveys has continued to rise. Stopping rules are one of the interventions used to improve the efficiency of data collection. However, most previously proposed stopping rules have focused only on the quality of a single estimate or assumed sufficient funds for nonresponse follow-up. In multipurpose surveys, there may be data quality objectives that must be met for multiple estimates with a constraint on costs. In this paper, we introduce a multivariate stopping rule that aims to maximize the tradeoff between the cost of data collection and the quality of multiple estimates. The multivariate stopping rule uses predicted costs and mean squared errors of different estimates to evaluate alternative sets of cases for stopping. The proposed stopping rule is illustrated using data from the Health and Retirement Study. In the multivariate stopping rule, each estimate receives a prespecified weight that determines the relative importance in the design. We found that assigning a relatively larger estimate-level weight to a survey variable did not necessarily result in better data quality for that variable. One plausible reason is that the prediction quality varies across variables.