Applying Multiple Imputation to Improve Estimates from Web Surveys: An Alternative to the Pseudoweighting Method

To improve the timeliness of data products, survey researchers and practitioners have increasingly used web surveys and alike to collect information for population health research and dissemination. From the statistical inferential perspective, these data are often referred to as nonprobability samples due to the lack of a well-defined probability sampling structure, or they come from probability panel surveys yet are subject to high nonresponse and/or coverage errors. Certain statistical adjustments are therefore needed to make proper inferences using web surveys. With a high-quality reference probability survey available, one popular adjustment approach is to create pseudoweights that properly “weight” the web survey samples back to the target population underlying the reference survey in order to produce population-weighted estimates of the target of interest. When the variable of interest is collected in the web survey but not in the reference survey, the analytical question can also be framed as a missing data problem. Thus we propose to multiply impute the missing variable on the reference survey which combines the information from both data sources. We illustrate main features and performances of the multiple imputation strategy using a simulation study. The simulation results have shown that the multiple imputation analysis method is comparable to the pseudoweighting method, both of which can adequately account for the nonprobability feature of web surveys to improve the statistical inference. We also present a real data analysis based on the web-based Research and Development Survey and the interviewer-administered National Health Interview Survey. Our research shows that results from different imputation and pseudoweighting models can be compared to better understand features of the web survey data and improve the analysis.