Measuring racial attitudes pre-treatment in survey experiments

Abstract

Experimental researchers often need to assess the moderating effect of racial attitudes. However, it is unclear whether the racial attitude battery deployed may bias treatment effects. Less subtle batteries could bias treatment effects toward zero if measuring racial attitudes primes these attitudes for subjects assigned to all conditions, including the control condition. We assess whether bias is introduced across four conditions, pre-treatment: not measuring racial attitudes, using stereotype endorsement measures, using the American National Election Study racial resentment battery, or using the simplified Cooperative Election Study racial resentment battery. We do not uncover widespread instances of treatment bias. However, we do identify bias when the stereotype endorsement measures are used pre-treatment. We conclude with recommendations for researchers.