Spurious Reliability Increase?: The Number of Response Options in the Likert-Type Scale Influences Only Internal Consistency, Not Criterion Validity

The existing research on the number of options in the Likert-type response format has focused primarily on reliability and descriptive statistics, often overlooking validity or examining it with limitations. This study addressed this gap through a within-subject experiment ( N = 846, 69% women), manipulating response options (two, six, and 10) in two Likert-type scales: the Height Inventory and the autonomy subscale of the Basic Needs Satisfaction in General Scale. Two-point variants significantly differed in means and reliability compared to six- and 10-point versions. While the magnitudes of differences were small in Height Inventory, it did not follow for the autonomy subscale. On the other hand, validity (criterion, measurement model, and trait criterion validity) remained unaffected. Thus, the increased reliability may be spurious, stemming from systematic but construct-irrelevant variance related to response format (i.e., method variance). These findings suggest that response formats with fewer options can be viable, particularly in scales with more items. Future research should explore differences in cognitive processes across response formats.