A Systematic Validation of the Jian et al. (2000) “Trust in Automated Systems Scale”

The “Trust in Automated Systems Scale” by Jian et al. is one of the most used scales in human factors research. However, there is an inconsistency in how researchers interpreted and used the scale, including a common practice of scale modifications. As it remains unclear how and to what extent modifications influence the scale’s factor structure, the current analysis addresses this question by conducting confirmatory factor analyses using publicly available datasets to evaluate and compare the factor structure of the original and modified (word “system” changed to “AI”) scales. The results support a two-factor structure, suggesting trust and distrust as separate factors, and reveal differences in the factor structure between the unmodified and AI-modified scales, suggesting consequences of modification on construct validity. Overall, this work illustrates the risks of modifications and the importance of properly validating scales to ensure the quality and reproducibility of research findings in trust measurement.