Selecting Between Whole-Sample and Split-Sample Strategies for Exploratory and Confirmatory Factor Analyses: Guidance for Psychometric Scale Development

Development of robust psychometric scales demands careful consideration of both sample size and analytic strategy. This simulation compared whole-sample, bootstrap, and split-sample approaches for exploratory and confirmatory factor analysis (EFA/CFA) across eight realistic sample sizes. Rather than asking whether small samples are problematic, a question that is already well established, we examined when splitting an available sample compromises factor-pattern recovery and CFA stability. Using synthetic data and resampling, we assessed effects on model fit, reliability, and factor structure reproducibility. Results showed that split-sample validation became most defensible when each half independently met minimum adequacy conditions; otherwise, whole-sample analysis supplemented by resampling-based stability checks provided more consistent evidence. Based on these findings, we propose a pragmatic decision framework for applied scale development. As lower-bound guidance, our results support a minimum of 200 cases for EFA and CFA and a case-to-item ratio of at least 10:1 to ensure replicability. When these conditions are not met, whole-sample or bootstrap-based approaches are likely to provide more stable solutions. Researchers should also consider model- and design-specific requirements, which may necessitate substantially larger samples than these minimum recommendations. In conclusion, these empirically grounded recommendations provide researchers with actionable guidance for optimising sample size and analytic approach, enhancing the validity and generalisability of psychometric instruments in psychoeducational measurement.