Comparison of Standard Setting Methods in Health Professional Education: A Systematic Review and Network Meta-Analysis

Background: Standard setting methods (SSMs) are essential for determining defensible pass marks in health professional education. While numerous SSMs exist, a comprehensive assessment comparing their outcomes is lacking. This study aimed to synthesize and compare the performance of various SSMs regarding pass rates and cut scores. Methods: A systematic review and network meta-analysis were conducted, including studies assessing standard setting in undergraduate and postgraduate health profession education from electronic databases. The primary outcome was the pass rate; and the secondary outcome was the cut score. Mixed treatment comparisons were conducted using a random-effects model, with subgroup analyses based on the field, educational level, item type, number of judges, and test items. The study quality was assessed using the MMERSQI tool, and estimates were graded using the GRADE approach. Results: A total of 61 studies were included in the systematic review and 57 in the meta-analysis. Across studies, Angoff (n = 21), conventional fixed (n = 17), Hofstee (n = 14), and modified Angoff (n = 13) were the most common. Compared to conventional fixed methods, higher pass rates were observed for the following SSMs: Angoff (OR: 3.15; 95% CI: 1.24–8.01), modified borderline group (OR: 2.44; 95% CI: 1.73–3.44), Cohen (OR: 3.02; 95% CI: 1.71–5.32), and contrasting groups (OR: 11.38; 95% CI: 5.83–22.21). Exploratory, hypothesis-generating subgroup analyses indicated higher pass rates for several methods in undergraduate settings and OSCEs. Pooled cut scores ranged from 51.05% (modified Angoff) to 73.77% (bookmark with reality check). A high variability and overlapping confidence intervals were noted. A moderator analysis showed that the number of judges (but not test items) significantly influenced the cut scores (p = 0.05). The overall strength of evidence was rated very low due to heterogeneity and imprecision. Conclusion: SSMs can influence pass rates and cut scores in health profession education. Compared with conventional fixed approaches, criterion-referenced methods such as Angoff and modified borderline group were associated with higher pass rates in several analyses; however, no method emerged as universally superior, and the overall certainty of the evidence was very low owing to heterogeneity and imprecision. These findings underscore the need for the context-specific, transparent selection of SSMs guided by defensibility, the purpose and stakes of the assessment, and the quality of available evidence, rather than using pass rates alone.