DOI: 10.21449/ijate.1888820 ISSN: 2148-7456

Strategic item selection and power in true/false tests: Insights for negative marking

Niklas Karlsson, Anders Lunander
Show PDF Cite
We compare Number-Correct Scoring (NCS) and Negative Marking (NM; +1/−1/0) for pass–fail decisions in short true/false tests. Using a framework that models optimal responding under NM and distinguishes between examinees' true and perceived success probabilities, we evaluate statistical power across heterogeneous distributions of item success probabilities. NM yields higher power when difficulties vary substantially and examinees are at least roughly well-calibrated, while NCS is more robust when miscalibration takes the form of systematic additive shifts in perceived probabilities, including both underconfidence and the more detrimental case of overconfidence for NM. Notably, overconfidence under NM can lead to overly selective rather than excessively risky responding, as examinees may omit items that would increase their true probability of passing. The results provide a decision-theoretic comparison of NCS and NM, showing how scoring-rule performance depends jointly on the scoring rule and the answering strategy implied by examinees’ perceived probabilities. Findings further inform the choice of scoring rules in educational assessments by identifying conditions under which their relative performance differs.

More from our Archive