Item Analysis of Short-answer and Essay-type Questions in Microbiology Subject: A Psychometric Perspective
Gurjeet Singh, Raksha Singh, Nilakantan Ananthakrishnan
A
BSTRACT
Objectives:
While multiple-choice questions (MCQs) are widely used in assessments due to objectivity and ease of scoring, short-answer questions (SAQs) and essay questions (EQs) continue to play a critical role in evaluating higher-order cognitive domains. This study explores the application of item analysis to SAQs and EQs in medical education.
Materials and Methods:
A cross-sectional, psychometric analysis was conducted using data from 120 second MBBS students. Ten SAQs and five EQs from a formative assessment were evaluated using difficulty index, discrimination index, inter-rater reliability (Cohen’s Kappa), and internal consistency (Cronbach’s alpha). Two independent raters scored the items based on structured rubrics. The data were analyzed in SPSS v22.
Results:
The SAQs had a mean difficulty index of 62.5 and a mean discrimination index of 0.31. The discrimination index of the EQs was a little lower at 0.26. Inter-rater reliability was higher with SAQs (κ =0.71) than EQs (κ =0.62). Cronbach’s alpha was 0.78 for SAQs and 0.70 for EQs.
Conclusion:
Item analysis could be used in both SAQs and EQs to evaluate and improve test quality. It calls for using rubric-based scoring and training of raters to uphold reliability. These findings support the need for item analysis in tests other than MCQs in competency-based medical education.