Item analysis of multiple choice question tests in undergraduate health programs of Universidad Mayor

Abstract

Introduction: Multiple-choice question tests (MCTs) are widely used to assess undergraduate students’ learning at the Health Sciences Schools (HSS) of our Universidad Mayor. In the last decade, the process of checking MCQs has been accomplished by optical mark recognition. This fact allowed this research study to obtain a diagnostic vision of the quality of this process.
Objective: To analyze the difficulty, discrimination, reliability and distractors of MCTs applied in seven HSS of Universidad Mayor during the 2013-2017 period.
Method: For this quantitative, descriptive, non-experimental, cross-sectional, and retrospective study, of the population of tests under study, i.e. 2640 MCTs, 337 were randomly selected by stratified probabilistic sampling with systematic selection. Psychometric indicators were estimated from the framework of Classical Test Theory. ANOVA tests were used to compare between programs.
Results: For item difficulty, item discrimination, and reliability coefficient, the respective means were: 68%, 0.23 and 0.50. Only for item difficulty and discrimination were there significant differences among the participating HSS. Regarding distractors, on average, 1,51 distractors were found to be functional (1,52 for items with four choices and 1,49 for items with five choices).
Conclusions: These results reveal that there is considerable room for improvement in the application of MCTs in the assessment of students‘ learning in our HSS. An in-depth reflection between faculty and university authorities should be carried out to ensure the future validity inferences and quality of MCTs.