Distractors in multiple-choice questions for medical students: a descriptive analysis in a high-stakes summative exam

Abstract

Introduction: Multiple-choice question (MCQ) exams are the most frequently used tool for knowledge assessment of medical students. It has been shown that three options in each item are enough; however, many tests in our country still use questions with five options. The study of non-functional distractors (NFD) is needed in order to improve the quality of our assessment instruments.

Objective: To identify NFD and their behavior in a high-stakes summative exam in medical students.

Method: A psychometric analysis was performed of the Theoretical Professional Exam at UNAM Faculty of Medicine, the 2008 version. Difficulty, discrimination and point-biserial correlation were calculated for each item and its four distractors. The frequency of items with zero to four NFD was obtained, and the differences of NFD per item and their psychometric characteristics were assessed. The psychometric behavior of the whole exam was compared to a version where items with four NFD were removed.

Results: The test had 420 MCQ items with five options, and was answered by 882 students. The instrument had a Cronbach’s alpha of 0.93. Of the 1 680 distractors evaluated, only 788 (46.9%) were functional. More than two-thirds of the total of items had two or more NFD. There was an average of 2.12 ± 0.99 NFD per item, the majority of which were chosen by less than 5% of the test takers. With higher quantity of NFD, the items’ difficulty index was higher and the discrimination index was lower.

Conclusions: Items with two functional distractors were the majority in the exam. Two plausible distractors are a reasonable alternative for item development, maintaining or improving the reliability and psychometric profile of these types of tests.