Medical Professional Competencies Integral Assessment Method (MIDECOMP) of a high-stakes examination at the FacMed, UNAM

Abstract

Introduction: The theoretical phase of the high-stakes examination (EP-FT) evaluated the acquisition of five competencies established in the graduate profile described in the 2010 Curriculum (PE-2010) of the Faculty of Medicine (FM) UNAM, used the Medical Professional Competencies Integral Assessment Method (MIDECOMP).

Objective: Describe the method used for the design, elaboration, delivery, and analysis of the EP-FT of January of 2024 of the Medical Degree of FM.

Method: This assessment tool was designed based on a strategy that integrated the sequential analysis of the competencies, their attributes, the learning outcomes, and the taxonomic level, in a generalist medicine based practice environment. The design, development, and crossover review of the cases and the items was performed by inter and multidisciplinary committees. Therefore, for this task and the assembly of the test, we used the Automated System for Medicine Item Bank (SABERMED). For the delivery of the test we utilized the Integral System For Test Delivery (SIAEX), both of these systems were developed by the FM. The psychometric analysis established the parameters that showed the evidence of the quality of the tool and allowed us to make inferences of the results.

Results: The MIDECOMP was used to construct the EP-FT. We were able to design an assessment tool composed of 300 clinical cases with multiple-choice questions with three different options, based on the national epidemiological outlook. We integrated sequentially conceptual and logistics elements, informatic solutions, and psychometric analyses that let us achieve the quality standards to assess 1,312 candidates. We obtained an internal consistency (Cronbach’s Alpha) of 0.90, difficulty (P mean) of 0.63, and a discrimination index (Rpbis) of 0.17; these values represented adequate values.

Conclusions: The implementation of the MIDECOMP permitted the design, elaboration, delivery, and analysis of the 2024 EP-FT in a massive and efficient manner, with high quality standards, aligned with the PE-2010 of the FM, UNAM.