Methodology of a Practice-Based Formative Assessment with an Integrative OSCE in the Medical Degree

Abstract

Introduction: The Objective Structured Clinical Examination (OSCE) is a standardized assessment of performance focused on clinical competency. Since 2012, the Faculty of Medicine at UNAM has implemented diagnostic-formative assessments known as Advanced Academic Assessments (AAA). During the practical phase of the AAA, the OSCE has been used, undergoing adaptations and modifications over the years, ultimately evolving into the integrative OSCE.

Objective: Describe the methodological process of a practice-based formative assessment using an integrative OSCE to fourth year medical students.

Methods: The methodological process was structured into three stages: 1) Design and planning, 2) Implementation, and 3) Analysis of results. To develop the assessment method, we used a table of specifications. A blueprint of clinical cases was designed, integrated by four competencies, and each circuit consisted of four stations, each lasting 12 minutes. An analytical rubric was developed with four performance levels: excellent, sufficient, insufficient, and deficient. A total of 2,000 students were assessed across six different locations over the course of three days, with the participation of 124 assessors and 124 standardized patients. At the end of each assessment shift, feedback was provided to groups of up to 35 students.

Results: We calculated a discrimination index of <0.0001 between high and low scores across the 91 items, along with a Cronbach’s Alpha of 0.95. To assess internal consistency, we conducted a principal component analysis, which identified four factors explaining 68.96% of the variance after 17 iterations.

Conclusions: The methodology described, along with the technological resources and statistical analyses performed, enabled the development of assessment instruments and tools that provide students with key information to adjust their learning activities effectively.