Quality of Problems in PBL. Validity evidence of an instrument

Abstract

Introduction: In Problem Based Learning (PBL), the quality of the problems is essential for the acquisition and integration of knowledge. Nevertheless, there is a lack of instruments with validity evidence to measure it.

Objective: To develop a questionnaire to evaluate the quality of the problems of the PBL, obtaining its evidence of validity.

Method: The questionnaire was developed upon revising literature. Validity evidences were obtained according to the Standards for Educational and Psychological Testing, specifically sources of content validity evidence, response process, internal structure and assessment consequences.
The questionnaire was submitted to evaluation by a jury and it was calculated according to the level of agreement related to the Fleiss’ Kappa Test. In order to know the psychometric characteristics of the questionnaire, discrimination analysis of items was performed using Student’s t test; looking to identify the internal structure of the test, an exploratory factorial analysis was performed and the Cronbach’s Alpha was calculated as statistic of internal consistency.

Results: The obtained Kappa index for the 24 items was of 0.45 to 1. The Student’s t test got values of bilateral significance of 0.000. The exploratory factorial analysis with a Varimax rotation extracted six factors that showed an explained variance of 83.08%. The instrument final version was constituted by 20 items with a total Cronbach’s Alpha in the scale of 0.97.

Conclusion: This questionnaire has enough validity evidence as to be used to assess the quality of the problems in PBL. Besides, the six factors described amplify the measuring extent of the existing instruments and allowing us to detect the strengths and weaknesses and provide suggestions from the perspective of learners and professors.