Process of validation of an instrument to evaluate the critical reading in medical research papers

Abstract

Introduction: Critical appraisal is a learning tool that should be familiar to professors and students. Educational strategies should focus on the student’s active learning. Valid and reliable tools should be constructed in order to measure progress in this field. 

Objective: To construct, validate and determine the consistency of a tool for evaluating critical reading.

Method: Tool: a review was conducted on published articles on the construction of evaluation tools, surveys, cases and controls, diagnostic tests, randomised clinical trials, cohorts, follow-up and meta-analyses. Eight articles on each design were finally selected. They were summarised and statements and items were developed, by examining the document validity, consistency, design, statistics, results, and the discussion. The items were based on critical reading indicators: interpretation, judgment, and proposals. Validation was obtained using the Delphi technique in 2 rounds that included six Doctors or Masters in Science or Education, previously experienced in the use of these tools. It was balanced with 96 items, 16 per design and 32 per indicator, with 48 ‘‘true or false’’ answers. Grading was blind and using a software program to avoid capture errors. Medians were obtained for analysis. A pilot test was applied to two groups of student interns (neophytes G1 and intervened G2) and consistency was established with the Kuder-Richardson formula. Extremes were compared with the Mann-Whitney U test. The level of randomness was also determined.

Results: A consistency of 0.79 was obtained, although concurrent validity was inadequate (Spearman = 0.31). There were overall differences in the group median values, G1 vs. G2 (11 vs. 29) as well as in random answers (70% vs.8%), and in comparison of extremes (P < .0019). 

Conclusions: It is necessary to construct and update tools for measuring the development of this complex and transcendental ability required for the critical appraisal of published medical information.