Relationships Among Multiple-Choice Reasoning Items and a Constructed-Response Generating-Explanations Task

Author(s):
Enright, Mary K.; Rock, Donald A.; Bennett, Randy Elliot.
Publication Year:
1998
Report Number:
RR-98-41
GREB-94-15P
Source:
Document Type:
Subject/Key Words:
Multiple-choice items constructed response computer-assisted testing computer-adaptive test (CAT)

Abstract

The potential benefits of computer-based testing include the ability to present a wider variety of item formats and to tailor tests for specific purposes. In this study we examined the relationships among revised reasoning items that had more varied formats than the traditional items, a constructed-response generating-explanations task, and the current GRE General Test. In addition we examined the relationship of these measures to other indicators of achievement. A computer-based test of the revised reasoning items and the generating-explanations task was administered to a sample of examinees who previously had taken a paper-and-pencil version of the GRE General Test. The revised reasoning item types had acceptable psychometric characteristics and were correlated highly with the current logical reasoning items as well as with items on the GRE verbal measure. The generating-explanations task was only marginally related to the GRE measures and to the revised reasoning items and was more strongly related to ideational fluency measures. Factor analytic results suggest that if the revised reasoning items were added to the analytical measure, the correlation between the verbal measure and the analytical measure would increase. A better fitting factor model resulted when some of the current and the revised reasoning items were included in the verbal measure. The relationship of the generating-explanations task and the analytical measure to other indicators of achievement varied for type of achievement and for broad major undergraduate fields of study. The implications of these results for understanding the nature of the skills assessed by different tasks and formats, and for long-term modifications in the GRE General Test, are discussed.

Read More