Graphical Modeling: A New Response Type for Measuring the Qualitative Component of Mathematical Reasoning
- Bennett, Randy Elliot; Morley, Mary; Quardt, Dennis; Rock, Donald A.
- Publication Year:
- Report Number:
- RR-99-21, GREB-97-01P
- Document Type:
- Subject/Key Words:
- Computer-based testing automated scoring constructed-response testing
This study investigated the psychometric functioning of Graphical Modeling (GM), a new computer-delivered response type for assessing mathematical reasoning that asks candidates to respond to a problem situation by creating a graphical representation. GM problems can be like the single-best-answer items currently found on the GRE® General Test, or they can be more loosely defined, allowing for multiple correct responses. Two GM tests differing from one another in the manipulation of specific item features were randomly spiraled among study participants. Analyses were performed relating to internal consistency reliability, relations with external criteria, features that contribute to item difficulty, adverse gender impact, and examinee perceptions. Results showed that GM scores were very reliable and moderately related to the General Test's quantitative section, suggesting that the introduction of GM items on the General Test might help broaden the GRE quantitative construct. In exploratory analyses of difficulty, one of three manipulated item features, problem structure, had a significant effect. Our impact analyses detected no significant gender differences independent of those associated with the GRE quantitative section. Finally, while more participants preferred regular multiple-choice graphical reasoning questions to GM items, more also thought GM was the fairer indicator of their ability to undertake graduate study.