The Convergent Validity of Expert System Scores for Complex Constructed-Response Quantitative Items

Author(s):
Bennett, Randy E.; Sebrechts, Marc M.; Rock, Donald A.
Publication Year:
1991
Report Number:
RR-91-12
GREB-88-07bP (1991)
Source:
Applied Psychological Measurement, 1991, 15(3), 227-239.
Document Type:
Subject/Key Words:
Computer software constructed responses evaluation methods expert systems scoring validity studies

Abstract

This study investigated the convergent validity of expert-system scores for four mathematical constructed-response item formats. A five-factor model was posed comprised of four constructed-response format factors and a GRE General Test quantitative factor. Confirmatory factor analysis was used to test the fit of this model and to compare it with several alternatives. The five-factor model fit well, although a solution comprised of two highly correlated dimensions--GRE-quantitative and constructed-response--represented the data almost as well. These results extend the meaning of the expert system's constructed-response scores by relating them to a well-established quantitative measure and by indicating that they signify the same underlying proficiency across item formats.

Read More