Graphical Models and Computerized Adaptive Testing

Author(s):
Almond, Russell G.; Mislevy, Robert J.
Publication Year:
1998
Report Number:
RR-98-04
TOEFL-TR-14
Source:
Document Type:
Subject/Key Words:
Item response theory language tests computer-assisted testing graphical modeling (statistics)

Abstract

This paper synthesizes ideas from the fields of graphical modeling and educational testing, particularly item response theory (IRT) applied to computerized adaptive testing (CAT). Graphical Modeling can offer IRT a language for describing multifaceted skills and knowledge, and disentangling evidence from complex performances. IRT-CAT can offer graphical modelers several ways of treating sources of variability other than including more variables in the model. In particular, variables can enter into the modeling process at five levels: (a) in validity studies (but not in the ordinarily used model), (b) in task construction (in particular, in defining link parameters), (c) in test or model assembly (blocking and randomization constraints in selecting tasks or other model pieces, (d) in response characterization (that is, as part of task models that characterize a response), or (e) in the main (student) model. The GRE® (Graduate Record Examinations) are used to illustrate ideas in the context of IRT-CAT, and extensions are discussed in the context of language proficiency testing.