Design and Analysis in a Cognitive Assessment

Author(s):

Yan, Duanli
Mislevy, Robert J.
Almond, Russell G.

Publication Year:

2003

Report Number:

RR-03-32

Abstract:

There is growing interest in educational assessments that coordinate substantive considerations, learning psychology, task design, and measurement models. This paper concerns an analysis of responses from an assessment of mixed-number subtraction that was created by Kikumi Tatsuoka in light of cognitive analyses of students' problem solutions. In particular, we fit a binary-skills multivariate latent class model to the data and compare results to those obtained with an item-response theory model and a modified latent class model suggested by model criticism indices. Markov chain Monte Carlo (MCMC) techniques are used to estimate the parameters in the model in a Bayesian framework that integrates information from substantive theory, expert judgment, and empirical data.