skip to main content skip to footer

Integrating Cognitive and Psychometric Models to Measure Document Literacy IRT NAEP

Sheehan, Kathleen M.; Mislevy, Robert J.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Office of Naval Research, Bayesian Statistics, Cognitive Processes, Item Response Theory (IRT), Linear Logistic Test Model, Literacy, Models, National Assessment of Educational Progress (NAEP), Psychometrics, Survey of Young Adult Literacy


The Survey of Young Adult Literacy conducted in 1985 by the National Assessment of Educational Progress included sixty-three items that elicited skills in acquiring and using information from written documents. These items were analyzed in two distinct ways: (1) with an item response theory (IRT) model, which characterized items' difficulties and respondents' proficiencies as revealed simply by tendencies toward correct response, and (2) a qualitative cognitive model, which characterized items in terms of the processing tasks they required. This paper demonstrates how a generalization of Fischer and Scheiblechner's Linear Logistic Test Model can be used to integrate information from the cognitive analysis into the IRT analysis. (45pp.)

Read More