skip to main content skip to footer

Integrating Cognitive Views Into Psychometric Models for Reading Comprehension Assessment IRT

Author(s):
Rahman, Taslima; Mislevy, Robert J.
Publication Year:
2017
Report Number:
RR-17-35
Source:
ETS Research Report
Document Type:
Report
Page Count:
18
Subject/Key Words:
Psychometric Models, Item Response Theory (IRT), Comprehension Testing, Linear Logistic Test Model, Domain-Referenced Testing, Prior Learning, Reading Comprehension, Self Report, Interest (Psychology), Proficiency Measurement

Abstract

To demonstrate how methodologies for assessing reading comprehension can grow out of views of the construct suggested in the reading research literature, we constructed tasks and carried out psychometric analyses that were framed in accordance with 2 leading reading models. In estimating item difficulty and subsequently, examinee proficiency, an item response theory (IRT) model called the linear logistic test model was extended to incorporate reader as well as task attributes as covariates. A novel aspect of this modeling was reader effects—interest and prior knowledge—specific to text passages that the examinees read in the assessment. In the demonstration, the theory-motivated task and reader attributes were found to be significantly related to item difficulty. In particular, examinees’ comprehension proficiency estimates positively affected within-person effects concerning the reader’s familiarity and interest in a passage. This study suggests that it is both feasible and informative to incorporate variables for various comprehension components into the psychometric analysis.

Read More