skip to main content skip to footer

Psychometric and Cognitive Analysis as a Bias for the Design and Revision of Quantitative Item Models AIG

Author(s):
Graf, Edith Aurora; Peterson, Stephen; Steffen, Manfred; Lawless, Rene
Publication Year:
2005
Report Number:
RR-05-25
Source:
ETS Research Report
Document Type:
Report
Page Count:
33
Subject/Key Words:
Item Model, Item Family, Automatic Item Generation (AIG), Cognitive Analysis, Mathematics Assessment

Abstract

We describe the item modeling development and evaluation process as applied to a quantitative assessment with high-stakes outcomes. In addition to expediting the item-creation process, a model-based approach may reduce pretesting costs, if the difficulty and discrimination of model-generated items may be predicted to a predefined level of accuracy. The development and evaluation of item models represents a collaborative effort among content specialists, statisticians, and cognitive scientists. A cycle for developing and revising item models that generate items with more predictable statistics is described. We review the goals of item modeling from different perspectives and recommend a method for structuring families of models that span content and generate items with more predictable psychometric parameters.

Read More