skip to main content skip to footer

A Generative Approach to the Development of Hidden-Figure Items IRT

Bejar, Isaac I.; Yocom, Peter
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Office of Naval Research, Cognitive Psychology, Computer Assisted Testing, Hidden Figures Tests, Item Response Theory (IRT), Psychometric Models, Spatial Ability, Test Construction


This report explores an approach to item development and psychometric modeling which explicitly incorporates knowledge about the mental models used by examinees in the solution of items into a psychometric model that characterize performances on a test, as well as incorporating that knowledge into the item development process. The paper focuses on the hidden-figure item type. Although there is extensive literature on the correlates of performance for this type of item, little is known about the mental models that may explain performance on the item. The approach taken in this paper is to search for a complexity dimension that accounts for the difficulty of hidden figures. Although several complexity dimensions can be postulated, we chose one inspired by artificial intelligence research on vision. A computer-based system was developed to analyze as well as generate items based on this framework. To empirically determine the validity of the chosen framework two experiments were conducted. The results suggest that this approach to psychometric modeling is viable. The practical and theoretical implications of the research are discussed. (48pp.)

Read More