skip to main content skip to footer

Generative Response Modeling: Leveraging the Computer as a Test Delivery Medium CAT

Bejar, Isaac I.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Computer Assisted Testing, Constructed Responses, Monte Carlo Methods, Test Administration, Test Items


Generative response modeling is an approach to test development and response modeling that calls for the creation of items in such a way that the parameters of the items on some response model can be anticipated through knowledge of the psychological processes and knowledge required to respond to the item. That is, the computer would not merely retrieve an item from a database, as is the case in adaptive testing, but would compose it, or assist in doing so, according to desired specifications. This approach to assessment has implications for both the economics and validity of computer administered tests. To illustrate the concept, a system for measuring writing skills will be outlined where the examinee is expected to rewrite sentences, rather than just recognize errors in a sentence, using a multiple choice format. The possibility of estimating the psychometric parameters of items based on a psychological analysis of the response process will then be examined and shown to be feasible. Such estimates are less precise than estimates based on large samples of test takers. A Monte Carlo study is presented to investigate the possibility of compensating for the imprecision when estimating ability or proficiency. The paper concludes that a generative approach is feasible, and can be a mechanism for taking advantage of the considerable investment required for computer-based testing. (43pp.)

Read More